Skip to content

Python 3.9: Code folding #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
bnavigator opened this issue Mar 31, 2021 · 1 comment · Fixed by #9
Closed

Python 3.9: Code folding #8

bnavigator opened this issue Mar 31, 2021 · 1 comment · Fixed by #9

Comments

@bnavigator
Copy link
Contributor

With parso 0.8.1:

[   28s] ============================= test session starts ==============================
[   28s] platform linux -- Python 3.9.2, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3.9
[   28s] cachedir: .pytest_cache
[   28s] rootdir: /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2
[   28s] plugins: flaky-3.7.0
[   28s] collecting ... collected 119 items
...
[   77s] =================================== FAILURES ===================================
[   77s] _________________________________ test_folding _________________________________
[   77s] 
[   77s] workspace = <pyls.workspace.Workspace object at 0x7fcd982f7310>
[   77s] 
[   77s]     def test_folding(workspace):
[   77s]         doc = Document(DOC_URI, workspace, DOC)
[   77s]         ranges = pyls_folding_range(doc)
[   77s]         expected = [{'startLine': 1, 'endLine': 6},
[   77s]                     {'startLine': 2, 'endLine': 3},
[   77s]                     {'startLine': 5, 'endLine': 6},
[   77s]                     {'startLine': 8, 'endLine': 11},
[   77s]                     {'startLine': 12, 'endLine': 20},
[   77s]                     {'startLine': 13, 'endLine': 14},
[   77s]                     {'startLine': 15, 'endLine': 16},
[   77s]                     {'startLine': 17, 'endLine': 18},
[   77s]                     {'startLine': 19, 'endLine': 20},
[   77s]                     {'startLine': 22, 'endLine': 35},
[   77s]                     {'startLine': 23, 'endLine': 35},
[   77s]                     {'startLine': 24, 'endLine': 25},
[   77s]                     {'startLine': 27, 'endLine': 29},
[   77s]                     {'startLine': 28, 'endLine': 29},
[   77s]                     {'startLine': 30, 'endLine': 31},
[   77s]                     {'startLine': 32, 'endLine': 34},
[   77s]                     {'startLine': 33, 'endLine': 34},
[   77s]                     {'startLine': 38, 'endLine': 39},
[   77s]                     {'startLine': 41, 'endLine': 43},
[   77s]                     {'startLine': 42, 'endLine': 43},
[   77s]                     {'startLine': 45, 'endLine': 54},
[   77s]                     {'startLine': 47, 'endLine': 51},
[   77s]                     {'startLine': 49, 'endLine': 51},
[   77s]                     {'startLine': 50, 'endLine': 51},
[   77s]                     {'startLine': 52, 'endLine': 54},
[   77s]                     {'startLine': 53, 'endLine': 54},
[   77s]                     {'startLine': 56, 'endLine': 57},
[   77s]                     {'startLine': 59, 'endLine': 65},
[   77s]                     {'startLine': 60, 'endLine': 61},
[   77s]                     {'startLine': 62, 'endLine': 63},
[   77s]                     {'startLine': 64, 'endLine': 65},
[   77s]                     {'startLine': 67, 'endLine': 68}]
[   77s] >       assert ranges == expected
[   77s] E       AssertionError: assert [{'endLine': 6, 'startLine': 1},\n {'endLine': 3, 'startLine': 2},\n {'endLine': 6, 'startLine': 5},\n {'endLine': 11, 'startLine': 8},\n {'endLine': 10, 'startLine': 9},\n {'endLine': 20, 'startLine': 12},\n {'endLine': 14, 'startLine': 13},\n {'endLine': 16, 'startLine': 15},\n {'endLine': 18, 'startLine': 17},\n {'endLine': 20, 'startLine': 19},\n {'endLine': 35, 'startLine': 22},\n {'endLine': 35, 'startLine': 23},\n {'endLine': 25, 'startLine': 24},\n {'endLine': 29, 'startLine': 27},\n {'endLine': 29, 'startLine': 28},\n {'endLine': 31, 'startLine': 30},\n {'endLine': 34, 'startLine': 32},\n {'endLine': 34, 'startLine': 33},\n {'endLine': 39, 'startLine': 38},\n {'endLine': 43, 'startLine': 41},\n {'endLine': 43, 'startLine': 42},\n {'endLine': 54, 'startLine': 45},\n {'endLine': 51, 'startLine': 47},\n {'endLine': 51, 'startLine': 49},\n {'endLine': 51, 'startLine': 50},\n {'endLine': 54, 'startLine': 52},\n {'endLine': 54, 'startLine': 53},\n {'endLine': 57, 'startLine': 56},\n {'endLine': 65, 'startLine': 59},\n {'endLine': 61, 'startLine': 60},\n {'endLine': 63, 'startLine': 62},\n {'endLine': 65, 'startLine': 64},\n {'endLine': 68, 'startLine': 67}] == [{'endLine': 6, 'startLine': 1},\n {'endLine': 3, 'startLine': 2},\n {'endLine': 6, 'startLine': 5},\n {'endLine': 11, 'startLine': 8},\n {'endLine': 20, 'startLine': 12},\n {'endLine': 14, 'startLine': 13},\n {'endLine': 16, 'startLine': 15},\n {'endLine': 18, 'startLine': 17},\n {'endLine': 20, 'startLine': 19},\n {'endLine': 35, 'startLine': 22},\n {'endLine': 35, 'startLine': 23},\n {'endLine': 25, 'startLine': 24},\n {'endLine': 29, 'startLine': 27},\n {'endLine': 29, 'startLine': 28},\n {'endLine': 31, 'startLine': 30},\n {'endLine': 34, 'startLine': 32},\n {'endLine': 34, 'startLine': 33},\n {'endLine': 39, 'startLine': 38},\n {'endLine': 43, 'startLine': 41},\n {'endLine': 43, 'startLine': 42},\n {'endLine': 54, 'startLine': 45},\n {'endLine': 51, 'startLine': 47},\n {'endLine': 51, 'startLine': 49},\n {'endLine': 51, 'startLine': 50},\n {'endLine': 54, 'startLine': 52},\n {'endLine': 54, 'startLine': 53},\n {'endLine': 57, 'startLine': 56},\n {'endLine': 65, 'startLine': 59},\n {'endLine': 61, 'startLine': 60},\n {'endLine': 63, 'startLine': 62},\n {'endLine': 65, 'startLine': 64},\n {'endLine': 68, 'startLine': 67}]
[   77s] E         At index 4 diff: {'startLine': 9, 'endLine': 10} != {'startLine': 12, 'endLine': 20}
[   77s] E         Left contains one more item: {'endLine': 68, 'startLine': 67}
[   77s] E         Full diff:
[   77s] E           [
[   77s] E            {'endLine': 6,
[   77s] E             'startLine': 1},
[   77s] E            {'endLine': 3,
[   77s] E             'startLine': 2},
[   77s] E            {'endLine': 6,
[   77s] E             'startLine': 5},
[   77s] E            {'endLine': 11,
[   77s] E             'startLine': 8},
[   77s] E         +  {'endLine': 10,
[   77s] E         +   'startLine': 9},
[   77s] E            {'endLine': 20,
[   77s] E             'startLine': 12},
[   77s] E            {'endLine': 14,
[   77s] E             'startLine': 13},
[   77s] E            {'endLine': 16,
[   77s] E             'startLine': 15},
[   77s] E            {'endLine': 18,
[   77s] E             'startLine': 17},
[   77s] E            {'endLine': 20,
[   77s] E             'startLine': 19},
[   77s] E            {'endLine': 35,
[   77s] E             'startLine': 22},
[   77s] E            {'endLine': 35,
[   77s] E             'startLine': 23},
[   77s] E            {'endLine': 25,
[   77s] E             'startLine': 24},
[   77s] E            {'endLine': 29,
[   77s] E             'startLine': 27},
[   77s] E            {'endLine': 29,
[   77s] E             'startLine': 28},
[   77s] E            {'endLine': 31,
[   77s] E             'startLine': 30},
[   77s] E            {'endLine': 34,
[   77s] E             'startLine': 32},
[   77s] E            {'endLine': 34,
[   77s] E             'startLine': 33},
[   77s] E            {'endLine': 39,
[   77s] E             'startLine': 38},
[   77s] E            {'endLine': 43,
[   77s] E             'startLine': 41},
[   77s] E            {'endLine': 43,
[   77s] E             'startLine': 42},
[   77s] E            {'endLine': 54,
[   77s] E             'startLine': 45},
[   77s] E            {'endLine': 51,
[   77s] E             'startLine': 47},
[   77s] E            {'endLine': 51,
[   77s] E             'startLine': 49},
[   77s] E            {'endLine': 51,
[   77s] E             'startLine': 50},
[   77s] E            {'endLine': 54,
[   77s] E             'startLine': 52},
[   77s] E            {'endLine': 54,
[   77s] E             'startLine': 53},
[   77s] E            {'endLine': 57,
[   77s] E             'startLine': 56},
[   77s] E            {'endLine': 65,
[   77s] E             'startLine': 59},
[   77s] E            {'endLine': 61,
[   77s] E             'startLine': 60},
[   77s] E            {'endLine': 63,
[   77s] E             'startLine': 62},
[   77s] E            {'endLine': 65,
[   77s] E             'startLine': 64},
[   77s] E            {'endLine': 68,
[   77s] E             'startLine': 67},
[   77s] E           ]
[   77s] 
...
[   77s] FAILED test/plugins/test_folding.py::test_folding - AssertionError: assert [{...
[   77s] ============ 1 failed, 108 passed, 10 skipped, 8 warnings in 49.44s ============

https://github.com/python-ls/python-ls/blob/18338751a420ffa88f533930d837f2e9ec1c5cde/test/plugins/test_folding.py#L11-L31
@bnavigator
Copy link
Contributor Author

bnavigator commented Apr 1, 2021 •
edited
Loading

There are some extra nodes in the parsed tree. With Python 3.8, the parent of the argument list is a Decorator, with Python3.9 it is a PythonNode:

build@skylab:~/rpmbuild/BUILD/python-language-server-0.36.2> pytest-3.8 --trace -k test_folding 
================================================= test session starts ==================================================
platform linux -- Python 3.8.8, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2
plugins: flaky-3.7.0
collected 119 items / 117 deselected / 2 selected                                                                      

test/plugins/test_folding.py 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB runcall (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/test/plugins/test_folding.py(115)test_folding()
-> doc = Document(DOC_URI, workspace, DOC)
(Pdb) b pyls/plugins/folding.py:194, node.start_pos[0]==10  
Breakpoint 1 at /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py:194
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue (IO-capturing resumed) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py(194)__compute_folding_ranges()
-> start_line, end_line, stack = __compute_start_end_lines(
(Pdb) b pyls/plugins/folding.py:150
Breakpoint 2 at /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py:150
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue (IO-capturing resumed) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py(194)__compute_folding_ranges()
-> start_line, end_line, stack = __compute_start_end_lines(
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue (IO-capturing resumed) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py(150)__compute_start_end_lines()
-> last_leaf = node.get_last_leaf()
(Pdb) pp node
PythonNode(arglist, [<Name: param1@10,4>, <Operator: ,>, <Name: param2@11,4>])
(Pdb) pp node.parent
<Decorator: @decorator(     param1,     param2 )@9,0>
(Pdb) pp node.parent.parent
PythonNode(decorated, [<Decorator: @decorator(     param1,     param2 )@9,0>, <Function: decorated_func@13-22>])
(Pdb) cl
Clear all breaks? y
Deleted breakpoint 1 at /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py:194
Deleted breakpoint 2 at /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py:150
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue (IO-capturing resumed) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB runcall (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/test/plugins/test_folding.py(153)test_folding_syntax_error()
-> doc = Document(DOC_URI, workspace, SYNTAX_ERR)
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue (IO-capturing resumed) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
.                                                                                  [100%]

===================================== 2 passed, 117 deselected in 84.37s (0:01:24) =====================================


abuild@skylab:~/rpmbuild/BUILD/python-language-server-0.36.2> pytest-3.9 --trace -k test_folding                        ================================================= test session starts ==================================================
platform linux -- Python 3.9.2, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2
plugins: flaky-3.7.0
collected 119 items / 117 deselected / 2 selected                                                                      

test/plugins/test_folding.py 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB runcall (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/test/plugins/test_folding.py(115)test_folding()
-> doc = Document(DOC_URI, workspace, DOC)
(Pdb) b pyls/plugins/folding.py:194, node.start_pos[0]==10
Breakpoint 1 at /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py:194
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue (IO-capturing resumed) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py(194)__compute_folding_ranges()
-> start_line, end_line, stack = __compute_start_end_lines(
(Pdb) b pyls/plugins/folding.py:150
Breakpoint 2 at /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py:150
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue (IO-capturing resumed) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py(194)__compute_folding_ranges()
-> start_line, end_line, stack = __compute_start_end_lines(
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue (IO-capturing resumed) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py(150)__compute_start_end_lines()
-> last_leaf = node.get_last_leaf()
(Pdb) pp node
PythonNode(arglist, [<Name: param1@10,4>, <Operator: ,>, <Name: param2@11,4>])
(Pdb) pp node.parent
PythonNode(trailer, [<Operator: (>, PythonNode(arglist, [<Name: param1@10,4>, <Operator: ,>, <Name: param2@11,4>]), <Operator: )>])
(Pdb) pp node.parent.parent
PythonNode(atom_expr, [<Name: decorator@9,1>, PythonNode(trailer, [<Operator: (>, PythonNode(arglist, [<Name: param1@10,4>, <Operator: ,>, <Name: param2@11,4>]), <Operator: )>])])
(Pdb) pp node.parent.parent.parent
<Decorator: @decorator(     param1,     param2 )@9,0>
(Pdb) pp node.parent.parent.parent.parent
PythonNode(decorated, [<Decorator: @decorator(     param1,     param2 )@9,0>, <Function: decorated_func@13-22>])
(Pdb) cl
Clear all breaks? y
Deleted breakpoint 1 at /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py:194
Deleted breakpoint 2 at /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/pyls/plugins/folding.py:150
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue (IO-capturing resumed) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
F
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB runcall (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILD/python-language-server-0.36.2/test/plugins/test_folding.py(153)test_folding_syntax_error()
-> doc = Document(DOC_URI, workspace, SYNTAX_ERR)
(Pdb) c

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB continue (IO-capturing resumed) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
.                                                                                  [100%]

======================================================= FAILURES =======================================================
_____________________________________________________ test_folding _____________________________________________________

workspace = <pyls.workspace.Workspace object at 0x7fc49e16af40>

    def test_folding(workspace):
        doc = Document(DOC_URI, workspace, DOC)
        ranges = pyls_folding_range(doc)
        expected = [{'startLine': 1, 'endLine': 6},
                    {'startLine': 2, 'endLine': 3},
                    {'startLine': 5, 'endLine': 6},
                    {'startLine': 8, 'endLine': 11},
                    {'startLine': 12, 'endLine': 20},
                    {'startLine': 13, 'endLine': 14},
                    {'startLine': 15, 'endLine': 16},
                    {'startLine': 17, 'endLine': 18},
                    {'startLine': 19, 'endLine': 20},
                    {'startLine': 22, 'endLine': 35},
                    {'startLine': 23, 'endLine': 35},
                    {'startLine': 24, 'endLine': 25},
                    {'startLine': 27, 'endLine': 29},
                    {'startLine': 28, 'endLine': 29},
                    {'startLine': 30, 'endLine': 31},
                    {'startLine': 32, 'endLine': 34},
                    {'startLine': 33, 'endLine': 34},
                    {'startLine': 38, 'endLine': 39},
                    {'startLine': 41, 'endLine': 43},
                    {'startLine': 42, 'endLine': 43},
                    {'startLine': 45, 'endLine': 54},
                    {'startLine': 47, 'endLine': 51},
                    {'startLine': 49, 'endLine': 51},
                    {'startLine': 50, 'endLine': 51},
                    {'startLine': 52, 'endLine': 54},
                    {'startLine': 53, 'endLine': 54},
                    {'startLine': 56, 'endLine': 57},
                    {'startLine': 59, 'endLine': 65},
                    {'startLine': 60, 'endLine': 61},
                    {'startLine': 62, 'endLine': 63},
                    {'startLine': 64, 'endLine': 65},
                    {'startLine': 67, 'endLine': 68}]
>       assert ranges == expected
E       AssertionError: assert [{'endLine': ...ne': 12}, ...] == [{'endLine': ...ne': 13}, ...]
E         At index 4 diff: {'startLine': 9, 'endLine': 10} != {'startLine': 12, 'endLine': 20}
E         Left contains one more item: {'endLine': 68, 'startLine': 67}
E         Use -v to get the full diff

test/plugins/test_folding.py:149: AssertionError
...
=============================================== short test summary info ================================================
FAILED test/plugins/test_folding.py::test_folding - AssertionError: assert [{'endLine': ...ne': 12}, ...] == [{'endLi...
========================== 1 failed, 1 passed, 117 deselected, 1 warning in 68.36s (0:01:08) ===========================
abuild@skylab:~/rpmbuild/BUILD/python-language-server-0.36.2>

Which makes a difference here:

https://github.com/python-ls/python-ls/blob/18338751a420ffa88f533930d837f2e9ec1c5cde/pyls/plugins/folding.py#L158

@bnavigator bnavigator mentioned this issue Apr 1, 2021
Merged
@andfoy andfoy closed this as completed in #9 Apr 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

TST: accept folding of decorator parameters in Python 3.9
1 participant
@bnavigator