Pylint 2.7 and 2.8 are very slow #4120
Comments
|
Update: In the latest run, it is 22:42 vs 2:10. |
|
Thank you for reporting the issue. |
|
I can confirm that this issue is present due pylint. I have tested it using astroid master and pylint 2.5.2 and the time was around 3 minutes. @andy-maier have you tried to run pylint using multiple processes ( |
|
@andy-maier thanks for your report. I will continue to investigate. |
|
@andy-maier after some investigations i don't think there is a bug in the way the similarity checker works. |
|
@matusvalo @hippo91 We are running with |
|
To have a better workaround than pinning to <2.7.0, I tried to disable the similarities checker but so far was not able to achieve that. Here is what I tried (all changes are relative to the current master branch):
How can I disable the similarities checker? |
Pierre-Sassoulas
changed the title
|
@andy-maier you can try the option |
I've tried this, and saw the same speed problem. Takes more than the 2 minute build system timeout to run the test on 2k lines of source. |
|
@hadess can you give me the exact command you launched please? |
$ time /usr/bin/pylint-3 -d C0111 -d W0511 -d F0401 -d C0326 -d bad-continuation --disable=RP0801 opensubtitles.py
--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
real 0m12.306s
user 0m12.021s
sys 0m0.266sFiles are from https://gitlab.gnome.org/GNOME/totem/-/tree/master/src/plugins/opensubtitles For reasons I do not understand (aggressive multi-jobs from meson? cache thrashing?) running those tests in meson's test suite times out after 120 seconds for the above. |
|
(and I just realised that I'm using 2.6.0 and that the issue is about 2.7 and later... Feel free to disregard) |
|
The more checks we add the slower pylint get, this is a problem that started before 2.7 and will get bigger and bigger with more functionality added. But there's also genuine recursion issues that make pylint abnormally long or even unusable (#3602 for example). |
3 tasks
For those reading this issue in the future: Using pylint 2.9.2, I tried disabling "R0801" in the |
|
@andy-maier thanks for the hint. The Similarity checker have been deeply reworked and its performance improved thanks to #4565. So if you use the main branch of pylint you don't have to disable the Similarity checker. |
gnomesysadmins
pushed a commit
to GNOME/totem
that referenced
this issue
Aug 31, 2021
Disable "similarity" report in pylint so as to fix the pathological speed regression that caused most of the checks to timeout. See pylint-dev/pylint#4120 (comment)
|
Just for info: PR #4565 was released in pylint 2.10. |
gnomesysadmins
pushed a commit
to GNOME/totem
that referenced
this issue
Jun 22, 2022
Disable "similarity" report in pylint so as to fix the pathological speed regression that caused most of the checks to timeout. See pylint-dev/pylint#4120 (comment)
Thanks for the bump, this made me realise that all this time, the problem was caused by the way the test suite was run. meson sets Disabling that envvar fixed the problem for me. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Steps to reproduce
Note, the project is not using Generic types as described in issue #4079.
Current behavior
This takes 18 minutes with pylint 2.7.0 / astroid 2.5 (see also this GitHub Actions run).
It took 2 minutes with pylint 2.5.2 / astroid 2.4.0 (see also this GitHub Actions run).
Expected behavior
Get back to the time it took for pylint 2.5.2 / astroid 2.4.
pylint --version output
Slow version:
pylint 2.7.0
astroid 2.5
Python 3.8.7 (default, Feb 3 2021, 06:31:03)
[Clang 12.0.0 (clang-1200.0.32.29)]
Fast version:
pylint 2.5.2
astroid 2.4.0
Python 3.8.7 (default, Feb 3 2021, 06:31:03)
[Clang 12.0.0 (clang-1200.0.32.29)]
The text was updated successfully, but these errors were encountered: