Skip to content

Don't doubly-parallelise sphinx-lint #110617

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

AA-Turner
Copy link
Member

Sphinx-Lint internally parallelises, and pre-commit also parallelises, leading to seeming resource contention. I observe a speed-up of ~5 seconds (from ~12 to ~7) when we let sphinx-lint control the parallelism.

   5       12.582 pre-commit run sphinx-lint --all-files
   6       11.193 pre-commit run sphinx-lint --all-files
   7       13.530 pre-commit run sphinx-lint --all-files
   8        0.013 history
   9        7.422 pre-commit run sphinx-lint --all-files
  10        7.067 pre-commit run sphinx-lint --all-files
  11        7.344 pre-commit run sphinx-lint --all-files

Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! On my laptop this speeds things up from around 22 seconds to around 17 seconds. On @hugovk's, it went down from 5.5 seconds to around 3.4 seconds.

@AlexWaygood AlexWaygood merged commit e24f9ae into python:main Oct 10, 2023
@miss-islington
Copy link
Contributor

Thanks @AA-Turner for the PR, and @AlexWaygood for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 10, 2023
(cherry picked from commit e24f9ae)

Co-authored-by: Adam Turner <[email protected]>
@bedevere-app
Copy link

bedevere-app bot commented Oct 10, 2023

GH-110626 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 only security fixes label Oct 10, 2023
@bedevere-app
Copy link

bedevere-app bot commented Oct 10, 2023

GH-110627 is a backport of this pull request to the 3.11 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.11 only security fixes label Oct 10, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 10, 2023
(cherry picked from commit e24f9ae)

Co-authored-by: Adam Turner <[email protected]>
hugovk pushed a commit that referenced this pull request Oct 10, 2023
hugovk pushed a commit that referenced this pull request Oct 10, 2023
Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Documentation in the Doc dir skip issue skip news
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants