Skip to content

gh-118761: Optimise import time for string #132037

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 8, 2025
Merged

Conversation

AA-Turner
Copy link
Member

@AA-Turner AA-Turner commented Apr 3, 2025

This PR achieves a 27x improvement in import time for the string module. The main improvement comes from replacing Template.__init_subclass__() (GH-16256) with a descriptor class, allowing lazy import of the re module.

Current:

import string: cumulative time
mean: 9162.100 µs
median: 9133.000 µs
stdev: 66.662
min: 9071
max: 9301

This PR:

import string: cumulative time
mean: 334.967 µs
median: 329.000 µs
stdev: 13.438
min: 316
max: 368

Copy link
Member

@picnixz picnixz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm very happy with this because I need to access many times string.* constants without needing the Template class so it's a very good optimization.

@python-cla-bot
Copy link

python-cla-bot bot commented Apr 6, 2025

All commit authors signed the Contributor License Agreement.

CLA signed

@AA-Turner AA-Turner requested a review from picnixz April 6, 2025 17:56
Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting idea. The code looks intimidating, but it might work.

How the help output looks now?

Is __init_subclass__() needed anymore?

@AA-Turner
Copy link
Member Author

Help output looks fine:

>>> from string import Template
>>> assert Template.flags is None
>>> help(Template)
Help on class Template in module string:

class Template(builtins.object)
 |  Template(template)
 |
 |  A string class for supporting $-substitutions.
 |
 |  Methods defined here:
 |
 |  __init__(self, template)
 |      Initialize self.  See help(type(self)) for accurate signature.
 |
 |  get_identifiers(self)
 |
 |  is_valid(self)
 |
 |  safe_substitute(self, mapping={}, /, **kws)
 |
 |  substitute(self, mapping={}, /, **kws)
 |
 |  ----------------------------------------------------------------------
 |  Class methods defined here:
 |
 |  __init_subclass__()
 |      This method is called when a class is subclassed.
 |
 |      The default implementation does nothing. It may be
 |      overridden to extend subclasses.
 |
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |
 |  __dict__
 |      dictionary for instance variables
 |
 |  __weakref__
 |      list of weak references to the object
 |
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |
 |  braceidpattern = None
 |
 |  delimiter = '$'
 |
 |  flags = re.IGNORECASE
 |
 |  idpattern = '(?a:[_a-z][_a-z0-9]*)'
 |
 |  pattern = re.compile('\n            \\$(?:\n              ...identifie...

>>> 

Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It perhaps could be made simpler with classmethod + property, but the future of this feature is not clear.

Co-authored-by: Serhiy Storchaka <[email protected]>
@AA-Turner AA-Turner enabled auto-merge (squash) April 8, 2025 09:44
@AA-Turner AA-Turner merged commit ee36572 into python:main Apr 8, 2025
39 checks passed
@AA-Turner AA-Turner deleted the opt-string branch April 8, 2025 10:07
seehwan pushed a commit to seehwan/cpython that referenced this pull request Apr 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Performance or resource usage
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants