Skip to content

gh-93259: Validate arg to Distribution.from_name. #94270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 26, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 31 additions & 8 deletions Doc/library/importlib.metadata.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@

**Source code:** :source:`Lib/importlib/metadata/__init__.py`

``importlib.metadata`` is a library that provides for access to installed
package metadata. Built in part on Python's import system, this library
``importlib.metadata`` is a library that provides access to installed
package metadata, such as its entry points or its
top-level name. Built in part on Python's import system, this library
intends to replace similar functionality in the `entry point
API`_ and `metadata API`_ of ``pkg_resources``. Along with
:mod:`importlib.resources` (with new features backported to the
`importlib_resources`_ package), this can eliminate the need to use the older
and less efficient
:mod:`importlib.resources`,
this package can eliminate the need to use the older and less efficient
``pkg_resources`` package.

By "installed package" we generally mean a third-party package installed into
Expand All @@ -32,6 +32,13 @@ By default, package metadata can live on the file system or in zip archives on
anywhere.


.. seealso::

https://importlib-metadata.readthedocs.io/
The documentation for ``importlib_metadata``, which supplies a
backport of ``importlib.metadata``.


Overview
========

Expand All @@ -54,9 +61,9 @@ You can get the version string for ``wheel`` by running the following:
>>> version('wheel') # doctest: +SKIP
'0.32.3'

You can also get the set of entry points keyed by group, such as
You can also get a collection of entry points selectable by properties of the EntryPoint (typically 'group' or 'name'), such as
``console_scripts``, ``distutils.commands`` and others. Each group contains a
sequence of :ref:`EntryPoint <entry-points>` objects.
collection of :ref:`EntryPoint <entry-points>` objects.

You can get the :ref:`metadata for a distribution <metadata>`::

Expand Down Expand Up @@ -91,7 +98,7 @@ Query all entry points::
>>> eps = entry_points() # doctest: +SKIP

The ``entry_points()`` function returns an ``EntryPoints`` object,
a sequence of all ``EntryPoint`` objects with ``names`` and ``groups``
a collection of all ``EntryPoint`` objects with ``names`` and ``groups``
attributes for convenience::

>>> sorted(eps.groups) # doctest: +SKIP
Expand Down Expand Up @@ -174,6 +181,13 @@ all the metadata in a JSON-compatible form per :PEP:`566`::
>>> wheel_metadata.json['requires_python']
'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*'

.. note::

The actual type of the object returned by ``metadata()`` is an
implementation detail and should be accessed only through the interface
described by the
`PackageMetadata protocol <https://importlib-metadata.readthedocs.io/en/latest/api.html#importlib_metadata.PackageMetadata>`.

.. versionchanged:: 3.10
The ``Description`` is now included in the metadata when presented
through the payload. Line continuation characters have been removed.
Expand Down Expand Up @@ -295,6 +309,15 @@ The full set of available metadata is not described here. See :pep:`566`
for additional details.


Distribution Discovery
======================

By default, this package provides built-in support for discovery of metadata for file system and zip file packages. This metadata finder search defaults to ``sys.path``, but varies slightly in how it interprets those values from how other import machinery does. In particular:

- ``importlib.metadata`` does not honor :class:`bytes` objects on ``sys.path``.
- ``importlib.metadata`` will incidentally honor :py:class:`pathlib.Path` objects on ``sys.path`` even though such values will be ignored for imports.


Extending the search algorithm
==============================

Expand Down
48 changes: 34 additions & 14 deletions Lib/importlib/metadata/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -543,21 +543,21 @@ def locate_file(self, path):
"""

@classmethod
def from_name(cls, name):
def from_name(cls, name: str):
"""Return the Distribution for the given package name.

:param name: The name of the distribution package to search for.
:return: The Distribution instance (or subclass thereof) for the named
package, if found.
:raises PackageNotFoundError: When the named package's distribution
metadata cannot be found.
:raises ValueError: When an invalid value is supplied for name.
"""
for resolver in cls._discover_resolvers():
dists = resolver(DistributionFinder.Context(name=name))
dist = next(iter(dists), None)
if dist is not None:
return dist
else:
if not name:
raise ValueError("A distribution name is required.")
try:
return next(cls.discover(name=name))
except StopIteration:
raise PackageNotFoundError(name)

@classmethod
Expand Down Expand Up @@ -945,13 +945,26 @@ def _normalized_name(self):
normalized name from the file system path.
"""
stem = os.path.basename(str(self._path))
return self._name_from_stem(stem) or super()._normalized_name
return (
pass_none(Prepared.normalize)(self._name_from_stem(stem))
or super()._normalized_name
)

def _name_from_stem(self, stem):
name, ext = os.path.splitext(stem)
@staticmethod
def _name_from_stem(stem):
"""
>>> PathDistribution._name_from_stem('foo-3.0.egg-info')
'foo'
>>> PathDistribution._name_from_stem('CherryPy-3.0.dist-info')
'CherryPy'
>>> PathDistribution._name_from_stem('face.egg-info')
'face'
>>> PathDistribution._name_from_stem('foo.bar')
"""
filename, ext = os.path.splitext(stem)
if ext not in ('.dist-info', '.egg-info'):
return
name, sep, rest = stem.partition('-')
name, sep, rest = filename.partition('-')
return name


Expand Down Expand Up @@ -991,6 +1004,15 @@ def version(distribution_name):
return distribution(distribution_name).version


_unique = functools.partial(
unique_everseen,
key=operator.attrgetter('_normalized_name'),
)
"""
Wrapper for ``distributions`` to return unique distributions by name.
"""


def entry_points(**params) -> Union[EntryPoints, SelectableGroups]:
"""Return EntryPoint objects for all installed packages.

Expand All @@ -1008,10 +1030,8 @@ def entry_points(**params) -> Union[EntryPoints, SelectableGroups]:

:return: EntryPoints or SelectableGroups for all installed packages.
"""
norm_name = operator.attrgetter('_normalized_name')
unique = functools.partial(unique_everseen, key=norm_name)
eps = itertools.chain.from_iterable(
dist.entry_points for dist in unique(distributions())
dist.entry_points for dist in _unique(distributions())
)
return SelectableGroups.load(eps).select(**params)

Expand Down
16 changes: 16 additions & 0 deletions Lib/test/test_importlib/fixtures.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import pathlib
import tempfile
import textwrap
import functools
import contextlib

from test.support.os_helper import FS_NONASCII
Expand Down Expand Up @@ -296,3 +297,18 @@ def setUp(self):
# Add self.zip_name to the front of sys.path.
self.resources = contextlib.ExitStack()
self.addCleanup(self.resources.close)


def parameterize(*args_set):
"""Run test method with a series of parameters."""

def wrapper(func):
@functools.wraps(func)
def _inner(self):
for args in args_set:
with self.subTest(**args):
func(self, **args)

return _inner

return wrapper
91 changes: 49 additions & 42 deletions Lib/test/test_importlib/test_main.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import re
import json
import pickle
import textwrap
import unittest
import warnings
import importlib.metadata
Expand All @@ -16,6 +15,7 @@
Distribution,
EntryPoint,
PackageNotFoundError,
_unique,
distributions,
entry_points,
metadata,
Expand Down Expand Up @@ -51,6 +51,14 @@ def test_package_not_found_mentions_metadata(self):
def test_new_style_classes(self):
self.assertIsInstance(Distribution, type)

@fixtures.parameterize(
dict(name=None),
dict(name=''),
)
def test_invalid_inputs_to_from_name(self, name):
with self.assertRaises(Exception):
Distribution.from_name(name)


class ImportTests(fixtures.DistInfoPkg, unittest.TestCase):
def test_import_nonexistent_module(self):
Expand Down Expand Up @@ -78,48 +86,50 @@ def test_resolve_without_attr(self):

class NameNormalizationTests(fixtures.OnSysPath, fixtures.SiteDir, unittest.TestCase):
@staticmethod
def pkg_with_dashes(site_dir):
def make_pkg(name):
"""
Create minimal metadata for a package with dashes
in the name (and thus underscores in the filename).
Create minimal metadata for a dist-info package with
the indicated name on the file system.
"""
metadata_dir = site_dir / 'my_pkg.dist-info'
metadata_dir.mkdir()
metadata = metadata_dir / 'METADATA'
with metadata.open('w', encoding='utf-8') as strm:
strm.write('Version: 1.0\n')
return 'my-pkg'
return {
f'{name}.dist-info': {
'METADATA': 'VERSION: 1.0\n',
},
}

def test_dashes_in_dist_name_found_as_underscores(self):
"""
For a package with a dash in the name, the dist-info metadata
uses underscores in the name. Ensure the metadata loads.
"""
pkg_name = self.pkg_with_dashes(self.site_dir)
assert version(pkg_name) == '1.0'

@staticmethod
def pkg_with_mixed_case(site_dir):
"""
Create minimal metadata for a package with mixed case
in the name.
"""
metadata_dir = site_dir / 'CherryPy.dist-info'
metadata_dir.mkdir()
metadata = metadata_dir / 'METADATA'
with metadata.open('w', encoding='utf-8') as strm:
strm.write('Version: 1.0\n')
return 'CherryPy'
fixtures.build_files(self.make_pkg('my_pkg'), self.site_dir)
assert version('my-pkg') == '1.0'

def test_dist_name_found_as_any_case(self):
"""
Ensure the metadata loads when queried with any case.
"""
pkg_name = self.pkg_with_mixed_case(self.site_dir)
pkg_name = 'CherryPy'
fixtures.build_files(self.make_pkg(pkg_name), self.site_dir)
assert version(pkg_name) == '1.0'
assert version(pkg_name.lower()) == '1.0'
assert version(pkg_name.upper()) == '1.0'

def test_unique_distributions(self):
"""
Two distributions varying only by non-normalized name on
the file system should resolve as the same.
"""
fixtures.build_files(self.make_pkg('abc'), self.site_dir)
before = list(_unique(distributions()))

alt_site_dir = self.fixtures.enter_context(fixtures.tempdir())
self.fixtures.enter_context(self.add_sys_path(alt_site_dir))
fixtures.build_files(self.make_pkg('ABC'), alt_site_dir)
after = list(_unique(distributions()))

assert len(after) == len(before)


class NonASCIITests(fixtures.OnSysPath, fixtures.SiteDir, unittest.TestCase):
@staticmethod
Expand All @@ -128,11 +138,12 @@ def pkg_with_non_ascii_description(site_dir):
Create minimal metadata for a package with non-ASCII in
the description.
"""
metadata_dir = site_dir / 'portend.dist-info'
metadata_dir.mkdir()
metadata = metadata_dir / 'METADATA'
with metadata.open('w', encoding='utf-8') as fp:
fp.write('Description: pôrˈtend')
contents = {
'portend.dist-info': {
'METADATA': 'Description: pôrˈtend',
},
}
fixtures.build_files(contents, site_dir)
return 'portend'

@staticmethod
Expand All @@ -141,19 +152,15 @@ def pkg_with_non_ascii_description_egg_info(site_dir):
Create minimal metadata for an egg-info package with
non-ASCII in the description.
"""
metadata_dir = site_dir / 'portend.dist-info'
metadata_dir.mkdir()
metadata = metadata_dir / 'METADATA'
with metadata.open('w', encoding='utf-8') as fp:
fp.write(
textwrap.dedent(
"""
contents = {
'portend.dist-info': {
'METADATA': """
Name: portend

pôrˈtend
"""
).strip()
)
pôrˈtend""",
},
}
fixtures.build_files(contents, site_dir)
return 'portend'

def test_metadata_loads(self):
Expand Down
6 changes: 3 additions & 3 deletions Lib/test/test_importlib/test_metadata_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,15 +89,15 @@ def test_entry_points_distribution(self):
self.assertIn(ep.dist.name, ('distinfo-pkg', 'egginfo-pkg'))
self.assertEqual(ep.dist.version, "1.0.0")

def test_entry_points_unique_packages(self):
def test_entry_points_unique_packages_normalized(self):
"""
Entry points should only be exposed for the first package
on sys.path with a given name.
on sys.path with a given name (even when normalized).
"""
alt_site_dir = self.fixtures.enter_context(fixtures.tempdir())
self.fixtures.enter_context(self.add_sys_path(alt_site_dir))
alt_pkg = {
"distinfo_pkg-1.1.0.dist-info": {
"DistInfo_pkg-1.1.0.dist-info": {
"METADATA": """
Name: distinfo-pkg
Version: 1.1.0
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Now raise ``ValueError`` when ``None`` or an empty string are passed to
``Distribution.from_name`` (and other callers).