python_files=*.py breaks with Python 3.10

[trehn]

Consider this simple setup (no other files in the directory):

# setup.cfg
[tool:pytest]
python_files=*.py

# broken.py
from requests import certs

def test_foo():
    print(certs.where())

This has worked fine for previous versions of Python. Since 3.10, I'm getting this:

> py.test .
======================================= test session starts =======================================
platform linux -- Python 3.10.0, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /home/trehn/git/aur/python310/venv/frob, configfile: setup.cfg
collected 0 items / 1 error                                                                       

============================================= ERRORS ==============================================
_____________________________________ ERROR collecting broken.py __________________________________
broken.py:1: in <module>
    from requests import certs
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
../lib/python3.10/site-packages/_pytest/assertion/rewrite.py:170: in exec_module
    exec(co, module.__dict__)
../lib/python3.10/site-packages/requests/__init__.py:133: in <module>
    from . import utils
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
../lib/python3.10/site-packages/_pytest/assertion/rewrite.py:170: in exec_module
    exec(co, module.__dict__)
../lib/python3.10/site-packages/requests/utils.py:41: in <module>
    DEFAULT_CA_BUNDLE_PATH = certs.where()
../lib/python3.10/site-packages/certifi/core.py:37: in where
    _CACERT_PATH = str(_CACERT_CTX.__enter__())
/usr/lib/python3.10/contextlib.py:135: in __enter__
    return next(self.gen)
/usr/lib/python3.10/importlib/_common.py:88: in _tempfile
    fd, raw_path = tempfile.mkstemp(suffix=suffix)
/usr/lib/python3.10/tempfile.py:337: in mkstemp
    return _mkstemp_inner(dir, prefix, suffix, flags, output_type)
/usr/lib/python3.10/tempfile.py:249: in _mkstemp_inner
    file = _os.path.join(dir, pre + name + suf)
E   TypeError: can only concatenate str (not "method") to str
----------------------------------------- Captured stdout -----------------------------------------
<class 'method'> <bound method DegenerateFiles.Path.name of <importlib._adapters.DegenerateFiles.Path object at 0x7f1860f59420>>
===================================== short test summary info =====================================
ERROR broken.py - TypeError: can only concatenate str (not "method") to str
!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
======================================== 1 error in 0.29s =========================================

When changing python_files in setup.cfg to anything other than *.py (e.g. b*.py), things work as expected again.

I tried digging into why this happens, but it seems rather involved. The method mentioned in the TypeError is this one:

https://github.com/python/cpython/blob/67148254146948041a77d8a2989f41b88cdb2f99/Lib/importlib/_adapters.py#L49

I'm not sure what DegenerateFiles is and under which circumstances it is used. Also note that it has been removed from Python's main branch in this commit:

python/cpython@aaa83cd

If python_files=*.py is problematic in general, pytest should throw a better error message. But of course I'd prefer it to be fixed since it lets me use shorter filenames when all my tests live in a dedicated directory.

Thanks!

[The-Compiler]

What happens if you run that broken.py with Python 3.10, without pytest involved?

[trehn]

It works and prints lib/python3.10/site-packages/certifi/cacert.pem (I just added test_foo() to the file)

[RonnyPfannschmidt]

Does it work with --assert=plain?

[trehn]

It does!

[asottile]

I think your python_files may be misconfigured -- it's causing the assertion rewriting to rewrite things inside your site-packages which are unrelated to your tests. that said, I also think there's some sort of regression in importlib-resources itself so it may be worth a https://bugs.python.org issue for that

[RonnyPfannschmidt]

@jaraco i believe its a bug in the degraded files in import-lib, at first glance name is a method instead of a property,

[jaraco]

Agreed. The TypeError appears to be a bug, and a bug that's only present in importlib_resources 5.0.x and importlib.resources as found on CPython 3.10. I'm confident that making DegenerateFiles.Path.name a property will correct the proximate error, but correcting that error leads to another proximate failure:

____________________________________________________________________ ERROR collecting broken.py _____________________________________________________________________
broken.py:2: in <module>
    from requests import certs
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-2jazv_1w/_pytest/assertion/rewrite.py:170: in exec_module
    exec(co, module.__dict__)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-2jazv_1w/requests/__init__.py:133: in <module>
    from . import utils
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-2jazv_1w/_pytest/assertion/rewrite.py:170: in exec_module
    exec(co, module.__dict__)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-2jazv_1w/requests/utils.py:41: in <module>
    DEFAULT_CA_BUNDLE_PATH = certs.where()
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-2jazv_1w/certifi/core.py:37: in where
    _CACERT_PATH = str(_CACERT_CTX.__enter__())
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py:135: in __enter__
    return next(self.gen)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/_common.py:89: in _tempfile
    os.write(fd, reader())
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/abc.py:371: in read_bytes
    with self.open('rb') as strm:
E   TypeError: DegenerateFiles.Path.open() takes 1 positional argument but 2 were given
====================================================================== short test summary info ======================================================================
ERROR broken.py - TypeError: DegenerateFiles.Path.open() takes 1 positional argument but 2 were given
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Correcting that issue leads to a new failure, this one intended:

____________________________________________________________________ ERROR collecting broken.py _____________________________________________________________________
broken.py:2: in <module>
    from requests import certs
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-_f1ljnq3/_pytest/assertion/rewrite.py:170: in exec_module
    exec(co, module.__dict__)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-_f1ljnq3/requests/__init__.py:133: in <module>
    from . import utils
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:688: in _load_unlocked
    ???
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-_f1ljnq3/_pytest/assertion/rewrite.py:170: in exec_module
    exec(co, module.__dict__)
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-_f1ljnq3/requests/utils.py:41: in <module>
    DEFAULT_CA_BUNDLE_PATH = certs.where()
/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-_f1ljnq3/certifi/core.py:37: in where
    _CACERT_PATH = str(_CACERT_CTX.__enter__())
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py:135: in __enter__
    return next(self.gen)
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/_common.py:89: in _tempfile
    os.write(fd, reader())
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/abc.py:371: in read_bytes
    with self.open('rb') as strm:
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/_adapters.py:54: in open
    raise ValueError()
E   ValueError

The ValueError here is indicating that DegenerateFiles is not expected to be encountered. Something about pytest is causing the resources of certifi not to resolve.

[jaraco]

I can confirm the issue was fixed in main:

draft $ cat broken.py
# broken.py
from requests import certs


def test_foo():
    print(certs.where())
draft $ cat setup.cfg
[tool:pytest]
python_files=*.py
draft $ pip-run -q requests git+https://github.com/pytest-dev/pytest -- -m pytest
======================================================================== test session starts ========================================================================
platform darwin -- Python 3.10.0, pytest-6.3.0.dev775+g14a879b6d, py-1.10.0, pluggy-1.0.0
rootdir: /Users/jaraco/draft, configfile: setup.cfg
collected 1 item                                                                                                                                                    

broken.py .                                                                                                                                                   [100%]

========================================================================= 1 passed in 0.23s =========================================================================

I'll take care of reporting and fixing the interface issues with DegenerateFiles.Path.

[jaraco]

Filed bpo-45419 to track the identified interface issue.

[nicoddemus]

Thanks @jaraco and everyone.

Closing this then as it is not something actionable on pytest. 👍

[trehn]

The fix for bpo-45419 just does what @jaraco already did in #9174 (comment), leaving this unresolved:

The ValueError here is indicating that DegenerateFiles is not expected to be encountered. Something about pytest is causing the resources of certifi not to resolve.

I'm still not sure if that's a Python issue, a certifi issue or a pytest issue. But right now the problem isn't gone, it's just producing a different error.

[asottile]

I'm inclined to believe this is a cpython big since it used to work and changes there caused a regression

[jaraco]

I'm still not sure if that's a Python issue, a certifi issue or a pytest issue. But right now the problem isn't gone, it's just producing a different error.

The underlying issue was corrected in #9173.

I'm inclined to believe this is a cpython big since it used to work and changes there caused a regression

If there is an outstanding Python bug, it's not yet clear what it is. The diff from Python 3.9 to 3.10 is roughly approximated by the diff between importlib_resources 1.3 -> 5.0. The path() call is basically unchanged there, but there may be some subtle differences. I'd be glad to investigate further if someone could provide a more minimal reproducer to illustrate the issue (something that doesn't require being in a pytest session).

I think it's possible that accessing resources for a loader without a resource loader was silently broken on Python 3.9 and is more noisily broken on 3.10, and perhaps that's for the best. I'm unsure if that shift in expectation was deliberate, but the impact seems to be minor (given that the issue hasn't bubbled up to a report in importlib_resources or importlib.resources). Perhaps it would make sense to turn the ValueError() into a warning (restoring the suppression of the error, but not restoring the silence fully).

[asottile]

I think it would be best if you investigated this @jaraco given you're responsible for the bulk of the changes in importlib and that's the only variable here -- I don't think it's fair to discard this as "minor" just because you haven't heard about it yet

[felix-hilden]

I can confirm that this is still happening, but my error is yet another one: a PermissionError for accessing a temporary file that is already being used. Although GH Actions produces the ValueError.

I'm on Windows 10. Taking either requests or *.py out of the equation fixes the error. To (hopefully) reproduce:

# tests/test_requests
import requests

conda create -c conda-forge -n py10 python=3.10.1
conda activate py10
pip install pytest requests
pytest tests (with python_files = "*.py")

Thanks for all the effort so far! But should this issue be reopened?

[alexlambson]

Just echoing it's happening to me too to hopefully make google show this to other people more easily. I was debugging and searching for two hours before getting here.

Hopefully the other packages update soon.