import from test broken by pip install

[TomGoBravo]

I'm working in a repo that has tests and some test helper libraries in a directory named test (https://github.com/covid-projections/covid-data-model/tree/main/test). Mysteriously pip install dash seems to break our tests. I'm using python 3.7.9 in pyenv on a debian laptop. I reproduced the problem with a pretty simple example:

Activate a fresh pyenv virtualenv and install the latest pytest with pip install git+https://github.com/pytest-dev/pytest. Create an empty ./test/__init__.py and the following two files:

$ cat ./test/real_test.py
from test import test_helpers

def test_one():
  assert "foobar" == test_helpers.SOME_CONSTANT
$ cat ./test/test_helpers.py
SOME_CONSTANT = "foo"

Running test/real_test.py does the expected, failing a test. Now pip install dash==1.19.0 and the test fails to run with

_____________________________________________________________________________ ERROR collecting test/real_test.py _____________________________________________________________________________
ImportError while importing test module '/home/thecap/tmp/nomodule/test/real_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../.pyenv/versions/3.7.9/lib/python3.7/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'test.real_test'

Running pytest --import-mode=importlib test/real_test.py (found in the comments of #7408) gives me a little hint that it isn't including the cwd in the path used to search for libraries because it is looking at a test in the pyenv, not in the cwd.

ImportError while importing test module '/home/thecap/tmp/nomodule/test/real_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test/real_test.py:1: in <module>
    from test import test_helpers
E   ImportError: cannot import name 'test_helpers' from 'test' (/home/thecap/.pyenv/versions/3.7.9/lib/python3.7/test/__init__.py)

Mysteriously to me after running pip uninstall dash the test runs as expected with pytest test/real_test.py but is still broken with pytest --import-mode=importlib test/real_test.py.

I suspect moving our tests from test/ to tests/ will fix this but I'd like to understand why installing dash breaks things and if our current setup is a problem, help other people avoid it.

[TomGoBravo]

@hoefling provided a rather complete explanation in comments to this question at https://stackoverflow.com/questions/65910240/pytest-breaks-with-pip-install

"Short answer - don't create packages or modules that are name-shadowing the standard library, this will always break something sooner or later"

"As a workaround, I can suggest creating a conftest.py in the project root dir with the contents import sys; sys.modules.pop("test", None) - this should resolve the import issue."

Thank you Oleg!