Invoking tests against editable installs have been masking bugs

[jacobtylerwalls]

We've been invoking tests on CI against an editable install of pylint for a long time. Doing so has the potential to mask issues. See blog post.

We now have evidence of some of those issues from the failures in the PR testing against Python 3.12, which for unrelated reasons, forced us to upgrade setuptools. Without using the legacy behavior, bugs became evident.

One of the most minimal bugs is this one:

pylint/tests/lint/test_pylinter.py

Lines 39 to 41 in 9f2de91

	with mock.patch(
	"pylint.lint.PyLinter.check_astroid_module", side_effect=RuntimeError
	):

That mock shouldn't work, since the real location of the PyLinter class is at pylint.lint.pylinter.PyLinter. It was only working because the pylint.lint module imports PyLinter, but AFIAU there's no reason to expect that to work when testing the installation of the pylint package, which won't install pylint.lint (whereas an editable install will, assuming you're invoking tests from pylint.

That's just a bug in a test, but there are functional bugs, too, such as the moved check in #8702 potentially crashing:

File "/Users/<user>/pylint/pylint/config/config_initialization.py", line 114, in _config_initialization
    for exc_name in linter.config.overgeneral_exceptions:
AttributeError: 'Namespace' object has no attribute 'overgeneral_exceptions'

---

I've found a workaround for the 3.12 branch to keep that one using legacy behavior, but suggesting we also look at this fixing this for the next pylint alpha.

[jacobtylerwalls]

This is no longer as crash-y as before. I haven't worked out when or why, but it's still worth doing.