test_launcher fails on installed 3.11.0rc1

[terryjreedy]

@zooba The new-in-3.11 test_launcher fails when run from the non-admin account used to install. It runs when I run command prompt as admin. I believe I installed for all users, which required the admin pin, and checked to (re)install the py launcher, to get any changes.

@pablogsal What is the policy with regard to installed-only failures like this?

f:\dev\3x>py -m test -v test_launcher
== CPython 3.11.0rc1 (main, Aug 8 2022, 11:30:54) [MSC v.1932 64 bit (AMD64)]
== Windows-10-10.0.19044-SP0 little-endian
== cwd: C:\Users\Terry\AppData\Local\Temp\test_python_15520æ
== CPU count: 12
== encodings: locale=cp1252, FS=utf-8
0:00:00 Run tests sequentially
0:00:00 [1/1] test_launcher
...
FAILED (errors=13, skipped=2)

Most test_py* tests and a couple of others fail with

======================================================================
ERROR: test_py2_default (test.test_launcher.TestLauncher.test_py2_default)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Programs\Python311\Lib\test\test_launcher.py", line 431, in test_py2_default
    with self.py_ini(TEST_PY_COMMANDS):
  File "C:\Programs\Python311\Lib\test\test_launcher.py", line 138, in __enter__
    self.path.write_text(self.content, encoding="utf-16")
  File "C:\Programs\Python311\Lib\pathlib.py", line 1078, in write_text
    with self.open(mode='w', encoding=encoding, errors=errors, newline=newline) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Programs\Python311\Lib\pathlib.py", line 1044, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: 'C:\\WINDOWS\\py.ini'

If there is no way to make these tests pass, perhaps they could be skipped and 1, not 13, suggestions given to run as admin to completely test the py launcher.

[eryksun]

RunPyMixin.py_ini() can use "%LOCALAPPDATA%\py.ini", which takes precedence over "%__APPDIR__%\py.ini". For example:

    def py_ini(self, content):
        local_appdata = os.environ.get("LOCALAPPDATA")
        if not local_appdata:
            raise unittest.SkipTest("LOCALAPPDATA environment variable is "
                                    "missing or empty")
        return PreservePyIni(Path(local_appdata) / "py.ini", content)

[zooba]

If you run the tests as admin, do they pass? If they're failing because they can't write the required .ini file, that's understandable, and we should add handling for that case.

This seems to be a test-only issue, so it could go into 3.11.0 safely enough, or can also wait until 3.11.1 safely enough.

[eryksun]

we should add handling for that case.

Failing to write to the file should be handled, but also change py_ini() to use the per-user "%LOCALAPPDATA%\py.ini", which works both for standard users and admins and also takes precedence over "py.ini" beside the executable. Tests should not require admin access or encourage developers to always work with admin access. Also, a per-user "py.ini" should not interfere with the launcher tests. It has to be temporarily replaced if it exists, else many tests will fail if the user has set a default "python", "python3", or "python2" in "%LOCALAPPDATA%\py.ini". By setting them all, I get 12 failures and 1 error.

[terryjreedy]

As I said above, pass as admin. As user, I have no problem running as user with 'py', 'py -3.10 ...', 'py -m ...'. I am curious whether there any 'py ...' operation that writes the .ini.

Anyway, Eryk's patch fixes the issue nicely. I'll make a PR, credit Eryk, and assign to Steve to decide whether to change anything and when to merge.

EDIT: I spoke too soon as I needed to test on the installed 3.11 that failed before. But same result of pass with only skips being for 2.7 and 32-bit 3.x. Does the test run with repository main use the installed py.ini or a local one without a permission issue?

[eryksun]

I am curious whether there any 'py ...' operation that writes the .ini.

The launcher only reads from the "py.ini" file(s). It first tries the user .ini file. If it doesn't exist or doesn't define the command name in the target section (either "defaults" or "commands"), then the launcher tries the .ini that's beside the executable.

Does the test run with repository main use the installed py.ini or a local one without a permission issue?

The tests check sysconfig.is_python_build() to use the launcher from the build directory. In this case, they write to "py.ini" beside the launcher in the build directory, which should be allowed. OTOH, for running tests with an installed Python and launcher, where the launcher is installed for all users, a standard user is not allowed to write to "C:\Windows\py.ini".

decide whether to change anything and when to merge.

Even with the change to write to the user "py.ini", PreservePyIni.__enter__() should still skip the test if writing to self.path raises a PermissionError.

[terryjreedy]

Close? or more to do?