test_tools: test_freeze_simple_script() fails on s390x SLES 3.x: [Errno 2] No such file or directory: (...) #109615

vstinner · 2023-09-20T15:22:19Z

s390x SLES 3.x:

test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script) ... ERROR

Stdout:
creating the script to be frozen at /tmp/test_python_3vf4i07o/tmp7pw_cxg6/app.py
copying the source tree into /tmp/test_python_3vf4i07o/tmp7pw_cxg6/cpython...

(...)

======================================================================
ERROR: test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 146, in prepare
    copy_source_tree(srcdir, SRCDIR)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 95, in copy_source_tree
    shutil.copytree(oldroot, newroot, ignore=ignore_non_src)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/shutil.py", line 588, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/shutil.py", line 542, in _copytree
    raise Error(errors)
shutil.Error: [('/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/build/test_python_29475æ', '/tmp/test_python_3vf4i07o/tmp7pw_cxg6/cpython/build/test_python_29475æ', "[Errno 2] No such file or directory: '/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/build/test_python_29475æ'")]

Stdout:
creating the script to be frozen at /tmp/test_python_3vf4i07o/tmp7pw_cxg6/app.py
copying the source tree into /tmp/test_python_3vf4i07o/tmp7pw_cxg6/cpython...

----------------------------------------------------------------------
Ran 36 tests in 7.203s

FAILED (errors=1)
test test_tools failed

build: https://buildbot.python.org/all/#/builders/540/builds/6583

Linked PRs

The text was updated successfully, but these errors were encountered:

vstinner · 2023-09-21T20:17:03Z

The test fails also on s390x Debian 3.x: https://buildbot.python.org/all/#builders/49/builds/6610

serhiy-storchaka · 2023-09-26T19:53:32Z

It tries to copy the source tree from /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/build/test_python_29475æ, but the correct path is /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/.

vstinner · 2023-09-26T22:50:40Z

build: https://buildbot.python.org/all/#/builders/540/builds/6583

Git was run in directory: /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build.
configure command: ./configure --prefix '$(PWD)/target' --with-pydebug
test.pythoninfo:
- sysconfig[abs_builddir]: /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build
- sysconfig[srcdir]: /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build
Tests are run in directory: /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/build/test_python_worker_8799æ

Tools/freeze/freeze.py computes:

SRCDIR = os.path.dirname(os.path.dirname(TOOL_ROOT))

It tries to copy the source tree from /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/build/test_python_29475æ, but the correct path is /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/.

Oh. Maybe the code defining SRCDIR is wrong.

* Use sysconfig to get the source directory. * Use sysconfig.get_config_var() to get CONFIG_ARGS variable.

vstinner · 2023-09-26T23:03:04Z

I proposed PR #109935 to fix the issue. I'm not sure yet if my fix is correct.

Fix copy_source_tree() function of test_tools.test_freeze: * Don't copy SRC_DIR/build/ anymore. This directory is modified by other tests running in parallel. * Add test.support.copy_python_src_ignore(). * Use sysconfig to get the source directory. * Use sysconfig.get_config_var() to get CONFIG_ARGS variable.

vstinner · 2023-09-27T00:32:38Z

It seems like freeze copy_source_tree() copies build/ directory which is modified by other tests running in parallel. So it's likely that the copy can fail. I updated my PR #109935 to ignore this directory in the copy. Recently, I fixed a similar issue in venv with __pycache__/ directories: see issue gh-109748.

Fix copy_source_tree() function of test_tools.test_freeze: * Don't copy SRC_DIR/build/ anymore. This directory is modified by other tests running in parallel. * Add test.support.copy_python_src_ignore(). * Use sysconfig to get the source directory. * Use sysconfig.get_config_var() to get CONFIG_ARGS variable.

Fix copy_source_tree() function of test_tools.test_freeze: * Don't copy SRC_DIR/build/ anymore. This directory is modified by other tests running in parallel. * Add test.support.copy_python_src_ignore(). * Use sysconfig to get the source directory. * Use sysconfig.get_config_var() to get CONFIG_ARGS variable. (cherry picked from commit 1512d6c)

gh-109615: Fix test_tools.test_freeze SRCDIR (#109935) Fix copy_source_tree() function of test_tools.test_freeze: * Don't copy SRC_DIR/build/ anymore. This directory is modified by other tests running in parallel. * Add test.support.copy_python_src_ignore(). * Use sysconfig to get the source directory. * Use sysconfig.get_config_var() to get CONFIG_ARGS variable. (cherry picked from commit 1512d6c)

Fix the test when the test is run on an installed Python: get abs_srcdir of sysconfig, and skip the test if the Python source code cannot be found.

Fix the test when the test is run on an installed Python: use "abs_srcdir" of sysconfig, and skip the test if the Python source code cannot be found. * Tools/patchcheck/patchcheck.py, Tools/freeze/test/freeze.py and Lib/test/libregrtest/utils.py now first try to get "abs_srcdir" from sysconfig, before getting "srcdir" from sysconfig. * test.pythoninfo logs sysconfig "abs_srcdir".

Fix the test when run on an installed Python: use "abs_srcdir" of sysconfig, and skip the test if the Python source code cannot be found. * Tools/patchcheck/patchcheck.py, Tools/freeze/test/freeze.py and Lib/test/libregrtest/utils.py now first try to get "abs_srcdir" from sysconfig, before getting "srcdir" from sysconfig. * test.pythoninfo logs sysconfig "abs_srcdir".

…#109962) gh-109615: Fix support test_copy_python_src_ignore() (#109958) Fix the test when run on an installed Python: use "abs_srcdir" of sysconfig, and skip the test if the Python source code cannot be found. * Tools/patchcheck/patchcheck.py, Tools/freeze/test/freeze.py and Lib/test/libregrtest/utils.py now first try to get "abs_srcdir" from sysconfig, before getting "srcdir" from sysconfig. * test.pythoninfo logs sysconfig "abs_srcdir". (cherry picked from commit b89ed9d)

Not only check if src_dir exists, but look also for Lib/os.py landmark.

…ythonGH-109970) Not only check if src_dir exists, but look also for Lib/os.py landmark. (cherry picked from commit cc54bcf) Co-authored-by: Victor Stinner <[email protected]>

…H-109970) (#109976) gh-109615: Fix support test_copy_python_src_ignore() on WASM (GH-109970) Not only check if src_dir exists, but look also for Lib/os.py landmark. (cherry picked from commit cc54bcf) Co-authored-by: Victor Stinner <[email protected]>

Fix copy_source_tree() function of test_tools.test_freeze: * Don't copy SRC_DIR/build/ anymore. This directory is modified by other tests running in parallel. * Add test.support.copy_python_src_ignore(). * Use sysconfig to get the source directory. * Use sysconfig.get_config_var() to get CONFIG_ARGS variable.

…9958) Fix the test when run on an installed Python: use "abs_srcdir" of sysconfig, and skip the test if the Python source code cannot be found. * Tools/patchcheck/patchcheck.py, Tools/freeze/test/freeze.py and Lib/test/libregrtest/utils.py now first try to get "abs_srcdir" from sysconfig, before getting "srcdir" from sysconfig. * test.pythoninfo logs sysconfig "abs_srcdir".

…ython#109970) Not only check if src_dir exists, but look also for Lib/os.py landmark.

zooba · 2023-09-29T13:55:49Z

The last change makes test_copy_python_src_ignore fail rather than skip on Windows installs, because the standard library exists but the Modules directory (which is the actual important landmark) does not.

PR incoming.

…src_ignore

…nore (GH-110108)

…src_ignore (pythonGH-110108) (cherry picked from commit 20bc5f7) Co-authored-by: Steve Dower <[email protected]>

zooba · 2023-09-29T15:26:57Z

There's a few fixes here waiting on 3.12 branch to open up before they'll be merged, so I'm leaving this open until they're done.

…nore (GH-110108) (cherry picked from commit 20bc5f7) Co-authored-by: Steve Dower <[email protected]>

gh-109615: Fix test_tools.test_freeze SRCDIR (#109935) Fix copy_source_tree() function of test_tools.test_freeze: * Don't copy SRC_DIR/build/ anymore. This directory is modified by other tests running in parallel. * Add test.support.copy_python_src_ignore(). * Use sysconfig to get the source directory. * Use sysconfig.get_config_var() to get CONFIG_ARGS variable. (cherry picked from commit 1512d6c)

…9958) Fix the test when run on an installed Python: use "abs_srcdir" of sysconfig, and skip the test if the Python source code cannot be found. * Tools/patchcheck/patchcheck.py, Tools/freeze/test/freeze.py and Lib/test/libregrtest/utils.py now first try to get "abs_srcdir" from sysconfig, before getting "srcdir" from sysconfig. * test.pythoninfo logs sysconfig "abs_srcdir". (cherry picked from commit b89ed9d)

…ython#109970) Not only check if src_dir exists, but look also for Lib/os.py landmark. (cherry picked from commit cc54bcf)

…src_ignore (pythonGH-110108) (cherry picked from commit 20bc5f7)

…#110340) * gh-109615: Fix support test_copy_python_src_ignore() (#109958) Fix the test when run on an installed Python: use "abs_srcdir" of sysconfig, and skip the test if the Python source code cannot be found. * Tools/patchcheck/patchcheck.py, Tools/freeze/test/freeze.py and Lib/test/libregrtest/utils.py now first try to get "abs_srcdir" from sysconfig, before getting "srcdir" from sysconfig. * test.pythoninfo logs sysconfig "abs_srcdir". (cherry picked from commit b89ed9d) * gh-109615: Fix support test_copy_python_src_ignore() on WASM (#109970) Not only check if src_dir exists, but look also for Lib/os.py landmark. (cherry picked from commit cc54bcf) * gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ignore (GH-110108) (cherry picked from commit 20bc5f7) * gh-109748: Fix again venv test_zippath_from_non_installed_posix() (#110149) Call also copy_python_src_ignore() on listdir() names. shutil.copytree(): replace set() with an empty tuple. An empty tuple becomes a constant in the compiler and checking if an item is in an empty tuple is cheap. (cherry picked from commit 0def8c7) --------- Co-authored-by: Steve Dower <[email protected]>

vstinner · 2023-10-17T06:44:52Z

Ok, all changes should now be backported. I close the issue.

Fix copy_source_tree() function of test_tools.test_freeze: * Don't copy SRC_DIR/build/ anymore. This directory is modified by other tests running in parallel. * Add test.support.copy_python_src_ignore(). * Use sysconfig to get the source directory. * Use sysconfig.get_config_var() to get CONFIG_ARGS variable.

…9958) Fix the test when run on an installed Python: use "abs_srcdir" of sysconfig, and skip the test if the Python source code cannot be found. * Tools/patchcheck/patchcheck.py, Tools/freeze/test/freeze.py and Lib/test/libregrtest/utils.py now first try to get "abs_srcdir" from sysconfig, before getting "srcdir" from sysconfig. * test.pythoninfo logs sysconfig "abs_srcdir".

…ython#109970) Not only check if src_dir exists, but look also for Lib/os.py landmark.

…src_ignore (pythonGH-110108)

vstinner added the tests Tests in the Lib/test dir label Sep 20, 2023

vstinner mentioned this issue Sep 26, 2023

gh-109566, regrtest: Add --fast-ci and --slow-ci options #109570

Merged

vstinner added a commit to vstinner/cpython that referenced this issue Sep 26, 2023

pythongh-109615: Fix test_tools.test_freeze SRCDIR

a57099f

* Use sysconfig to get the source directory. * Use sysconfig.get_config_var() to get CONFIG_ARGS variable.

bedevere-app bot mentioned this issue Sep 26, 2023

gh-109615: Fix test_tools.test_freeze SRCDIR #109935

Merged

serhiy-storchaka mentioned this issue Sep 27, 2023

test_interpreters: test_create_many_threaded() failed on FreeBSD: log: RuntimeError: interpreter creation failed #109700

Open

bedevere-app bot mentioned this issue Sep 27, 2023

[3.12] gh-109615: Fix test_tools.test_freeze SRCDIR (#109935) #109950

Merged

bedevere-app bot mentioned this issue Sep 27, 2023

[3.11] gh-109615: Fix test_tools.test_freeze SRCDIR (#109935) #109951

Merged

bedevere-app bot mentioned this issue Sep 27, 2023

gh-109615: Fix support test_copy_python_src_ignore() #109958

Merged

bedevere-app bot mentioned this issue Sep 27, 2023

[3.11] gh-109615: Fix support test_copy_python_src_ignore() (#109958) #109962

Merged

vstinner added a commit to vstinner/cpython that referenced this issue Sep 27, 2023

pythongh-109615: Fix support test_copy_python_src_ignore() on WASM

c7d02fc

Not only check if src_dir exists, but look also for Lib/os.py landmark.

bedevere-app bot mentioned this issue Sep 27, 2023

gh-109615: Fix support test_copy_python_src_ignore() on WASM #109970

Merged

vstinner added a commit that referenced this issue Sep 27, 2023

gh-109615: Fix support test_copy_python_src_ignore() on WASM (#109970)

cc54bcf

Not only check if src_dir exists, but look also for Lib/os.py landmark.

bedevere-app bot mentioned this issue Sep 27, 2023

[3.11] gh-109615: Fix support test_copy_python_src_ignore() on WASM (GH-109970) #109976

Merged

csm10495 pushed a commit to csm10495/cpython that referenced this issue Sep 28, 2023

pythongh-109615: Fix support test_copy_python_src_ignore() on WASM (p…

c98e6ba

…ython#109970) Not only check if src_dir exists, but look also for Lib/os.py landmark.

vstinner closed this as completed Sep 29, 2023

zooba reopened this Sep 29, 2023

zooba added a commit to zooba/cpython that referenced this issue Sep 29, 2023

pythongh-109615: Look for 'Modules' as landmark for test_copy_python_…

ee811e5

…src_ignore

bedevere-app bot mentioned this issue Sep 29, 2023

gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ignore #110108

Merged

zooba added a commit that referenced this issue Sep 29, 2023

gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ig…

20bc5f7

…nore (GH-110108)

bedevere-app bot mentioned this issue Sep 29, 2023

[3.11] gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ignore (GH-110108) #110110

Merged

zooba added a commit that referenced this issue Sep 29, 2023

gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ig…

a95e8cf

…nore (GH-110108) (cherry picked from commit 20bc5f7) Co-authored-by: Steve Dower <[email protected]>

vstinner added a commit to vstinner/cpython that referenced this issue Oct 4, 2023

pythongh-109615: Fix support test_copy_python_src_ignore() on WASM (p…

c310ecb

…ython#109970) Not only check if src_dir exists, but look also for Lib/os.py landmark. (cherry picked from commit cc54bcf)

vstinner pushed a commit to vstinner/cpython that referenced this issue Oct 4, 2023

pythongh-109615: Look for 'Modules' as landmark for test_copy_python_…

4f4c799

…src_ignore (pythonGH-110108) (cherry picked from commit 20bc5f7)

bedevere-app bot mentioned this issue Oct 4, 2023

[3.12] gh-109615: Fix support test_copy_python_src_ignore() (#109958) #110340

Merged

vstinner closed this as completed Oct 17, 2023

Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024

pythongh-109615: Fix support test_copy_python_src_ignore() on WASM (p…

1e8bee1

…ython#109970) Not only check if src_dir exists, but look also for Lib/os.py landmark.

Glyphack pushed a commit to Glyphack/cpython that referenced this issue Sep 2, 2024

pythongh-109615: Look for 'Modules' as landmark for test_copy_python_…

4e5e6f3

…src_ignore (pythonGH-110108)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test_tools: test_freeze_simple_script() fails on s390x SLES 3.x: [Errno 2] No such file or directory: (...) #109615

test_tools: test_freeze_simple_script() fails on s390x SLES 3.x: [Errno 2] No such file or directory: (...) #109615

vstinner commented Sep 20, 2023 •

edited by bedevere-app bot

Loading

vstinner commented Sep 21, 2023

serhiy-storchaka commented Sep 26, 2023

vstinner commented Sep 26, 2023

vstinner commented Sep 26, 2023

vstinner commented Sep 27, 2023

zooba commented Sep 29, 2023

zooba commented Sep 29, 2023

vstinner commented Oct 17, 2023

test_tools: test_freeze_simple_script() fails on s390x SLES 3.x: [Errno 2] No such file or directory: (...) #109615

test_tools: test_freeze_simple_script() fails on s390x SLES 3.x: [Errno 2] No such file or directory: (...) #109615

Comments

vstinner commented Sep 20, 2023 • edited by bedevere-app bot Loading

Linked PRs

vstinner commented Sep 21, 2023

serhiy-storchaka commented Sep 26, 2023

vstinner commented Sep 26, 2023

vstinner commented Sep 26, 2023

vstinner commented Sep 27, 2023

zooba commented Sep 29, 2023

zooba commented Sep 29, 2023

vstinner commented Oct 17, 2023

vstinner commented Sep 20, 2023 •

edited by bedevere-app bot

Loading