Tests fail with Meson 1.8.0 in openSUSE package build

[dnicolodi]

They were passing with meson 1.7.1, but as we rebuild everything as part of the testing, the errors could come from incompatibilities in other packages caused by build with meson 1.8.0. As you are certainly more knowledgeable regarding the meson ecosystem than me, could you please take a look and provide some insight?

Full error log:

[   68s] =================================== FAILURES ===================================
[   68s] ________________________ test_ndebug[-Dbuildtype=debug] ________________________
[   68s] 
[   68s] package_purelib_and_platlib = PosixPath('/home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/purelib-and-platlib')
[   68s] tmp_path = PosixPath('/tmp/pytest-of-abuild/pytest-0/test_ndebug__Dbuildtype_debug_0')
[   68s] args = ['-Dbuildtype=debug'], expected = False
[   68s] 
[   68s]     @pytest.mark.skipif(NINJA_VERSION < (1, 10), reason='Ninja version too old')
[   68s]     @pytest.mark.parametrize(
[   68s]         ('args', 'expected'),
[   68s]         [
[   68s]             ([], True),
[   68s]             (['-Dbuildtype=release'], True),
[   68s]             (['-Dbuildtype=debug'], False),
[   68s]         ],
[   68s]         ids=['', '-Dbuildtype=release', '-Dbuildtype=debug'],
[   68s]     )
[   68s]     def test_ndebug(package_purelib_and_platlib, tmp_path, args, expected):
[   68s]         with mesonpy._project({'setup-args': args}) as project:
[   68s]             command = subprocess.run(
[   68s]                 # Ask ninja what is the command that would be used to
[   68s]                 # compile a C source file (the trailing ^ is used to
[   68s]                 # specify the target that is the first output of the rule
[   68s]                 # containing the specified source file).
[   68s]                 ['ninja', '-C', os.fspath(project._build_dir), '-t', 'commands', '../plat.c^'],
[   68s]                 stdout=subprocess.PIPE, check=True).stdout
[   68s] >           assert (b'-DNDEBUG' in command) == expected
[   68s] E           AssertionError: assert (b'-DNDEBUG' in b'cc -Iplat.cpython-311-x86_64-linux-gnu.so.p -I. -I.. -I/usr/include/python3.11 -fvisibility=hidden -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -O3 -fPIC -MD -MQ plat.cpython-311-x86_64-linux-gnu.so.p/plat.c.o -MF plat.cpython-311-x86_64-linux-gnu.so.p/plat.c.o.d -o plat.cpython-311-x86_64-linux-gnu.so.p/plat.c.o -c ../plat.c\n') == False
[   68s] 
[   68s] /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/test_options.py:37: AssertionError
[   68s] ----------------------------- Captured stdout call -----------------------------
[   68s] + meson setup --reconfigure /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/purelib-and-platlib /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/purelib-and-platlib/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md -Dbuildtype=debug --native-file=/home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/purelib-and-platlib/build/meson-python-native-file.ini
[   68s] The Meson build system
[   68s] Version: 1.8.0
[   68s] Source dir: /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/purelib-and-platlib
[   68s] Build dir: /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/purelib-and-platlib/build
[   68s] Build type: native build
[   68s] Project name: purelib-and-platlib
[   68s] Project version: 1.0.0
[   68s] C compiler for the host machine: cc (gcc 14.2.1 "cc (SUSE Linux) 14.2.1 20250220 [revision 9ffecde121af883b60bbe60d00425036bc873048]")
[   68s] C linker for the host machine: cc ld.bfd 2.43.1.20241209-7
[   68s] Host machine cpu family: x86_64
[   68s] Host machine cpu: x86_64
[   68s] Program python found: YES (/usr/bin/python3.11)
[   68s] Build targets in project: 1
[   68s] 
[   68s] purelib-and-platlib 1.0.0
[   68s] 
[   68s]   User defined options
[   68s]     Native files: /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/purelib-and-platlib/build/meson-python-native-file.ini
[   68s]     b_ndebug    : if-release
[   68s]     b_vscrt     : md
[   68s]     buildtype   : debug
[   68s] 
[   68s] Found ninja-1.12.1 at /usr/bin/ninja
[   68s] _____________________________ test_limited_api_bad _____________________________
[   68s] 
[   68s] package_limited_api = PosixPath('/home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api')
[   68s] tmp_path = PosixPath('/tmp/pytest-of-abuild/pytest-0/test_limited_api_bad0')
[   68s] 
[   68s]     @pytest.mark.skipif(MESON_VERSION < (1, 2, 99), reason='Meson version too old')
[   68s]     @pytest.mark.skipif(NOGIL_BUILD, reason='Free-threaded CPython does not support the limited API')
[   68s]     @pytest.mark.xfail('__pypy__' in sys.builtin_module_names, reason='PyPy does not use special modules suffix for stable ABI')
[   68s]     def test_limited_api_bad(package_limited_api, tmp_path):
[   68s] >       with pytest.raises(mesonpy.BuildError, match='The package declares compatibility with Python limited API but '):
[   68s] E       Failed: DID NOT RAISE <class 'mesonpy.BuildError'>
[   68s] 
[   68s] /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/test_wheel.py:307: Failed
[   68s] ----------------------------- Captured stdout call -----------------------------
[   68s] + meson setup --reconfigure /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md -Dextra=true --native-file=/home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api/build/meson-python-native-file.ini
[   68s] Cleaning... 0 files.
[   68s] The Meson build system
[   68s] Version: 1.8.0
[   68s] Source dir: /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api
[   68s] Build dir: /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api/build
[   68s] Build type: native build
[   68s] WARNING: Project does not target a minimum version but uses feature introduced in '1.1': meson.options file. Use meson_options.txt instead
[   68s] Project name: limited-api
[   68s] Project version: 1.0.0
[   68s] C compiler for the host machine: cc (gcc 14.2.1 "cc (SUSE Linux) 14.2.1 20250220 [revision 9ffecde121af883b60bbe60d00425036bc873048]")
[   68s] C linker for the host machine: cc ld.bfd 2.43.1.20241209-7
[   68s] Host machine cpu family: x86_64
[   68s] Host machine cpu: x86_64
[   68s] Program python found: YES (/usr/bin/python3.11)
[   68s] ../meson.build:9: WARNING: Project does not target a minimum version but uses feature introduced in '1.3.0': limited_api arg in python.extension_module.
[   68s] Build targets in project: 1
[   68s] WARNING: Project specifies no minimum version but uses features which were added in versions:
 [   68s]  * 1.1: {'meson.options file'}
[   68s]  * 1.3.0: {'limited_api arg in python.extension_module'}
[   68s] 
[   68s] limited-api 1.0.0
[   68s] 
[   68s]   User defined options
[   68s]     Native files: /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api/build/meson-python-native-file.ini
[   68s]     b_ndebug    : if-release
[   68s]     b_vscrt     : md
[   68s]     buildtype   : release
[   68s]     extra       : true
[   68s] 
[   68s] Found ninja-1.12.1 at /usr/bin/ninja
[   68s] + /usr/bin/ninja
[   68s] ninja: no work to do.
[   68s] [1/1] /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api/build/module.abi3.so
[   68s] __________________________ test_limited_api_disabled ___________________________
[   68s] 
[   68s] package_limited_api = PosixPath('/home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api')
[   68s] tmp_path = PosixPath('/tmp/pytest-of-abuild/pytest-0/test_limited_api_disabled0')
[   68s] 
[   68s]     @pytest.mark.skipif(MESON_VERSION < (1, 2, 99), reason='Meson version too old')
[   68s]     def test_limited_api_disabled(package_limited_api, tmp_path):
[   68s]         filename = mesonpy.build_wheel(tmp_path, {'setup-args': ['-Dpython.allow_limited_api=false']})
[   68s]         artifact = wheel.wheelfile.WheelFile(tmp_path / filename)
[   68s]         name = artifact.parsed_filename
[   68s]         assert name.group('pyver') == INTERPRETER
[   68s] >       assert name.group('abi') == ABI
[   68s] E       AssertionError: assert 'abi3' == 'cp311'
[   68s] E         
[   68s] E         - cp311
[   68s] E         + abi3
[   68s] 
[   68s] /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/test_wheel.py:319: AssertionError
[   68s] ----------------------------- Captured stdout call -----------------------------
[   68s] + meson setup --reconfigure /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md -Dpython.allow_limited_api=false --native-file=/home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api/build/meson-python-native-file.ini
[   68s] Cleaning... 0 files.
[   68s] The Meson build system
[   68s] Version: 1.8.0
[   68s] Source dir: /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api
[   68s] Build dir: /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api/build
[   68s] Build type: native build
[   68s] WARNING: Project does not target a minimum version but uses feature introduced in '1.1': meson.options file. Use meson_options.txt instead
[   68s] Project name: limited-api
[   68s] Project version: 1.0.0
[   68s] C compiler for the host machine: cc (gcc 14.2.1 "cc (SUSE Linux) 14.2.1 20250220 [revision 9ffecde121af883b60bbe60d00425036bc873048]")
[   68s] C linker for the host machine: cc ld.bfd 2.43.1.20241209-7
[   68s] Host machine cpu family: x86_64
[   68s] Host machine cpu: x86_64
[   68s] Program python found: YES (/usr/bin/python3.11)
[   68s] ../meson.build:9: WARNING: Project does not target a minimum version but uses feature introduced in '1.3.0': limited_api arg in python.extension_module.
[   68s] Build targets in project: 1
[   68s] WARNING: Project specifies no minimum version but uses features which were added in versions:
[   68s]  * 1.1: {'meson.options file'}
[   68s]  * 1.3.0: {'limited_api arg in python.extension_module'}
[   68s] 
[   68s] limited-api 1.0.0
[   68s] 
[   68s]   User defined options
[   68s]     Native files            : /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api/build/meson-python-native-file.ini
[   68s]     b_ndebug                : if-release
[   68s]     b_vscrt                 : md
[   68s]     buildtype               : release
[   68s]     extra                   : true
[   68s]     python.allow_limited_api: false
[   68s] 
[   68s] Found ninja-1.12.1 at /usr/bin/ninja
[   68s] + /usr/bin/ninja
[   68s] ninja: no work to do.
[   68s] [1/1] /home/abuild/rpmbuild/BUILD/python-meson-python-0.17.1-build/meson_python-0.17.1/tests/packages/limited-api/build/module.abi3.so
[   68s] =========================== short test summary info ============================
[   68s] SKIPPED [1] tests/test_project.py:204: Requires MSVC
[   68s] SKIPPED [3] tests/test_project.py:216: macOS specific test
[   68s] SKIPPED [2] tests/test_project.py:233: macOS specific test
[   68s] SKIPPED [1] tests/test_tags.py:47: macOS specific test
[   68s] SKIPPED [1] tests/test_tags.py:59: macOS specific test
[   68s] SKIPPED [1] tests/test_tags.py:69: macOS specific test
[   68s] SKIPPED [2] tests/test_wheel.py:251: macOS specific test
[   68s] FAILED tests/test_options.py::test_ndebug[-Dbuildtype=debug] - AssertionError...
[   68s] FAILED tests/test_wheel.py::test_limited_api_bad - Failed: DID NOT RAISE <cla...
[   68s] FAILED tests/test_wheel.py::test_limited_api_disabled - AssertionError: asser...
[   68s] =========== 3 failed, 108 passed, 11 skipped, 1 deselected in 55.22s ===========
[   68s] error: Bad exit status from /var/tmp/rpm-tmp.IPPPF1 (%check)

Originally posted by @MeggyCal in #722

[dnicolodi]

these test failures cannot be reproduced. Meson has no dependencies outside the Python standard library, thus just upgrading Meson to 1.8.0 should not bring any other change that may affect the tests. The test failures you report are all related to project option handling in Meson. Meson 1.8.0 had a large refactoring of this area. Are you sure that what you think is Meson 1.8.0 is really it as released upstream?

[dnicolodi]

openSUSE specific patches are visible here https://build.opensuse.org/package/show/devel:tools:building/meson

Adapted from comment by @dimstar in #722

[dnicolodi]

You won't believe it, but the difference is that when we build an RPM package, with the goal of reproducible builds, there is an env variable set, like:

SOURCE_DATE_EPOCH=1732060800 (value can vary, of course)

Removing that var inside the build env 'solves' it - setting that var outside of the buildenv results in the failure. That is quite unexpected

Originally posted by @DimStar77 in #722

[dnicolodi]

Thanks for tracking this down. But I still cannot reproduce the issue. Setting SOURCE_DATE_EPOCH=1732060800 or any other value while running the meson-pyhton tests has no effect on the outcome for me. Can you please provide the detailed steps necessary to reproduce the issue?