Singular 4.4.1 + Flint 3.3.2

[dimpase]

Update flint and singular.
This will catch up with Conda and Homebrew, possibly other distros

(Homebrew needs Homebrew/homebrew-core#222142 merged)
EDIT Homebrew/homebrew-core#222142 has been merged, so now you can install singular from Homebrew and use it!

📝 Checklist

• The title is concise and informative.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation and checked the documentation preview.

⌛ Dependencies

#39943: - just not to build patch from source (if you have systemwide patch 2.8)

#39977: - patches to allow gcc-15, etc

[JohnCremona]

I'll need some instructions on what reviewing this will entail. I just updated FLINT on my computer to 3.2.3 and am building develop from scratch, using that, so can start from there tomorrow.

[behackl]

I'll close #39967 in favor of this PR. Have not checked singular-4.4.2 yet, but I'd be optimistic that the build issue with clang17 resolved with the upgrade to singular-4.4.1 is (still) resolved with singular-4.4.2.

[dimpase]

I'll need some instructions on what reviewing this will entail. I just updated FLINT on my computer to 3.2.3 and am building develop from scratch, using that, so can start from there tomorrow.

The usual:

git pull pull/40033/head
make distclean # maybe an overkill
./configure  --disable-notebook --with-system-flint=no # 1st parameter is optional, 2nd makes sure you build flint+singular from source
make -j42 ptestlong # or not 42, but...

[dimpase]

I'll close #39967 in favor of this PR. Have not checked singular-4.4.2 yet, but I'd be optimistic that the build issue with clang17 resolved with the upgrade to singular-4.4.1 is (still) resolved with singular-4.4.2.

Not sure what 4.4.2 is. Note that https://github.com/Singular/Singular/tags is still on 4.4.1.

[behackl]

Ah, your PR title has Singular 4.4.2, and I didn't actually check the upstream sources or the diff here. 😄

[dimpase]

Ah, your PR title has Singular 4.4.2, and I didn't actually check the upstream sources or the diff here. 😄

thanks for pointing this out. Fixed

[dcoudert]

with this PR and using --with-system-flint=no, I can compile flint and singular.

[JohnCremona]

@dimpase I am following your instructions (except with make -j7 on my laptop) but the build has not yet finished.

[JohnCremona]

My build seems to have frozen in the middle of installing maxima. The last lines are

[maxima-5.47.0] [spkg-install] ;;; End of Pass 1.
[maxima-5.47.0] [spkg-install] ;;; Finished compiling /home/john/sage/local/var/tmp/sage/build/maxima-5.47.0/src/src/comm.lisp.
[maxima-5.47.0] [spkg-install] ;;;

and it has not moved for a long time, and there are no relevant processes running. I will kill it and restart.

[dimpase]

I've done a PR to fix Homebrew's singular build to link to flint - this should allow it to work with Sage
Homebrew/homebrew-core#222142

[enriqueartal]

With this PR it is possible to build singular with gcc 15 in fedora 42

[tobiasdiez]

CI is failing. Looks like it still refers to the old packages?!

ImportError: libflint.so.19: cannot open shared object file: No such file or directory

[enriqueartal]

I had this problem in one installation, rebuilding packages depending on flint solved the problem

[JohnCremona]

My build seems to have frozen in the middle of installing maxima. The last lines are

[maxima-5.47.0] [spkg-install] ;;; End of Pass 1. [maxima-5.47.0] [spkg-install] ;;; Finished compiling /home/john/sage/local/var/tmp/sage/build/maxima-5.47.0/src/src/comm.lisp. [maxima-5.47.0] [spkg-install] ;;;

and it has not moved for a long time, and there are no relevant processes running. I will kill it and restart.

I did that and this time it failed at this point:
[contourpy-1.3.1] [spkg-install] ninja: build stopped: subcommand failed.
[contourpy-1.3.1] [spkg-install]
[contourpy-1.3.1] [spkg-install] ERROR Backend subprocess exited when trying to invoke build_wheel
[contourpy-1.3.1] [spkg-install] ******************************************************************************************************************************************************
[contourpy-1.3.1] [spkg-install] Error building a wheel for contourpy-1.3.1
[contourpy-1.3.1] [spkg-install] ******************************************************************************************************************************************************
[contourpy-1.3.1] ************************************************************************
[contourpy-1.3.1] Error installing package contourpy-1.3.1
[contourpy-1.3.1] ************************************************************************
[contourpy-1.3.1] Please email sage-devel (http://groups.google.com/group/sage-devel)
[contourpy-1.3.1] explaining the problem and including the log files
[contourpy-1.3.1] /home/john/sage/logs/pkgs/contourpy-1.3.1.log
[contourpy-1.3.1] and
[contourpy-1.3.1] /home/john/sage/config.log
[contourpy-1.3.1] Describe your computer, operating system, etc.
[contourpy-1.3.1] If you want to try to fix the problem yourself, don't just cd to
[contourpy-1.3.1] /home/john/sage/local/var/lib/sage/venv-python3.12/var/tmp/sage/build/contourpy-1.3.1 and type 'make' or whatever is appropriate.
[contourpy-1.3.1] Instead, the following commands setup all environment variables
[contourpy-1.3.1] correctly and load a subshell for you to debug the error:
[contourpy-1.3.1] (cd '/home/john/sage/local/var/lib/sage/venv-python3.12/var/tmp/sage/build/contourpy-1.3.1' && '/home/john/sage/sage' --buildsh)
[contourpy-1.3.1] When you are done debugging, you can type "exit" to leave the subshell.

[dimpase]

added #39977 to the dependencies - not merged in

[dimpase]

CI is failing. Looks like it still refers to the old packages?!

ImportError: libflint.so.19: cannot open shared object file: No such file or directory

while you at are it, could you fix a Fedora 42 CI run?

[orlitzky]

CI is failing. Looks like it still refers to the old packages?!

ImportError: libflint.so.19: cannot open shared object file: No such file or directory

I don't think the sage-distro build system can handle soname changes. I used this masterpiece to get the CI to pass on the last GAP upgrade.

[dimpase]

are such things fixed in meson build?

[tobiasdiez]

are such things fixed in meson build?

Yes, I think meson (or better ninja) will correctly recognize when you upgrade an external dependency and recompile the cython files.

[tobiasdiez]

CI is failing. Looks like it still refers to the old packages?!

ImportError: libflint.so.19: cannot open shared object file: No such file or directory

I don't think the sage-distro build system can handle soname changes. I used this masterpiece to get the CI to pass on the last GAP upgrade.

Something like this is now needed again to fix the CI. It's broken currently for all PRs.

[dimpase]

Something like this is now needed again to fix the CI. It's broken currently for all PRs.

how far are we from the point where we can switch to meson?

[tobiasdiez]

Something like this is now needed again to fix the CI. It's broken currently for all PRs.

how far are we from the point where we can switch to meson?

Depends on what you want. If you just care about the long tests, then we can easily migrate them over to meson (on conda). Sage-the-distro changes would still be covered by CI using the "CI-incremental" runs. If you want to completely replace sagelib in sage-the-distro by the meson build, then things are more complicated.

[dimpase]

Something like this is now needed again to fix the CI. It's broken currently for all PRs.

how far are we from the point where we can switch to meson?

Depends on what you want. If you just care about the long tests, then we can easily migrate them over to meson (on conda). Sage-the-distro changes would still be covered by CI using the "CI-incremental" runs. If you want to completely replace sagelib in sage-the-distro by the meson build, then things are more complicated.

Ideally, the latter - this would alleviate this soname change issue all together.

[tobiasdiez]

Something like this is now needed again to fix the CI. It's broken currently for all PRs.

how far are we from the point where we can switch to meson?

Depends on what you want. If you just care about the long tests, then we can easily migrate them over to meson (on conda). Sage-the-distro changes would still be covered by CI using the "CI-incremental" runs. If you want to completely replace sagelib in sage-the-distro by the meson build, then things are more complicated.

Ideally, the latter - this would alleviate this soname change issue all together.

Okay, moving sage-the-distro to use meson was/is the point of #39030. But I'm not sure if this path is really a good way forward. I think it makes more sense to add a few missing, major dependencies (ecl, maxima, gap, singular) as meson subprojects, so that a pip install . works on most recent enough systems (eg as in #40038, #40071). Then just retire sage-the-distro completely. Makes more sense to me than fighting the uphill battle to reduce sage-the-distro to a manageable level.

[dimpase]

I am not sure I follow - I meant a sagelib build, moving it from setuptools to meson.
sage the distro is another story.

Or you mean that in such a setting meson won't be able to do a rebuild based on a soname change?

[tobiasdiez]

My point is that Meson is a complete build system with built-in support for building missing dependencies on-the-fly (see Meson documentation on subprojects). This means there's no need for something like sage-the-distro, because Meson can automatically build any missing dependencies during the build process.

If we combine sage-the-distro with Meson (e.g., by migrating sagelib to use Meson), we end up duplicating a lot of work, like resolving the correct dependencies, across both build systems.

Proposed migration path:

1. Add a few more subprojects to the Meson build, so it works on all reasonably recent systems using only system packages.
2. Once that’s achieved, make Meson the default installation method (using system packages on modern systems and falling back to Conda for older systems).
3. Migrate key optional Sage packages that are hard to install via system packages (mostly database-related ones) to Python packages.
4. Remove sage-the-distro entirely.

[dimpase]

Well, one way or another, meson build for sagelib has to check/resolve dependencies, isn't it?

And it should be environment-agnostic, in the sense it should not matter whether the dependencies come from Conda, or from OS, or are manually installed in /home/user/foobar and made available via pkg-config, isn't it?

I don't understand what duplication you mean.

I fear that this "installing on the fly" leads us into the same trap as sage the distro has led us into.

[dimpase]

@tobiasdiez - to elaborate a bit on my previous comment - I think it's important from the design modularity point of view, as there are uses of sagelib where no on the fly rebuilding of dependencies is needed.

Dependencies can come from another meson project, which can be thought of as a replacement of sage the distro (slimmed down a lot, of course).

[tobiasdiez]

I don't understand what duplication you mean.

The duplication would be that the configure scripts try to detect the correct dependencies - and in a second step meson will do it again to build sagelib. But both detection mechanisms are different, so either one might reject a certain dependency that the other build system would accept; and vice-versa. Also you would need to make sure that meson is able to find the dependencies that sage-the-distro build.

I think it's important from the design modularity point of view, as there are uses of sagelib where no on the fly rebuilding of dependencies is needed.

I completely agree! Meson actually works in such a modular way: it only builds a dependency, if no suitable version can be found (either as a system package, or conda, or manually installed). The whole subproject mechanism can also be disabled, which is important for downstream distros which need to provide all dependencies as separate packages. Meson's subprojects are very similar to the existing sage-the-distro except that the whole infrastructure is built into meson (so rather stable) and will be run as part of pip install. As an example, see https://github.com/sagemath/sage/actions/runs/15087133885/job/42411108409?pr=40071#step:7:632 which runs on Fedora 42 where no recent ecl is available. Meson detects this correctly and then builds ecl (based on a git checkout of the upstream repo, using autoconf/make) before compiling sage.
As long as we don't repeat the mistake to insist that the whole dependency tree needs to be available as meson subprojects, this hopefully doesn't require too much maintenance.

[dimpase]

I don't understand what duplication you mean.

The duplication would be that the configure scripts try to detect the correct dependencies - and in a second step meson will do it again to build sagelib. But both detection mechanisms are different, so either one might reject a certain dependency that the other build system would accept; and vice-versa. Also you would need to make sure that meson is able to find the dependencies that sage-the-distro build.

I think it would actually be a good step in switching to meson to start with sagelib. This would make sure that meson detections are done in such a seemingly unusual setting as sage's venv with its Sage-built things mixed with system-wide packages - and extra robustness. Is this a lot of work?

I think it's important from the design modularity point of view, as there are uses of sagelib where no on the fly rebuilding of dependencies is needed.

I completely agree! Meson actually works in such a modular way: it only builds a dependency, if no suitable version can be found (either as a system package, or conda, or manually installed). The whole subproject mechanism can also be disabled, which is important for downstream distros which need to provide all dependencies as separate packages.
As long as we don't repeat the mistake to insist that the whole dependency tree needs to be available as meson subprojects, this hopefully doesn't require too much maintenance.

How about tarballs/repos to install deps on the fly? Would it still be a variation of sage mirrors, or, apart from very few packages, this can go?

[tobiasdiez]

I don't understand what duplication you mean.

The duplication would be that the configure scripts try to detect the correct dependencies - and in a second step meson will do it again to build sagelib. But both detection mechanisms are different, so either one might reject a certain dependency that the other build system would accept; and vice-versa. Also you would need to make sure that meson is able to find the dependencies that sage-the-distro build.

I think it would actually be a good step in switching to meson to start with sagelib. This would make sure that meson detections are done in such a seemingly unusual setting as sage's venv with its Sage-built things mixed with system-wide packages - and extra robustness. Is this a lot of work?

If sage-the-distro sets the correct compiler flags and/or installs pkgconfig files in a place where meson can find them, then it should work.

I think it's important from the design modularity point of view, as there are uses of sagelib where no on the fly rebuilding of dependencies is needed.

I completely agree! Meson actually works in such a modular way: it only builds a dependency, if no suitable version can be found (either as a system package, or conda, or manually installed). The whole subproject mechanism can also be disabled, which is important for downstream distros which need to provide all dependencies as separate packages.
As long as we don't repeat the mistake to insist that the whole dependency tree needs to be available as meson subprojects, this hopefully doesn't require too much maintenance.

How about tarballs/repos to install deps on the fly? Would it still be a variation of sage mirrors, or, apart from very few packages, this can go?

You can specify an url where meson will download the tarball. Normally that would be the upstream release. Or you specify a git repo (+ branch/commit/tag). So, except if the hosting of the tarball is very unreliable, I don't see the need for a sage mirror. There is also an official db with such subprojects (https://mesonbuild.com/Wrapdb-projects.html), but I think no math project are in there yet.

[dimpase]

installs pkgconfig files in a place where meson can find them

does it mean the usual calling of pkg-config done by meson, or is it different, in the sense that it's a different API?

[tobiasdiez]

installs pkgconfig files in a place where meson can find them

does it mean the usual calling of pkg-config done by meson, or is it different, in the sense that it's a different API?

The usual call done by meson.

[dimpase]

OK, so what does stop us from switching sagelib build to meson now?

[tobiasdiez]

Someone needs to finish #39030.

[dimpase]

I have a feeling it's got to be me :-)
Could you rebase what you had there over the current stuff?