Skip to content

Add note about makeflags and ninja parallelism #39128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Feb 21, 2025
18 changes: 12 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -334,12 +334,18 @@ in the Installation Guide.
11. Optional, but highly recommended: Set some environment variables to
customize the build.

For example, the `MAKE` environment variable controls whether to
run several jobs in parallel. On a machine with 4 processors, say,
typing `export MAKE="make -j4"` will configure the build script to
perform a parallel compilation of Sage using 4 jobs. On some
powerful machines, you might even consider `-j16`, as building with
more jobs than CPU cores can speed things up further.
The `MAKEFLAGS` variable controls whether to run several jobs in parallel.
To saturate all the execution threads of your CPU, we recommend to run
`export MAKEFLAGS="-j$(nproc) -l$(nproc).5"` if you are on Linux, and
`export MAKEFLAGS="-j$(sysctl -n hw.ncpu) -l$(sysctl -n hw.ncpu).5"` if you
are on macOS.

Note that the compilation may nonetheless use a different number of
processes, e.g., for parts that are built with `ninja` which automatically
decides on the amount of parallelity to use. In practice, you might
therefore see twice as many processes during the build process than your
CPU has execution threads. Unless your system is low on RAM, this should
not affect the time the compilation takes substantially.

To reduce the terminal output during the build, type `export V=0`.
(`V` stands for "verbosity".)
Expand Down
2 changes: 1 addition & 1 deletion src/doc/en/faq/faq-usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ Sage. The command

.. CODE-BLOCK:: shell-session

$ export MAKE='make -j8'
$ export MAKEFLAGS='-j8'

will enable 8 threads for parts of the build that support
parallelism. Change the number 8 as appropriate to suit the number of
Expand Down
33 changes: 20 additions & 13 deletions src/doc/en/installation/source.rst
Original file line number Diff line number Diff line change
Expand Up @@ -669,8 +669,8 @@ Environment variables
Sage uses several environment variables to control its build process.
Most users won't need to set any of these: the build process just works on many
platforms.
(Note though that setting :envvar:`MAKE`, as described below, can significantly
speed up the process.)
(Note though that setting :envvar:`MAKEFLAGS`, as described below, can
significantly speed up the process.)
Building Sage involves building many packages, each of which has its own
compilation instructions.

Expand All @@ -680,19 +680,26 @@ Standard environment controlling the build process

Here are some of the more commonly used variables affecting the build process:

.. envvar:: MAKE
.. envvar:: MAKEFLAGS

One useful setting for this variable when building Sage is
``MAKE='make -jNUM'`` to tell the ``make`` program to run ``NUM`` jobs in
parallel when building.
Note that some Sage packages may not support this variable.
This variable can be set to tell the ``make`` program to build things in
parallel. Set it to ``-jNUM`` to run ``NUM`` jobs in parallel when building.
Add ``-lNUM`` to tell make not to spawn more processes when the load exceeds
``NUM``.

A good value for this variable is ``MAKEFLAGS="-j$(nproc) -l$(nproc).5"`` on
Linux and ``MAKEFLAGS="-j$(sysctl -n hw.ncpu) -l$(sysctl -n hw.ncpu).5"`` on
macOS. This instructs make to use all the execution threads of your CPU while
bounding the load if there are other processes generating load. If your
system does not have a lot of RAM, you might want to choose lower limits, if
you have lots of RAM, it can sometimes be beneficial to set these limits
slightly higher.

Note that some parts of the SageMath build system do not respect this
variable, e.g., when ninja gets invoked, it figures out the number of
processes to use on its own so the number of processes and the system load
you see might exceed the number configured here.

Some people advise using more jobs than there are CPU cores, at least if the
system is not heavily loaded and has plenty of RAM; for example, a good
setting for ``NUM`` might be between 1 and 1.5 times the number of cores.
In addition, the ``-l`` option sets a load limit: ``MAKE='make -j4 -l5.5``,
for example, tells ``make`` to try to use four jobs, but to not start more
than one job if the system load average is above 5.5.
See the manual page for GNU ``make``: `Command-line options
<https://www.gnu.org/software/make/manual/make.html#Options-Summary>`_
and `Parallel building
Expand Down
2 changes: 1 addition & 1 deletion src/doc/it/faq/faq-usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ questi prerequisiti come segue::
Se hai un sistema multiprocessore puoi scegliere una
compilazione parallela di Sage. Il comando ::

export MAKE='make -j8'
export MAKEFLAGS='-j8'

abiliterà 8 threads per quelle parti della compilazione che supportano
il parallelismo. Al posto del numero 8 metti il numero di
Expand Down
Loading