Merge pull request #10497 from pradyunsg/docs/build-system-reference

Add a dedicated "Build system interface" reference section

Author: pradyunsg

docs/html/cli/pip.rst

@@ -74,178 +74,11 @@ when decision is needed.
 *(a)abort*
     Abort pip and return non-zero exit status.
 
-.. _`build-interface`:
-
 
 Build System Interface
 ======================
 
-pip builds packages by invoking the build system. By default, builds will use
-``setuptools``, but if a project specifies a different build system using a
-``pyproject.toml`` file, as per :pep:`517`, pip will use that instead.  As well
-as package building, the build system is also invoked to install packages
-direct from source.  This is handled by invoking the build system to build a
-wheel, and then installing from that wheel.  The built wheel is cached locally
-by pip to avoid repeated identical builds.
-
-The current interface to the build system is via the ``setup.py`` command line
-script - all build actions are defined in terms of the specific ``setup.py``
-command line that will be run to invoke the required action.
-
-Setuptools Injection
-~~~~~~~~~~~~~~~~~~~~
-
-When :pep:`517` is not used, the supported build system is ``setuptools``.
-However, not all packages use ``setuptools`` in their build scripts. To support
-projects that use "pure ``distutils``", pip injects ``setuptools`` into
-``sys.modules`` before invoking ``setup.py``. The injection should be
-transparent to ``distutils``-based projects, but 3rd party build tools wishing
-to provide a ``setup.py`` emulating the commands pip requires may need to be
-aware that it takes place.
-
-Projects using :pep:`517` *must* explicitly use setuptools - pip does not do
-the above injection process in this case.
-
-Build System Output
-~~~~~~~~~~~~~~~~~~~
-
-Any output produced by the build system will be read by pip (for display to the
-user if requested). In order to correctly read the build system output, pip
-requires that the output is written in a well-defined encoding, specifically
-the encoding the user has configured for text output (which can be obtained in
-Python using ``locale.getpreferredencoding``). If the configured encoding is
-ASCII, pip assumes UTF-8 (to account for the behaviour of some Unix systems).
-
-Build systems should ensure that any tools they invoke (compilers, etc) produce
-output in the correct encoding. In practice - and in particular on Windows,
-where tools are inconsistent in their use of the "OEM" and "ANSI" codepages -
-this may not always be possible. pip will therefore attempt to recover cleanly
-if presented with incorrectly encoded build tool output, by translating
-unexpected byte sequences to Python-style hexadecimal escape sequences
-(``"\x80\xff"``, etc). However, it is still possible for output to be displayed
-using an incorrect encoding (mojibake).
-
-Under :pep:`517`, handling of build tool output is the backend's responsibility,
-and pip simply displays the output produced by the backend. (Backends, however,
-will likely still have to address the issues described above).
-
-PEP 517 and 518 Support
-~~~~~~~~~~~~~~~~~~~~~~~
-
-As of version 10.0, pip supports projects declaring dependencies that are
-required at install time using a ``pyproject.toml`` file, in the form described
-in :pep:`518`. When building a project, pip will install the required
-dependencies locally, and make them available to the build process.
-Furthermore, from version 19.0 onwards, pip supports projects specifying the
-build backend they use in ``pyproject.toml``, in the form described in
-:pep:`517`.
-
-When making build requirements available, pip does so in an *isolated
-environment*. That is, pip does not install those requirements into the user's
-``site-packages``, but rather installs them in a temporary directory which it
-adds to the user's ``sys.path`` for the duration of the build. This ensures
-that build requirements are handled independently of the user's runtime
-environment. For example, a project that needs a recent version of setuptools
-to build can still be installed, even if the user has an older version
-installed (and without silently replacing that version).
-
-In certain cases, projects (or redistributors) may have workflows that
-explicitly manage the build environment. For such workflows, build isolation
-can be problematic. If this is the case, pip provides a
-``--no-build-isolation`` flag to disable build isolation. Users supplying this
-flag are responsible for ensuring the build environment is managed
-appropriately (including ensuring that all required build dependencies are
-installed).
-
-By default, pip will continue to use the legacy (direct ``setup.py`` execution
-based) build processing for projects that do not have a ``pyproject.toml`` file.
-Projects with a ``pyproject.toml`` file will use a :pep:`517` backend. Projects
-with a ``pyproject.toml`` file, but which don't have a ``build-system`` section,
-will be assumed to have the following backend settings::
-
-    [build-system]
-    requires = ["setuptools>=40.8.0", "wheel"]
-    build-backend = "setuptools.build_meta:__legacy__"
-
-.. note::
-
-    ``setuptools`` 40.8.0 is the first version of setuptools that offers a
-    :pep:`517` backend that closely mimics directly executing ``setup.py``.
-
-If a project has ``[build-system]``, but no ``build-backend``, pip will also use
-``setuptools.build_meta:__legacy__``, but will expect the project requirements
-to include ``setuptools`` and ``wheel`` (and will report an error if the
-installed version of ``setuptools`` is not recent enough).
-
-If a user wants to explicitly request :pep:`517` handling even though a project
-doesn't have a ``pyproject.toml`` file, this can be done using the
-``--use-pep517`` command line option. Similarly, to request legacy processing
-even though ``pyproject.toml`` is present, the ``--no-use-pep517`` option is
-available (although obviously it is an error to choose ``--no-use-pep517`` if
-the project has no ``setup.py``, or explicitly requests a build backend). As
-with other command line flags, pip recognises the ``PIP_USE_PEP517``
-environment veriable and a ``use-pep517`` config file option (set to true or
-false) to set this option globally. Note that overriding pip's choice of
-whether to use :pep:`517` processing in this way does *not* affect whether pip
-will use an isolated build environment (which is controlled via
-``--no-build-isolation`` as noted above).
-
-Except in the case noted above (projects with no :pep:`518` ``[build-system]``
-section in ``pyproject.toml``), pip will never implicitly install a build
-system. Projects **must** ensure that the correct build system is listed in
-their ``requires`` list (this applies even if pip assumes that the
-``setuptools`` backend is being used, as noted above).
-
-.. _pep-518-limitations:
-
-**Historical Limitations**:
-
-* ``pip<18.0``: only supports installing build requirements from wheels, and
-  does not support the use of environment markers and extras (only version
-  specifiers are respected).
-
-* ``pip<18.1``: build dependencies using .pth files are not properly supported;
-  as a result namespace packages do not work under Python 3.2 and earlier.
-
-Future Developments
-~~~~~~~~~~~~~~~~~~~
-
-:pep:`426` notes that the intention is to add hooks to project metadata in
-version 2.1 of the metadata spec, to explicitly define how to build a project
-from its source. Once this version of the metadata spec is final, pip will
-migrate to using that interface. At that point, the ``setup.py`` interface
-documented here will be retained solely for legacy purposes, until projects
-have migrated.
-
-Specifically, applications should *not* expect to rely on there being any form
-of backward compatibility guarantees around the ``setup.py`` interface.
-
-
-Build Options
-~~~~~~~~~~~~~
-
-The ``--global-option`` and ``--build-option`` arguments to the ``pip install``
-and ``pip wheel`` inject additional arguments into the ``setup.py`` command
-(``--build-option`` is only available in ``pip wheel``).  These arguments are
-included in the command as follows:
-
-.. tab:: Unix/macOS
-
-    .. code-block:: console
-
-        python setup.py <global_options> BUILD COMMAND <build_options>
-
-.. tab:: Windows
-
-    .. code-block:: shell
-
-        py setup.py <global_options> BUILD COMMAND <build_options>
-
-The options are passed unmodified, and presently offer direct access to the
-distutils command line. Use of ``--global-option`` and ``--build-option``
-should be considered as build system dependent, and may not be supported in the
-current form if support for alternative build systems is added to pip.
-
+This is now covered in :doc:`../reference/build-system/index`.
 
 .. _`General Options`:
 

docs/html/cli/pip_install.rst

@@ -419,7 +419,7 @@ with other repeatability strategies.
    (rare) packages that use it will cause those dependencies to be downloaded
    by setuptools directly, skipping pip's hash-checking. If you need to use
    such a package, see :ref:`Controlling
-   setup_requires<controlling-setup-requires>`.
+   setup_requires <controlling-setup_requires>`.
 
 .. warning::
 
@@ -537,83 +537,10 @@ the project path.  This is one advantage over just using ``setup.py develop``,
 which creates the "egg-info" directly relative the current working directory.
 
 
-.. _`controlling-setup-requires`:
-
-Controlling setup_requires
---------------------------
-
-Setuptools offers the ``setup_requires`` `setup() keyword
-<https://setuptools.readthedocs.io/en/latest/userguide/keywords.html>`_
-for specifying dependencies that need to be present in order for the
-``setup.py`` script to run.  Internally, Setuptools uses ``easy_install``
-to fulfill these dependencies.
-
-pip has no way to control how these dependencies are located.  None of the
-package index options have an effect.
-
-The solution is to configure a "system" or "personal" `Distutils configuration
-file
-<https://docs.python.org/3/install/index.html#distutils-configuration-files>`_ to
-manage the fulfillment.
-
-For example, to have the dependency located at an alternate index, add this:
-
-::
-
-  [easy_install]
-  index_url = https://my.index-mirror.com
-
-To have the dependency located from a local directory and not crawl PyPI, add this:
-
-::
-
-  [easy_install]
-  allow_hosts = ''
-  find_links = file:///path/to/local/archives/
-
-
 Build System Interface
 ----------------------
 
-In order for pip to install a package from source, ``setup.py`` must implement
-the following commands::
-
-    setup.py egg_info [--egg-base XXX]
-    setup.py install --record XXX [--single-version-externally-managed] [--root XXX] [--compile|--no-compile] [--install-headers XXX]
-
-The ``egg_info`` command should create egg metadata for the package, as
-described in the setuptools documentation at
-https://setuptools.readthedocs.io/en/latest/userguide/commands.html#egg-info-create-egg-metadata-and-set-build-tags
-
-The ``install`` command should implement the complete process of installing the
-package to the target directory XXX.
-
-To install a package in "editable" mode (``pip install -e``), ``setup.py`` must
-implement the following command::
-
-    setup.py develop --no-deps
-
-This should implement the complete process of installing the package in
-"editable" mode.
-
-All packages will be attempted to built into wheels::
-
-    setup.py bdist_wheel -d XXX
-
-One further ``setup.py`` command is invoked by ``pip install``::
-
-    setup.py clean
-
-This command is invoked to clean up temporary commands from the build. (TODO:
-Investigate in more detail when this command is required).
-
-No other build system commands are invoked by the ``pip install`` command.
-
-Installing a package from a wheel does not invoke the build system at all.
-
-.. _PyPI: https://pypi.org/
-.. _setuptools extras: https://setuptools.readthedocs.io/en/latest/userguide/dependency_management.html#optional-dependencies
-
+This is now covered in :doc:`../reference/build-system/index`.
 
 
 .. _`pip install Options`:
@@ -906,3 +833,6 @@ Examples
 
 .. [1] This is true with the exception that pip v7.0 and v7.0.1 required quotes
        around specifiers containing environment markers in requirement files.
+
+.. _setuptools extras: https://setuptools.readthedocs.io/en/latest/userguide/dependency_management.html#optional-dependencies
+.. _PyPI: https://pypi.org/

docs/html/cli/pip_wheel.rst

@@ -28,60 +28,7 @@ Description
 Build System Interface
 ----------------------
 
-In order for pip to build a wheel, ``setup.py`` must implement the
-``bdist_wheel`` command with the following syntax:
-
-.. tab:: Unix/macOS
-
-   .. code-block:: shell
-
-      python setup.py bdist_wheel -d TARGET
-
-.. tab:: Windows
-
-   .. code-block:: shell
-
-      py setup.py bdist_wheel -d TARGET
-
-
-This command must create a wheel compatible with the invoking Python
-interpreter, and save that wheel in the directory TARGET.
-
-No other build system commands are invoked by the ``pip wheel`` command.
-
-Customising the build
-^^^^^^^^^^^^^^^^^^^^^
-
-It is possible using ``--global-option`` to include additional build commands
-with their arguments in the ``setup.py`` command. This is currently the only
-way to influence the building of C extensions from the command line. For
-example:
-
-.. tab:: Unix/macOS
-
-   .. code-block:: shell
-
-      python -m pip wheel --global-option bdist_ext --global-option -DFOO wheel
-
-.. tab:: Windows
-
-   .. code-block:: shell
-
-      py -m pip wheel --global-option bdist_ext --global-option -DFOO wheel
-
-
-will result in a build command of
-
-::
-
-    setup.py bdist_ext -DFOO bdist_wheel -d TARGET
-
-which passes a preprocessor symbol to the extension build.
-
-Such usage is considered highly build-system specific and more an accident of
-the current implementation than a supported interface.
-
-
+This is now covered in :doc:`../reference/build-system/index`.
 
 Options
 =======