Merge pull request #9044 from brainwane/docs-change-resolver-default

Author: pradyunsg

README.rst

@@ -19,7 +19,7 @@ We release updates regularly, with a new version every 3 months. Find more detai
 * `Release notes`_
 * `Release process`_
 
-In 2020, we're working on improvements to the heart of pip. Please `learn more and take our survey`_ to help us do it right.
+In pip 20.3, we're `making a big improvement to the heart of pip`_; `learn more`_. We want your input, so `sign up for our user experience research studies`_ to help us do it right.
 
 If you find bugs, need help, or want to talk to the developers, please use our mailing lists or chat rooms:
 
@@ -48,7 +48,9 @@ rooms, and mailing lists is expected to follow the `PSF Code of Conduct`_.
 .. _Release process: https://pip.pypa.io/en/latest/development/release-process/
 .. _GitHub page: https://github.com/pypa/pip
 .. _Development documentation: https://pip.pypa.io/en/latest/development
-.. _learn more and take our survey: https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year.html
+.. _making a big improvement to the heart of pip: https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year.html
+.. _learn more: https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020
+.. _sign up for our user experience research studies: https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year.html
 .. _Issue tracking: https://github.com/pypa/pip/issues
 .. _Discourse channel: https://discuss.python.org/c/packaging
 .. _Development mailing list: https://mail.python.org/mailman3/lists/distutils-sig.python.org/

docs/html/development/architecture/anatomy.rst

@@ -97,8 +97,7 @@ Within ``src/``:
 
     * ``pep425tags.py`` -- getting refactored into packaging.tags (a library on PyPI) which is external to pip (but vendored by pip). :pep:`425` tags: turns out lots of people want this! Compatibility tags for built distributions -> e.g., platform, Python version, etc.
     * ``pyproject.py`` -- ``pyproject.toml`` is a new standard (:pep:`518` and :pep:`517`). This file reads pyproject.toml and passes that info elsewhere. The rest of the processing happens in a different file. All the handling for 517 and 518 is in a different file.
-    * ``req/`` *[*\ **A DIRECTORY THAT NEEDS REFACTORING. A LOT**\ *\ …… Remember Step 3? Dependency resolution etc.? This is that step! Each file represents … have the entire flow of installing & uninstalling, getting info about packages…. Some files here are more than 1,000 lines long! (used to be longer?!) Refactor will deeply improve developer experience.]*
-    * ``resolve.py`` -- This is where the current dependency resolution algorithm sits. Pradyun is `improving the pip dependency resolver`_. Pradyun will get rid of this file and replace it with a directory called “resolution”. (this work is in git master…. There is further work that is going to be in a branch soon)
+    * ``req/`` *[*\ **A DIRECTORY THAT NEEDS REFACTORING. A LOT**\ *\ …… Remember Step 3? Dependency resolution etc.? This is that step! Each file represents … have the entire flow of installing & uninstalling, getting info about packages…. Some files here are more than 1,000 lines long! (used to be longer?!) Refactor will deeply improve developer experience. Also, we're `improving the pip dependency resolver`_ in 2020 so a bunch of this is changing.]*
     * ``utils/`` *[everything that is not “operationally” pip ….. Misc functions and files get dumped. There’s some organization here. There’s a models.py here which needs refactoring. Deprecation.py is useful, as are other things, but some things do not belong here. There ought to be some GitHub issues for refactoring some things here. Maybe a few issues with checkbox lists.]*
     * ``vcs/`` *[stands for Version Control System. Where pip handles all version control stuff -- one of the ``pip install`` arguments you can use is a version control link. Are any of these commands vendored? No, via subprocesses. For performance, it makes sense (we think) to do this instead of pygitlib2 or similar -- and has to be pure Python, can’t include C libraries, because you can’t include compiled C stuff, because you might not have it for the platform you are running on.]*
 

docs/html/development/architecture/upgrade-options.rst

@@ -6,6 +6,10 @@ When installing packages, pip chooses a distribution file, and installs it in
 the user's environment. There are many choices involved in deciding which file
 to install, and these are controlled by a variety of options.
 
+.. note::
+
+    This section of the documentation needs to be updated per
+    :ref:`Resolver changes 2020`.
 
 Controlling what gets installed
 ===============================

docs/html/development/release-process.rst

@@ -65,6 +65,8 @@ their merits.
   ``pip._internal.utils.deprecation.deprecated``. The function is not a part of
   pip's public API.
 
+.. _`Python 2 Support`:
+
 Python 2 Support
 ----------------
 

docs/html/index.rst

@@ -17,7 +17,7 @@ Please take a look at our documentation for how to install and use pip:
    ux_research_design
    news
 
-In 2020, we're working on improvements to the heart of pip: :ref:`Resolver changes 2020`. Please `learn more and take our survey`_ to help us do it right, and `join our user experience surveys pool`_.
+In pip 20.3, we're `making a big improvement to the heart of pip`_; :ref:`Resolver changes 2020`. We want your input, so `sign up for our user experience research studies`_ to help us do it right.
 
 If you find bugs, need help, or want to talk to the developers, please use our mailing lists or chat rooms:
 
@@ -40,8 +40,8 @@ rooms, and mailing lists is expected to follow the `PSF Code of Conduct`_.
 
 .. _package installer: https://packaging.python.org/guides/tool-recommendations/
 .. _Python Package Index: https://pypi.org
-.. _join our user experience surveys pool: ux_research_design
-.. _learn more and take our survey: https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year.html
+.. _making a big improvement to the heart of pip: https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year.html
+.. _sign up for our user experience research studies: https://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year.html
 .. _Installation: https://pip.pypa.io/en/stable/installing.html
 .. _Documentation: https://pip.pypa.io/en/stable/
 .. _Changelog: https://pip.pypa.io/en/stable/news.html

docs/html/user_guide.rst

@@ -197,16 +197,18 @@ In practice, there are 4 common uses of Requirements files:
          py -m pip freeze > requirements.txt
          py -m pip install -r requirements.txt
 
-2. Requirements files are used to force pip to properly resolve dependencies.
-   As it is now, pip `doesn't have true dependency resolution
-   <https://github.com/pypa/pip/issues/988>`_, but instead simply uses the first
-   specification it finds for a project. E.g. if ``pkg1`` requires
-   ``pkg3>=1.0`` and ``pkg2`` requires ``pkg3>=1.0,<=2.0``, and if ``pkg1`` is
-   resolved first, pip will only use ``pkg3>=1.0``, and could easily end up
-   installing a version of ``pkg3`` that conflicts with the needs of ``pkg2``.
-   To solve this problem, you can place ``pkg3>=1.0,<=2.0`` (i.e. the correct
-   specification) into your requirements file directly along with the other top
-   level requirements. Like so::
+2. Requirements files are used to force pip to properly resolve
+   dependencies.  In versions of pip prior to 20.3, pip `doesn't have
+   true dependency resolution
+   <https://github.com/pypa/pip/issues/988>`_, but instead simply uses
+   the first specification it finds for a project. E.g. if ``pkg1``
+   requires ``pkg3>=1.0`` and ``pkg2`` requires ``pkg3>=1.0,<=2.0``,
+   and if ``pkg1`` is resolved first, pip will only use ``pkg3>=1.0``,
+   and could easily end up installing a version of ``pkg3`` that
+   conflicts with the needs of ``pkg2``.  To solve this problem for
+   pip 20.2 and previous, you can place ``pkg3>=1.0,<=2.0`` (i.e. the
+   correct specification) into your requirements file directly along
+   with the other top level requirements. Like so::
 
      pkg1
      pkg2
@@ -1095,6 +1097,8 @@ archives are built with identical packages.
     to use such a package, see :ref:`Controlling
     setup_requires<controlling-setup-requires>`.
 
+.. _`Fixing conflicting dependencies`:
+
 Fixing conflicting dependencies
 ===============================
 
@@ -1103,8 +1107,10 @@ pip users who encounter an error where pip cannot install their
 specified packages due to conflicting dependencies (a
 ``ResolutionImpossible`` error).
 
-This documentation is specific to the new resolver, which you can use
-with the flag ``--use-feature=2020-resolver``.
+This documentation is specific to the new resolver, which is the
+default behavior in pip 20.3 and later. If you are using pip 20.2, you
+can invoke the new resolver by using the flag
+``--use-feature=2020-resolver``.
 
 Understanding your error message
 --------------------------------
@@ -1374,19 +1380,24 @@ of ability. Some examples that you could consider include:
 * ``distlib`` - Packaging and distribution utilities (including functions for
   interacting with PyPI).
 
+.. _changes-to-the-pip-dependency-resolver-in-20-2-2020:
+
 .. _`Resolver changes 2020`:
 
-Changes to the pip dependency resolver in 20.2 (2020)
+Changes to the pip dependency resolver in 20.3 (2020)
 =====================================================
 
-pip 20.1 included an alpha version of the new resolver (hidden behind
-an optional ``--unstable-feature=resolver`` flag). pip 20.2 removes
-that flag, and includes a robust beta of the new resolver (hidden
-behind an optional ``--use-feature=2020-resolver`` flag) that we
-encourage you to test. We will continue to improve the pip dependency
-resolver in response to testers' feedback. Please give us feedback
-through the `resolver testing survey`_. This will help us prepare to
-release pip 20.3, with the new resolver on by default, in October.
+pip 20.3 has a new dependency resolver, on by default. (pip 20.1 and
+20.2 included pre-release versions of the new dependency resolver,
+hidden behind optional user flags.) Read below for a migration guide,
+how to invoke the legacy resolver, and the deprecation timeline. We
+also made a `two-minute video explanation`_ you can watch.
+
+We will continue to improve the pip dependency resolver in response to
+testers' feedback. Please give us feedback through the `resolver
+testing survey`_.
+
+.. _`Migration guide for 2020 resolver changes`:
 
 Watch out for
 -------------
@@ -1441,29 +1452,26 @@ We are also changing our support for :ref:`Constraints Files`:
 * Links are not allowed as constraints (see :issue:`8253`)
 * Constraints cannot have extras (see :issue:`6628`)
 
+Per our :ref:`Python 2 Support` policy, pip 20.3 users who are using
+Python 2 will use the legacy resolver by default. Python 2 users
+should upgrade to Python 3 as soon as possible, since in pip 21.0 in
+January 2021, pip will drop support for Python 2 altogether.
 
-How to test
------------
 
-1. **Install pip 20.2** with ``python -m pip install --upgrade pip``.
+How to upgrade and migrate
+--------------------------
+
+1. **Install pip 20.3** with ``python -m pip install --upgrade pip``.
 
 2. **Validate your current environment** by running ``pip check``. This
    will report if you have any inconsistencies in your set of installed
    packages. Having a clean installation will make it much less likely
-   that you will hit issues when the new resolver is released (and may
+   that you will hit issues with the new resolver (and may
    address hidden problems in your current environment!). If you run
    ``pip check`` and run into stuff you can’t figure out, please `ask
    for help in our issue tracker or chat <https://pip.pypa.io/>`__.
 
-3. **Test the new version of pip** (see below). To test the new
-   resolver, use the ``--use-feature=2020-resolver`` flag, as in:
-
-   ``pip install example --use-feature=2020-resolver``
-
-   The more feedback we can get, the more we can make sure that the
-   final release is solid. (Only try the new resolver **in a
-   non-production environment**, though - it isn't ready for you to
-   rely on in production!)
+3. **Test the new version of pip**.
 
    While we have tried to make sure that pip’s test suite covers as
    many cases as we can, we are very aware that there are people using
@@ -1478,45 +1486,38 @@ How to test
       - using ``pip install --force-reinstall`` to check whether
         it does what you think it should
       - using constraints files
+      - the "Setups to test with special attention" and "Examples to try" below
 
    -  If you have a build pipeline that depends on pip installing your
       dependencies for you, check that the new resolver does what you
       need.
 
-   -  If you'd like pip to default to using the new resolver, run ``pip
-      config set global.use-feature 2020-resolver`` (for more on that
-      and the alternate ``PIP_USE_FEATURE`` environment variable
-      option, see `issue 8661`_).
-
    -  Run your project’s CI (test suite, build process, etc.) using the
       new resolver, and let us know of any issues.
    -  If you have encountered resolver issues with pip in the past,
-      check whether the new resolver fixes them. Also, let us know if
-      the new resolver has issues with any workarounds you put in to
-      address the current resolver’s limitations. We’ll need to ensure
-      that people can transition off such workarounds smoothly.
+      check whether the new resolver fixes them, and read :ref:`Fixing
+      conflicting dependencies`. Also, let us know if the new resolver
+      has issues with any workarounds you put in to address the
+      current resolver’s limitations. We’ll need to ensure that people
+      can transition off such workarounds smoothly.
    -  If you develop or support a tool that wraps pip or uses it to
       deliver part of your functionality, please test your integration
-      with pip 20.2.
+      with pip 20.3.
 
-4. **Please report bugs** through the `resolver testing survey`_.
+4.    **Temporarily use the old resolver when necessary.** If you run
+      into resolution errors and need a workaround while you're fixing
+      their root causes, you can choose the old resolver behavior
+      using the flag ``--use-deprecated=legacy-resolver``.
 
-Setups we might need more testing on
-------------------------------------
+5. **Please report bugs** through the `resolver testing survey`_.
 
-*    Windows, including Windows Subsystem for Linux (WSL)
 
-*    Macintosh
-
-*    Debian, Fedora, Red Hat, CentOS, Mint, Arch, Raspbian, Gentoo
-
-*    non-Latin localized filesystems and OSes, such as Japanese, Chinese, and Korean, and right-to-left such as Hebrew, Urdu, and Arabic
-
-*    Multi-user installations
+Setups to test with special attention
+-------------------------------------
 
 *    Requirements files with 100+ packages
 
-*    An installation workflow that involves multiple requirements files
+*    Installation workflows that involve multiple requirements files
 
 *    Requirements files that include hashes (:ref:`hash-checking mode`)
      or pinned dependencies (perhaps as output from ``pip-compile`` within
@@ -1530,12 +1531,6 @@ Setups we might need more testing on
 
 *    Installing from source code held in local directories
 
-*    Using the most recent versions of Python 3.6, 3.7, 3.8, and 3.9
-
-*    PyPy
-
-*    Customized terminals (where you have modified how error messages and standard output display)
-
 Examples to try
 ^^^^^^^^^^^^^^^
 
@@ -1580,15 +1575,26 @@ Specific things we'd love to get feedback on:
 
 Please let us know through the `resolver testing survey`_.
 
+.. _`Deprecation timeline for 2020 resolver changes`:
+
 Deprecation timeline
 --------------------
 
 We plan for the resolver changeover to proceed as follows, using
 :ref:`Feature Flags` and following our :ref:`Release Cadence`:
 
-*    pip 20.2: a beta of the new resolver is available, opt-in, using
-     the flag ``--use-feature=2020-resolver``. pip defaults to
-     legacy behavior.
+*    pip 20.1: an alpha version of the new resolver was available,
+     opt-in, using the optional flag
+     ``--unstable-feature=resolver``. pip defaulted to legacy
+     behavior.
+
+*    pip 20.2: a beta of the new resolver was available, opt-in, using
+     the flag ``--use-feature=2020-resolver``. pip defaulted to legacy
+     behavior. Users of pip 20.2 who want pip to default to using the
+     new resolver can run ``pip config set global.use-feature
+     2020-resolver`` (for more on that and the alternate
+     ``PIP_USE_FEATURE`` environment variable option, see `issue
+     8661`_).
 
 *    pip 20.3: pip defaults to the new resolver, but a user can opt-out
      and choose the old resolver behavior, using the flag
@@ -1616,6 +1622,7 @@ announcements on the `low-traffic packaging announcements list`_ and
 .. _resolver testing survey: https://tools.simplysecure.org/survey/index.php?r=survey/index&sid=989272&lang=en
 .. _issue 8661: https://github.com/pypa/pip/issues/8661
 .. _our announcement on the PSF blog: http://pyfound.blogspot.com/2020/03/new-pip-resolver-to-roll-out-this-year.html
+.. _two-minute video explanation: https://www.youtube.com/watch?v=B4GQCBBsuNU
 .. _tensorflow: https://pypi.org/project/tensorflow/
 .. _low-traffic packaging announcements list: https://mail.python.org/mailman3/lists/pypi-announce.python.org/
 .. _our survey on upgrades that create conflicts: https://docs.google.com/forms/d/e/1FAIpQLSeBkbhuIlSofXqCyhi3kGkLmtrpPOEBwr6iJA6SzHdxWKfqdA/viewform

news/9044.doc.rst

@@ -0,0 +1 @@
+Update user docs to reflect new resolver as default in 20.3.