update the minimum version policy (#4907)

* update the minimum version policy

* adapt the minimum versions check script

* improve the error message

* update the length of the support windows for python and numpy

* implement the new policy

* Refine wording

* add a entry to whats-new.rst

* properly format the minimum versions table [skip-ci]

* rewrite the error message for too new packages

* reformat the policy [skip-ci]

* remove the policy override for dask and distributed

Co-authored-by: Maximilian Roos <[email protected]>

Author: keewis

.github/workflows/ci-additional.yaml

@@ -184,6 +184,6 @@ jobs:
 
       - name: minimum versions policy
         run: |
-          mamba install -y pyyaml conda
+          mamba install -y pyyaml conda python-dateutil
           python ci/min_deps_check.py ci/requirements/py37-bare-minimum.yml
           python ci/min_deps_check.py ci/requirements/py37-min-all-deps.yml

ci/min_deps_check.py

@@ -4,11 +4,12 @@
 """
 import itertools
 import sys
-from datetime import datetime, timedelta
+from datetime import datetime
 from typing import Dict, Iterator, Optional, Tuple
 
 import conda.api
 import yaml
+from dateutil.relativedelta import relativedelta
 
 CHANNELS = ["conda-forge", "defaults"]
 IGNORE_DEPS = {
@@ -25,14 +26,9 @@
     "pytest-xdist",
 }
 
-POLICY_MONTHS = {"python": 42, "numpy": 24, "setuptools": 42}
+POLICY_MONTHS = {"python": 24, "numpy": 18, "setuptools": 42}
 POLICY_MONTHS_DEFAULT = 12
 POLICY_OVERRIDE = {
-    # dask < 2.9 has trouble with nan-reductions
-    # TODO remove this special case and the matching note in installing.rst
-    #      after January 2021.
-    "dask": (2, 9),
-    "distributed": (2, 9),
     # setuptools-scm doesn't work with setuptools < 36.7 (Nov 2017).
     # The conda metadata is malformed for setuptools < 38.4 (Jan 2018)
     # (it's missing a timestamp which prevents this tool from working).
@@ -148,28 +144,32 @@ def process_pkg(
         return pkg, fmt_version(req_major, req_minor, req_patch), "-", "-", "-", "(!)"
 
     policy_months = POLICY_MONTHS.get(pkg, POLICY_MONTHS_DEFAULT)
-    policy_published = datetime.now() - timedelta(days=policy_months * 30)
-
-    policy_major = req_major
-    policy_minor = req_minor
-    policy_published_actual = req_published
-    for (major, minor), published in reversed(sorted(versions.items())):
-        if published < policy_published:
-            break
-        policy_major = major
-        policy_minor = minor
-        policy_published_actual = published
+    policy_published = datetime.now() - relativedelta(months=policy_months)
+
+    filtered_versions = [
+        version
+        for version, published in versions.items()
+        if published < policy_published
+    ]
+    policy_major, policy_minor = max(filtered_versions, default=(req_major, req_minor))
 
     try:
         policy_major, policy_minor = POLICY_OVERRIDE[pkg]
     except KeyError:
         pass
+    policy_published_actual = versions[policy_major, policy_minor]
 
     if (req_major, req_minor) < (policy_major, policy_minor):
         status = "<"
     elif (req_major, req_minor) > (policy_major, policy_minor):
         status = "> (!)"
-        error("Package is too new: " + pkg)
+        delta = relativedelta(datetime.now(), policy_published_actual).normalized()
+        n_months = delta.years * 12 + delta.months
+        error(
+            f"Package is too new: {pkg}={req_major}.{req_minor} was "
+            f"published on {versions[req_major, req_minor]:%Y-%m-%d} "
+            f"which was {n_months} months ago (policy is {policy_months} months)"
+        )
     else:
         status = "="
 

doc/installing.rst

@@ -98,10 +98,10 @@ Minimum dependency versions
 xarray adopts a rolling policy regarding the minimum supported version of its
 dependencies:
 
-- **Python:** 42 months
+- **Python:** 24 months
   (`NEP-29 <https://numpy.org/neps/nep-0029-deprecation_policy.html>`_)
 - **setuptools:** 42 months (but no older than 40.4)
-- **numpy:** 24 months
+- **numpy:** 18 months
   (`NEP-29 <https://numpy.org/neps/nep-0029-deprecation_policy.html>`_)
 - **dask and dask.distributed:** 12 months (but no older than 2.9)
 - **sparse, pint** and other libraries that rely on
@@ -111,9 +111,9 @@ dependencies:
   numpy >=1.17.
 - **all other libraries:** 12 months
 
-The above should be interpreted as *the minor version (X.Y) initially published no more
-than N months ago*. Patch versions (x.y.Z) are not pinned, and only the latest available
-at the moment of publishing the xarray release is guaranteed to work.
+This means the latest minor (X.Y) version from N months prior. Patch versions (x.y.Z)
+are not pinned, and only the latest available at the moment of publishing the xarray
+release is guaranteed to work.
 
 You can see the actual minimum tested versions:
 

doc/whats-new.rst

@@ -24,15 +24,19 @@ Breaking changes
 ~~~~~~~~~~~~~~~~
 - xarray no longer supports python 3.6
 
+  The minimum version policy was changed to also apply to projects with irregular
+  releases.
+
   The minimum versions of some other dependencies were changed:
+
   ============ ====== ====
   Package      Old    New
   ============ ====== ====
   Python       3.6    3.7
   setuptools   38.4   40.4
   ============ ====== ====
 
-  (:issue:`4688`, :pull:`4720`)
+  (:issue:`4688`, :pull:`4720`, :pull:`4907`)
   By `Justus Magin <https://github.com/keewis>`_.
 - use ``pyproject.toml`` instead of the ``setup_requires`` option for
   ``setuptools`` (:pull:`4897`).