Invalid dev version identifiers in setup.py

[con-f-use]

<2.0dev is not a valid version matcher in the sense of PEP 440.
(Edit: it is after all valid, just hidden in the specification, and distlib doesn't parse it)

The canonical public version identifiers MUST comply with the following scheme:
[N!]N(.N)*[{a|b|rc}N][.postN][.devN]

It is also considered bad practice to have an upper bound on package versions and installers like pip do not typically consider development versions in any case (unless explicitly told to).

See: googleapis/google-api-python-client#2151

[dandhlee]

PEP 440 does allow the following:
https://peps.python.org/pep-0440/#development-release-separators
https://peps.python.org/pep-0440/#implicit-development-release-number

While this may not be considered best practice, <2.0dev is still a valid version matcher.

I believe upper bounds exist to avoid running into compatibility issues with breaking changes introduced by another package. If there's an upper bound, we'll have done it intentionally to ensure users aren't experiencing broken packages based on their environment.

Did something break and/or did you run into an unexpected case where the current system has caused an issue for you?

[con-f-use]

Yes, a packaging tool that used distlib failed to parse the requirements:

$ python -c 'from distlib.version import NormalizedMatcher; NormalizedMatcher("httplib2>=0.15.0,<1dev")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/jan/.local/share/virtualenvs/master_clean-V3vlFZeD/lib/python3.7/site-packages/distlib/version.py", line 125, in __init__
    vn, prefix = self.version_class(s), False
  File "/home/jan/.local/share/virtualenvs/master_clean-V3vlFZeD/lib/python3.7/site-packages/distlib/version.py", line 33, in __init__
    self._parts = parts = self.parse(s)
  File "/home/jan/.local/share/virtualenvs/master_clean-V3vlFZeD/lib/python3.7/site-packages/distlib/version.py", line 267, in parse
    result = _normalized_key(s)
  File "/home/jan/.local/share/virtualenvs/master_clean-V3vlFZeD/lib/python3.7/site-packages/distlib/version.py", line 188, in _pep_440_key
    raise UnsupportedVersionError('Not a valid version: %s' % s)
distlib.version.UnsupportedVersionError: Not a valid version: 1dev

As I understand it, distlib is supposed to be a common reference implementation for python packaging tools and a candidate for replacing packaging in the Python standard library. I kinda (worgnly) interpreted this part of PEP 400:

[N!]N(.N)*[{a|b|rc}N][.postN][.devN]

as mandating the dot and number in post and dev and apparently so did distlib. I'll raise an issue with them. Mandatory or not, the .dev0 in my PR might be more compatible, so merging it could be a good idea anyway. Or just saying something like <3.0.0 instead of 3.0.0dev everywhere.

[dandhlee]

Only specifying <3.0.0 still runs the risk of using 3.0.0.dev0 or any prerelease of 3.0.0 since 3.0.0 is higher version ordering in SemVer than 3.0.0-rc or 3.0.0.dev0, if it includes an incompatible breaking change.

Seems like you're running distlib directly - could you let me know of other situations where you've run into this issue? Normal imports through pip should be fine, just trying to understand your workflow and how you've encountered the issue other than directly using this util class from distlib.version.

[con-f-use]

Except that pip and most package tools do not consider pre-releases, unless the user explicitly specifies to use them. Which is the wanted behavior by the PyPA. But I agree, better safe than sorry in that regard and keep the dev. But having the upper bound on package versions at all, severely limits the ability for dependent projects to specify their dependencies, which is why most projects do not specify them and rather have a lock file or requirements.txt with recommended, pinned dependencies.

But back to the practical things: Is there a case you are worried about, where 1.dev0 would cause problems, that the present 1dev would not. Because as seen in the OP, 1dev does.

[dandhlee]

For the context of dev, the proposed change looks good, I just wanted to know if you run into this issue in a workflow that doesn't involve directly using this API, for example using an install and causing a failure issue. All good now, thanks for stopping by and making the changes 😄