PEP 376 compliance and the death of all dust puppies

[conrado]

I believe an emergent behavior from pip freeze is the root of all evil and it must be dealt with to promote agility in the python ecosystem.

This description defines what this issue should cover, describe why a possible refactor in the pip commands is necessary, and promote a discussion towards helping python package versions be semver compliant.

What is wrong with freeze?

While requirements.txt should be edited by hand and not rely on pip freeze, it is still often used to generate an initial list of packages and their versions to be used for requirements.txt, which is indeed often described in HOWTOs as the defacto way to freeze package dependency versions. My opinion is this has caused even several of the more core packages to be frozen down to patch.

eg. kmmbvnr/django-jenkins#250

Should packages be frozen down to patch?

Yes, but only in production.

Advertising the use of new flags to pip list (or a new command) such as --freeze, --dont-freeze and --freeze-minor would help promote more use of tests and continuous integration, optimism about the python ecosystem, facilitate contributions, help semver patch version mean what patch really means, and perhaps more. Other possible flags could come from PEP 376 compliance such as --graph and --swallow-subdeps (surely someone can think of a better name)

I would go as far as say all packages on PyPA should reference minor versions, though this is impossible and we will still be left with the case of curating dependency versions by hand, we will no longer entertain the notion that packages in development should be frozen down to patch.

Related Issues
#376 #377 #949 #1518 #1774 #2014 #2059 #2100 and many more... The idea is to get things failing, but failing fast.

References
http://legacy.python.org/dev/peps/pep-0376/

[kmmbvnr]

I didn't understand the reason why django-jenkins mentioned here. I used hand-written reqs.txt for reproducible test-only environment. I need exact version specification and I'm happy with current pip behavior.

[techtonik]

I am afraid there are no people who use pip freeze in automatic mode here. IMHO pip freeze should be replaced with more intuitive pip list -r <package> which parses package requirements and writes down only dependencies that this package uses in the format that this package provides.

[conrado]

@kmmbvnr imo code in revision control should be considered in development... so while commiting the CI will make builds against whatever is freshest. when you release a package you freeze versions, tag, and ship, then revert back to freezing only up to minor version number. I have mentioned django-jenkins as an example. of course i expect resistance, but it is futile.

[kmmbvnr]

@conrado it's not a resistance, you just didn't understand the important parts of development practice.

[pfmoore]

@conrado you do know that not all projects on PyPI use semver? And many don't even want to?

Other than that one (relatively minor, I suspect) point, I'm struggling to understand what you're proposing here. Perhaps you could get rid of some of the more emotive language and explain more specifically what you see as the problem, and what your proposed solution is?

[techtonik]

@pfmoore as far as I remember versioning rules enforced by newer PEPs are explicitly not compatible with semantic versioning. So I am not surprised if projects "choose" not to use semver.

[conrado]

@pfmoore with this issue i'm requesting PEP 376 compliance, the deprecation of the pip freeze command, and new flags to pip list, such as --graph, --freeze and --freeze-minor.

[pfmoore]

Um.

What this PEP proposes
To address those issues, this PEP proposes a few changes:

• A new .dist-info structure using a directory, inspired on one format of the EggFormats standard from setuptools.
• New APIs in pkgutil to be able to query the information of installed distributions.
• An uninstall function and an uninstall script in Distutils.

pip supports .dist-info, although there are some bits that are incomplete (e.g., INSTALLER). I'm not sure how that relates to the issue you describe, although I'm not against it.

The pkgutil and distutils changes are for core Python, not for pip.

I don't use pip freeze so I don't have an opinion on deprecation, although I suspect a number of people would.

As regards "new flags...such as", you'd need to be specific, ideally via a separate PR for each proposed new flag.

[dstufft]

I'm not sure what this issue is asking for, I think it may be looking for a solution like pip-compile from the pip-tools project. However it's had no activity for 3 years and with no real idea what it's asking for, I'm going to close it.