Use setuptools.dynamic.optional-dependencies feature

[mwtoews]

This PR leverages built-in features from setuptools>=62.6 (pypa/setuptools@3b86141) and documented here which allows 'optional-dependencies' to be dynamic by reading files. This feature is marked as "BETA", probably because the current limitation is that it does not allow a mixture of dynamic and non-dynamic optional-dependencies to be specified.

This replaces the need for tools/generate_pyproject.toml.py and associated template file.

[rgommers]

This replaces the need for tools/generate_pyproject.toml.py and associated template file.

I haven't really followed closely recently, but that mechanism doesn't look ideal, and this PR will go further down such a path it looks like to me. Dependencies for a simple pure Python package like this one really should be declared statically. Static analysis tools (e.g. what powers the GitHub dependency analysis, https://libraries.io/ or recipe translators for other packaging ecosystems like grayskull) can understand pyproject.toml, but not anything custom like this. That's way more important than whatever the motivation for dynamic dependencies (like reducing a minor amount of duplication in listed dependencies) is unless those dependencies are actually dynamic.

So my preference would be to not do any of this and simply use a regular pyproject.toml.

[mwtoews]

My preference would be to do-away with most requirements*txt files, and just use pyproject.toml which is backed by PEP 621. The requirements files exist by convention with pip install -r, not by any PEP (that I'm aware of). Some static analysis tools like GitHub's Dependency graph look at requirements/test.txt and requirements/doc.txt, but also pyproject.toml too. (I'm unsure if that tool would only use pyproject.toml in absence of the requirements files, however.)

Are there still needs to have separate requirements files? If no, then it would be much simpler to maintain the optional dependencies in pyproject.toml.

Xref #474 which introduced the tools/.

[stefanv]

At this point, there's still no equivalent to pip install -r requirements/x.txt. Until there is, I'd prefer not to put all dependencies in pyproject.toml only.

[rgommers]

My preference would be to do-away with most requirements*txt files, and just use pyproject.toml which is backed by PEP 621.

I agree with the PEP 621 part in particular, yes. I don't feel strongly about requirements files either way - if others have a need for them, it's very little effort to keep them around for the time being.

At this point, there's still no equivalent to pip install -r requirements/x.txt. Until there is, I'd prefer not to put all dependencies in pyproject.toml only.

True. That is fine with me. Then let's just keep duplicate dependencies in pyproject.toml and requirements/*.txt. A simple comment saying "if you update dependencies in this file, update them in <other file> as well" will do the job. And dependencies here should anyway change very rarely.

[larsoner]

+1 for living with some duplication for now

[mwtoews]

Thanks for the feedback! Here is an alternative proposal that I think will suite everyone is:

• Make pyproject.toml a primary file without any template
• Keep static optional-dependencies in pyproject.toml (i.e. not dynamic tool-dependant)
• Replace the pre-commit script to instead generate requirements/*.txt files

This proposal will keep the requirements files in-sync with a primary pyproject.toml file without the need of any template.

[stefanv]

That'd work; it's probably a better direction to sync in than we do currently. @jarrodmillman if that works, we can do it for skimage as well.

[mwtoews]

Superseded by #483