Rely on SPDX identifier for license

[jaraco]

• Replace copy of license with an SPDX identifier. (Replace copy of license with an SPDX identifier. jaraco/skeleton#171)
• Add news fragment.
• Update tests in setuptools/dist not to rely on Setuptools having a license file.

Summary of changes

Closes #4956

Pull Request Checklist

• Changes have tests
• News fragment added in newsfragments/.
  (See documentation for details)

[abravalheri]

There is a (mildly heated) debate in https://discuss.python.org/t/split-from-pep-639-expressing-project-vs-distribution-licenses-post-pep-639-mod-titled/90314 regarding the vendoring use case and the license expressions.

According to that discussion the license field in the metadata is meant to also include the license of all vendors artifacts¹.

This means that for setuptools it would have to be something in the shape of MIT AND ... AND ... AND ... we're all the licenses are joined in a logic expression.

Because we don't have any tool in the ecosystem yet to calculate that automatically that we can add to the vendoring script, this would add maintenance burden, because we would need to manually craft or at least verify the license expression every time we update the vendored dependencies).

I mentioned in the thread two different approaches that we could follow (and no one said they would be "wrong" so far):

1. Simply don't include the license expressions. We already ship of the license files, so licensing information is available and we are respecting the licensing conditions. (This is the approach pip maintainers are tempted to follow).
2. We use a custom valid, but vague license expression like: MIT AND LicenseRef-Others or MIT AND LicenseRef-Vendored-Dependencies or LicenseRef-Core-MIT

This is currently a limitation of the standard (it covers simple projects but edge cases that involve vendoring are not easy to deal with).

@jaraco do you have thoughts on it?

Footnotes

1. Comments of the main PEP editors: https://discuss.python.org/t/split-from-pep-639-expressing-project-vs-distribution-licenses-post-pep-639-mod-titled/90314/93 , https://discuss.python.org/t/split-from-pep-639-expressing-project-vs-distribution-licenses-post-pep-639-mod-titled/90314/94 ↩

[jaraco]

My opinion - it's a lot of hand wringing over nothing. I don't have time to read, much less synthesize that long conversation. Paul makes a good point that the current system is unable to accurately and generally reflect the truth of the matter (what licenses are applicable for which lines of code). It's already enough overhead to deal with multiple declarations of potentially inconsistent license declarations in simple projects without vendored dependencies, I am strongly opposed to creating mashups of entangled licenses.

The industry has already accepted that licenses don't need to be included in every file and that copyrights don't need to be included in every file and every license. Let's just do our best to communicate the truth of the matter, that Setuptools is licensed as MIT, the copyrights are implied, and vendored packages are licensed per their project licenses. If someone needs clarity, we can do that out of band, but let's keep the SPDX identifier clean and straightforward and not confuse matters by blending them together. If they want to insist that vendoring a package requires somehow absorbing their license, that's going to be enough motivation to just remove the vendored packages (which incidentally, I'm getting close to having a technique that may unlock that without breaking bootstrapping).