From 653cbc62db3319952902df4df8aad24174665c41 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Tue, 5 Dec 2023 12:41:46 +1000 Subject: [PATCH 01/31] WIP: simple repository API spec --- .../simple-repository-api.openapi.yml | 202 ++++++++++++++++++ .../specifications/simple-repository-api.rst | 27 +++ 2 files changed, 229 insertions(+) create mode 100644 source/specifications/simple-repository-api.openapi.yml diff --git a/source/specifications/simple-repository-api.openapi.yml b/source/specifications/simple-repository-api.openapi.yml new file mode 100644 index 000000000..81a919c3c --- /dev/null +++ b/source/specifications/simple-repository-api.openapi.yml @@ -0,0 +1,202 @@ +openapi: '3.1.0' + +info: + title: Simple Repository API + version: '1.1' + summary: A protocol for tools to list and and download Python packages + license: + name: Creative Commons Attribution-ShareAlike + url: https://creativecommons.org/licenses/by-sa/3.0 + +paths: + /: + get: + summary: Projects list + description: List all projects provided by this package index + responses: + 200: + description: Success + content: + text/html: + schema: + title: HTML response + type: string + application/vnd.pypi.simple.v1+html: + schema: + title: HTML response + type: string + application/vnd.pypi.simple.v1+json: + schema: + title: JSON response + type: object + required: + - meta + - projects + properties: + meta: + $ref: '#/components/schemas/meta' + projects: + title: Projects list + type: array + items: + title: Project reference + type: object + required: + - name + properties: + name: + title: Project name + type: string + example: + meta: + api-version: '1.0' + projects: + - name: Frob + - name: spamspamspam + + /{project}/: + parameters: + - name: project + in: path + description: Project normalised name + required: true + example: numpy + schema: + type: string + pattern: ^[a-z0-9](-?[a-z0-9])*[a-z0-9]?$ + + get: + summary: Project details + description: Some project details and all package files in the index for + the project + responses: + 200: + description: Success + content: + text/html: + schema: + title: HTML response + type: string + application/vnd.pypi.simple.v1+html: + schema: + title: HTML response + type: string + application/vnd.pypi.simple.v1+json: + schema: + title: JSON response + type: object + required: + - meta + - name + - files + properties: + meta: + $ref: '#/components/schemas/meta' + name: + title: Project name (normalised) + type: string + pattern: ^[a-z0-9](-?[a-z0-9])*[a-z0-9]?$ + files: + title: Project package files + type: array + items: + title: Package file details + type: object + required: + - filename + - url + - hashes + properties: + filename: + title: Package filename + type: string + url: + title: File download URL + type: string + format: uri-reference + hashes: + $ref: '#/components/schemas/hashes' + requires-python: + title: Python requirement + description: Value of distribution's + `Requires-Python` metadata field + type: string + dist-info-metadata: + title: Distribution metadata file details + description: Indicates whether the distribution + metadata file exists. This file is located at the + URL equal to the package file's URL appended with + `.metadata` + oneOf: + - title: Metadata file existance + type: boolean + - $ref: '#/components/schemas/hashes' + gpg-sig: + title: File GPG signature file existance + description: Indicates whether the GPG signature + file exists. This file is located at the URL equal + to the package file's URL appended with `.asc` + type: boolean + yanked: + title: Distribution yanked details + description: Indicates whether the distribution has + been yanked + oneOf: + - title: Yanked status + type: boolean + - title: Yanked reason + type: string + example: + meta: + api-version: '1.0' + name: holygrail + files: + - filename: holygrail-1.0.tar.gz + url: https://example.com/files/holygrail-1.0.tar.gz + hashes: + sha256: ... + blake2b: ... + requires-python: '>=3.7' + yanked: Had a vulnerability + - filename: holygrail-1.0-py3-none-any.whl + url: https://example.com/files/holygrail-1.0-py3-none-any.whl + hashes: + sha256: ... + blake2b: ... + requires-python: '>=3.7' + dist-info-metadata: true + + 404: + description: Project not known by the index + +components: + schemas: + meta: + title: Response meta-details + type: object + properties: + api-version: + type: string + enum: ['1.0', '1.1', '1.2'] + + hashes: + title: File hashes + type: object + additionalProperties: + title: Hash value + description: Hexadecimal encoding of hash value + type: string + pattern: ^[0-9a-f]+$ + propertyNames: + title: Hash name + description: Name of hash algorithm in `hashlib`, lower-case, + preferrably `sha256` + pattern: ^[a-z0-9_]+$ + examples: + - sha256: 4e388ab32b10dc8dbc7e28144f552830adc74787c1e2c0824032078a79f227fb + +servers: + - url: https://pypi.org/simple/ + description: PyPI, the default package index used by most Python packaging + tools. It runs [Warehouse](https://warehouse.pypa.io/) as the index + server. diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 70c0040d2..bdef2121a 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -5,6 +5,33 @@ Simple repository API ===================== +The simple repository API is an HTTP-based protocol for tools to list and +download Python :term:`packages `. It is the API which +:term:`package indexes ` implement to provide package managers +(eg :ref:`pip`) enough information to determine what to install for a given set +of :term:`requirements `, then go on to install those packages. + +There is one version series for the API: version 1. Minor versions add optional +features, and are described below. + +The API consists of two endpoints: listing :term:`projects ` available +in the index, and listing package files for (and some other details of) a given +project. These endpoints are provided as responses from an HTTP server. + +There are two representations of responses from the API: HTML and JSON. Apart +from optional features, these representations provide the same information. + +Specification +============= + +.. collapse:: OpenAPI + + An `OpenAPI `__ document which specifies + the full API for both representations. + + .. literalinclude:: simple-repository-api.openapi.yml + :language: yaml + The interface for querying available package versions and retrieving packages from an index server comes in two forms: HTML and JSON. From d99bb408a083d414095b7997fdc38f8845c78cd1 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 14:26:55 +1000 Subject: [PATCH 02/31] Convert history to list --- .../specifications/simple-repository-api.rst | 28 ++++++++++--------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index bdef2121a..6818f5743 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -32,16 +32,18 @@ Specification .. literalinclude:: simple-repository-api.openapi.yml :language: yaml -The interface for querying available package versions and -retrieving packages from an index server comes in two forms: -HTML and JSON. - -The HTML format is defined in :pep:`503`, with the addition of "yank" -support (allowing a kind of file deletion) in :pep:`592`, specifying -the interface version provided by an index server in :pep:`629`, and -providing package metadata independently from a package in -:pep:`658` and revised in :pep:`714`. - -The JSON format is defined in :pep:`691`, with additional fields -added in :pep:`700`, and revisions around providing package metadata -independently from a package in :pep:`714`. +History +======= + +* September 2015: initial form of the HTML format, in :pep:`503` +* May 2019: "yank" support, in :pep:`592` +* July 2020: API versioning convention and metadata, and declaring the HTML + format as API v1, in :pep:`629` +* May 2021: providing package metadata independently from a package, in + :pep:`658` +* May 2022: initial form of the JSON format, with a mechanism for clients to + choose between them, and declaring both formats as API v1, in :pep:`691` +* October 2022: project versions and file size and upload-time in the JSON + format, in :pep:`700` +* June 2023: renaming the field which provides package metadata independently + from a package, in :pep:`714` From 62a3514d7318b56f6bf55ecd3deb6552f7fe27dd Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 17:53:06 +1000 Subject: [PATCH 03/31] Add PEP 503 --- .../specifications/simple-repository-api.rst | 130 ++++++++++++++++++ 1 file changed, 130 insertions(+) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 6818f5743..ed1ddde7b 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -32,6 +32,132 @@ Specification .. literalinclude:: simple-repository-api.openapi.yml :language: yaml +Representations +############### + +The index server can respond with one of two different representation formats +for each endpoint: HTML and JSON. The format can be selected by the client +through the use of `content negotiation +`_. + +Endpoints +######### + +The API consists of two metadata endpoints: + +* :ref:`repo_api_projects_list` +* :ref:`repo_api_project_details` + +The root URL ``/`` represents the base URL, where it would be prefixed with +the index's URL to construct the full URL which tools make the request for. + +If a client makes a request to a URL without a trailing forward-slash ``/``, +then the index server should redirect the client to the same URL with the ``/`` +appended. + +.. _repo_api_projects_list: + +Projects list +------------- + +URL: ``/``, the root URL + +This endpoint returns a list of all of the projects provided by the index, +with each list item containing the project's name. + +HTML representation +^^^^^^^^^^^^^^^^^^^ + +The response from the index is a valid +`HTML5 `_ page. + +Each project provided by the index has a corresponding `anchor element`_ +````: + +* Its body text must exist and is the name of the project (not necessarily + :ref:`normalized `). + +* Its ``href`` attribute must exist and is a URL to the :ref:`project details + ` page for the project. This URL must end with a + forward-slash ``/``, but may be absolute or relative. + +An example response page: + +.. code-block:: html + + + + Projects + + frob + spamspamspam + + + +.. _repo_api_project_details: + +Project details +--------------- + +URL: ``//``, where ```` is replaced with the :ref:`normalized +name ` of the project. + +This endpoint returns some metadata of the project, along with a list of all +distribution package files provided by the index for the project. + +If a client uses an unnormalized name for ````, the index server may +redirect to the URL with the normalized name. Conformant client must always +make requests with normalized names. + +API file-related features: + +* The file can be hosted anywhere, not necessarily by the index server. + +* The file's URL in the list-item is a URL to fetch the file. It may be + absolute or relative. It's last path segment must be the file's filename. + +* Hashes of the file's contents are optional but recommended. The hash name is + the name of the hash algorithm's function, and the value is the hex-encoded + digest hash. The function should be one in the standard-library ``hashlib`` + module, and ``sha256`` is preferred. + +* A `GPG signature `_ + for the file can be accessed at the same URL as the file but with ``.asc`` + appended, if it is provided. For example, the file at + ``/packages/HolyGrail-1.0.tar.gz`` may have a signature at + ``/packages/HolyGrail-1.0.tar.gz.asc``. + +* The file's release's :ref:`core-metadata-requires-python` metadata field may + be provided. Clients should ignore the file when installing to an environment + for a version of Python which doesn't satisfy the requirement. + +HTML representation +^^^^^^^^^^^^^^^^^^^ + +The response from the index is a valid +`HTML5 `_ page. + +Each distribution package file provided by the index for the project has a +corresponding `anchor element`_ ````: + +* Its body text must exist and is the file's filename. + +* Its ``href`` attribute must exist and is the file's URL. + + * This URL should also include a URL fragment of the form + ``#=``, where ```` is the hash name and ```` is + hash value. + +* A ``data-gpg-sig`` `data attribute`_ may exist, and have value ``true`` to + indicate a file has a GPG signature (at the location described above), or + ``false`` to indicate no signature. Indexes should do this for none or all + files (not some). + +* A ``data-requires-python`` `data attribute`_ may exist, and have value equal + to the :ref:`core-metadata-requires-python` metadata field for the file's + release, with HTML-encoding (less-than ``<`` becomes the string ``<``, and + greater-than ``>`` becomes the string ``>``). + History ======= @@ -47,3 +173,7 @@ History format, in :pep:`700` * June 2023: renaming the field which provides package metadata independently from a package, in :pep:`714` + +.. _anchor element: https://html.spec.whatwg.org/#the-a-element + +.. _data attribute: https://html.spec.whatwg.org/#attr-data-* From 3d895a8d150da3710f2c5abd0349f4addacfd2e4 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 17:54:58 +1000 Subject: [PATCH 04/31] Link to spec for OpenAPI --- source/specifications/simple-repository-api.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index ed1ddde7b..59dd78350 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -26,8 +26,9 @@ Specification .. collapse:: OpenAPI - An `OpenAPI `__ document which specifies - the full API for both representations. + An `OpenAPI `__ document which + specifies the full API. It declares both representations, but only details + the JSON format's schema. .. literalinclude:: simple-repository-api.openapi.yml :language: yaml From 6d057b38a001e7ab3a28201a537385a14ffc76e7 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 17:55:14 +1000 Subject: [PATCH 05/31] Add history for requires-python --- source/specifications/simple-repository-api.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 59dd78350..99920bdf2 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -163,6 +163,7 @@ History ======= * September 2015: initial form of the HTML format, in :pep:`503` +* July 2016: Requires-Python metadata, in an update to :pep:`503` * May 2019: "yank" support, in :pep:`592` * July 2020: API versioning convention and metadata, and declaring the HTML format as API v1, in :pep:`629` From f8d34f77fc83f04501e1a15f2b490a3530089ec1 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 18:32:13 +1000 Subject: [PATCH 06/31] Add PEP 592 --- .../specifications/simple-repository-api.rst | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 99920bdf2..7f26af517 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -132,6 +132,8 @@ API file-related features: be provided. Clients should ignore the file when installing to an environment for a version of Python which doesn't satisfy the requirement. +* Files may be marked as `yanked `_. + HTML representation ^^^^^^^^^^^^^^^^^^^ @@ -159,6 +161,59 @@ corresponding `anchor element`_ ````: release, with HTML-encoding (less-than ``<`` becomes the string ``<``, and greater-than ``>`` becomes the string ``>``). +* A ``data-yanked`` `data attribute`_ may exist to indicate the file was + `yanked `_. The attribute may have a value which + specifies the reason the file is yanked. + +.. _simple_repo_api_yanked: + +Yanked files +############ + +A yanked :term:`package file ` is one intended to be +now-unavailable for installation from the index. The file's yank-status can be +changed at anypoint (to be unyanked, or even yanked again). + +Indexes may provide a textual reason for why the file has been yanked, and +clients may display that reason to end-users. + +From :pep:`592`, the intention for the behaviour with yanked files is: + + The desirable experience for users is that once a file is yanked, when a + human being is currently trying to directly install a yanked file, that it + fails as if that file had been deleted. However, when a human did that + awhile ago, and now a computer is just continuing to mechanically follow the + original order to install the now yanked file, then it acts as if it had not + been yanked. + +Installers must ignore yanked :term:`releases ` if a non-yanked +release satisfies the :term:`requirement `. Installers may refuse +to install a yanked release and not install anything. Installers should follow +the spirit of the intention quoted above and prevent new dependencies on yanked +releases and files. + +Installers should emit a warning when it does decide to install a yanked file. +That warning may utilize the reason for the yanking. + +What this means is left up to the specific installer, to decide how to best fit +into the overall usage of their installer. However, there are two suggested +approaches to take: + +* Yanked files are always ignored, unless they are the only file that matches a + version specifier that “pins” to an exact version using either ``==`` + (without any modifiers that make it a range, such as ``.*``) or ``===``. + Matching this version specifier should otherwise be done as per :pep:`440` + for things like local versions, zero padding, etc. + +* Yanked files are always ignored, unless they are the only file that matches + what a lock file (such as Pipfile.lock or poetry.lock) specifies to be + installed. In this case, a yanked file SHOULD not be used when creating or + updating a lock file from some input file or command. + +Mirror indexes may omit list-items for yanked files in their responses to +clients, or may include list-items for yanked files along with their +yank-status (this status must be present for yanked files). + History ======= From 8f22157033dcb1af9fffe093c4f534a18ce3b15a Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 18:32:57 +1000 Subject: [PATCH 07/31] More glossary linking --- source/specifications/simple-repository-api.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 7f26af517..8a8c9f06e 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -63,8 +63,8 @@ Projects list URL: ``/``, the root URL -This endpoint returns a list of all of the projects provided by the index, -with each list item containing the project's name. +This endpoint returns a list of all of the :term:`projects ` provided +by the index, with each list item containing the project's name. HTML representation ^^^^^^^^^^^^^^^^^^^ @@ -103,8 +103,9 @@ Project details URL: ``//``, where ```` is replaced with the :ref:`normalized name ` of the project. -This endpoint returns some metadata of the project, along with a list of all -distribution package files provided by the index for the project. +This endpoint returns some metadata of the :term:`project `, along +with a list of all :term:`package files ` provided by the +index for the project. If a client uses an unnormalized name for ````, the index server may redirect to the URL with the normalized name. Conformant client must always From 944c4f92bf386ce99698e74a35087d2916e53820 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 18:51:39 +1000 Subject: [PATCH 08/31] Add API version scheme and metadata element --- .../specifications/simple-repository-api.rst | 37 ++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 8a8c9f06e..cfc69f031 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -33,6 +33,26 @@ Specification .. literalinclude:: simple-repository-api.openapi.yml :language: yaml +API version scheme +################## + +The version of the API is a version number which follows the :ref:`version +specifier specification ` as only ``Major.Minor``. + +Incrementing the major version is used to signal a backwards incompatible +change such that existing clients would no longer be expected to be able to +meaningfully use the API. + +Incrementing the minor version is used to signal a backwards compatible change +such that existing clients would still be expected to be able to meaningfully +use the API. + +Clients should introspect each response for the repository version, and must +assume version 1.0 if missing. If the major version is greater than expected, +clients must fail with an appropriate error message for the user. If the minor +version is greater than expected, clients should warn the user with an +appropriate message. Clients may continue to use feature detection. + Representations ############### @@ -72,6 +92,11 @@ HTML representation The response from the index is a valid `HTML5 `_ page. +A `metadata element`_ ```` may exist anywhere in the HTML document, with +``name`` attribute value equal to the string ``pypi:repository-version``, and +``content`` attribute value equal the API version which the response +implements. + Each project provided by the index has a corresponding `anchor element`_ ````: @@ -88,7 +113,10 @@ An example response page: - Projects + + + Projects + frob spamspamspam @@ -141,6 +169,11 @@ HTML representation The response from the index is a valid `HTML5 `_ page. +A `metadata element`_ ```` may exist anywhere in the HTML document, with +``name`` attribute value equal to the string ``pypi:repository-version``, and +``content`` attribute value equal the API version which the response +implements. + Each distribution package file provided by the index for the project has a corresponding `anchor element`_ ````: @@ -235,3 +268,5 @@ History .. _anchor element: https://html.spec.whatwg.org/#the-a-element .. _data attribute: https://html.spec.whatwg.org/#attr-data-* + +.. _metadata element: https://html.spec.whatwg.org/#the-meta-element From cbf3770e292b88f4a83cadf4c2dcaeeaf4d51754 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 18:55:05 +1000 Subject: [PATCH 09/31] Add project-details HTML example --- .../specifications/simple-repository-api.rst | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index cfc69f031..a2312dd9b 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -199,6 +199,27 @@ corresponding `anchor element`_ ````: `yanked `_. The attribute may have a value which specifies the reason the file is yanked. +An example response page: + +.. code-block:: html + + + + + + Foo + + + foo-1.0.0.tar.gz + foo-1.0.1.tar.gz + + + .. _simple_repo_api_yanked: Yanked files From 39c56f2782951e9819453d24a1a91272ef3729ed Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 19:00:16 +1000 Subject: [PATCH 10/31] Fix spelling --- source/specifications/simple-repository-api.openapi.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/specifications/simple-repository-api.openapi.yml b/source/specifications/simple-repository-api.openapi.yml index 81a919c3c..118d2f468 100644 --- a/source/specifications/simple-repository-api.openapi.yml +++ b/source/specifications/simple-repository-api.openapi.yml @@ -128,11 +128,11 @@ paths: URL equal to the package file's URL appended with `.metadata` oneOf: - - title: Metadata file existance + - title: Metadata file existence type: boolean - $ref: '#/components/schemas/hashes' gpg-sig: - title: File GPG signature file existance + title: File GPG signature file existence description: Indicates whether the GPG signature file exists. This file is located at the URL equal to the package file's URL appended with `.asc` @@ -190,7 +190,7 @@ components: propertyNames: title: Hash name description: Name of hash algorithm in `hashlib`, lower-case, - preferrably `sha256` + preferably `sha256` pattern: ^[a-z0-9_]+$ examples: - sha256: 4e388ab32b10dc8dbc7e28144f552830adc74787c1e2c0824032078a79f227fb From c5d962e355744aa19db476067193cf71c30a1911 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 19:00:28 +1000 Subject: [PATCH 11/31] Fix references --- source/specifications/simple-repository-api.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index a2312dd9b..7aa166335 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -161,7 +161,7 @@ API file-related features: be provided. Clients should ignore the file when installing to an environment for a version of Python which doesn't satisfy the requirement. -* Files may be marked as `yanked `_. +* Files may be marked as :ref:`yanked `. HTML representation ^^^^^^^^^^^^^^^^^^^ @@ -196,7 +196,7 @@ corresponding `anchor element`_ ````: greater-than ``>`` becomes the string ``>``). * A ``data-yanked`` `data attribute`_ may exist to indicate the file was - `yanked `_. The attribute may have a value which + :ref:`yanked `. The attribute may have a value which specifies the reason the file is yanked. An example response page: From fa0788c29c5fbd2cc66a086231dc340f632a3f19 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 19:25:26 +1000 Subject: [PATCH 12/31] Add PEP 658 --- .../specifications/simple-repository-api.rst | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 7aa166335..ca01e7024 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -163,6 +163,18 @@ API file-related features: * Files may be marked as :ref:`yanked `. +* The file's :ref:`Core Metadata ` must be provided if its + existence is indicated. In addition, the file must contain this metadata + which will not be modified when the distribution is processed and/or + installed. + + The metadata must be accessed at the same URL as the file but with + ``.metadata`` appended. For example, the file at + ``/files/distribution-1.0-py3.none.any.whl`` may have its metadata at + ``/files/distribution-1.0-py3.none.any.whl.metadata``. + + The index should also provide a hash of the metadata. + HTML representation ^^^^^^^^^^^^^^^^^^^ @@ -199,6 +211,15 @@ corresponding `anchor element`_ ````: :ref:`yanked `. The attribute may have a value which specifies the reason the file is yanked. +* A ``data-core-metadata`` `data attribute`_ may exist to indicate the index + provides the file's core-metadata. The attribute's value should be of the + form ``=``, where ```` is the hash name and ```` is + hash value; otherwise, the value may the string ``true``, or not provided, if + the metadata's hash is not available. + + This attribute may be duplicated as the `data attribute`_ + ``data-dist-info-metadata``. + An example response page: .. code-block:: html From 6a82b48dd5f518d718bd94993fbe0988cf4bc867 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 19:25:43 +1000 Subject: [PATCH 13/31] Remove extra space --- source/specifications/simple-repository-api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index ca01e7024..b3e10de08 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -194,7 +194,7 @@ corresponding `anchor element`_ ````: * Its ``href`` attribute must exist and is the file's URL. * This URL should also include a URL fragment of the form - ``#=``, where ```` is the hash name and ```` is + ``#=``, where ```` is the hash name and ```` is hash value. * A ``data-gpg-sig`` `data attribute`_ may exist, and have value ``true`` to From 381203df19ba502f6e08f7d4967ea97ff77edad5 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 19:42:10 +1000 Subject: [PATCH 14/31] Common HTML5 spec link --- source/specifications/simple-repository-api.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index b3e10de08..9695cf3ba 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -57,8 +57,8 @@ Representations ############### The index server can respond with one of two different representation formats -for each endpoint: HTML and JSON. The format can be selected by the client -through the use of `content negotiation +for each endpoint: `HTML5`_ and JSON. The format can be selected by the +client through the use of `HTTP content negotiation `_. Endpoints @@ -89,8 +89,7 @@ by the index, with each list item containing the project's name. HTML representation ^^^^^^^^^^^^^^^^^^^ -The response from the index is a valid -`HTML5 `_ page. +The response from the index is a valid `HTML5`_ page. A `metadata element`_ ```` may exist anywhere in the HTML document, with ``name`` attribute value equal to the string ``pypi:repository-version``, and @@ -178,8 +177,7 @@ API file-related features: HTML representation ^^^^^^^^^^^^^^^^^^^ -The response from the index is a valid -`HTML5 `_ page. +The response from the index is a valid `HTML5`_ page. A `metadata element`_ ```` may exist anywhere in the HTML document, with ``name`` attribute value equal to the string ``pypi:repository-version``, and @@ -307,6 +305,8 @@ History * June 2023: renaming the field which provides package metadata independently from a package, in :pep:`714` +.. _HTML5: https://html.spec.whatwg.org/ + .. _anchor element: https://html.spec.whatwg.org/#the-a-element .. _data attribute: https://html.spec.whatwg.org/#attr-data-* From c5e579a5f37436b798dfc571efd3bd99359e63d1 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 19:56:45 +1000 Subject: [PATCH 15/31] Add core-metadata to HTML example --- source/specifications/simple-repository-api.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 9695cf3ba..024489221 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -235,6 +235,7 @@ An example response page: data-gpg-sig="true" data-requires-python=">=3.12" data-yanked="Too much bar" + data-core-metadata="sha256=abcd1234" >foo-1.0.1.tar.gz From 5adcc7ce964e83849ad5b4949ea96776e3017912 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 21:22:42 +1000 Subject: [PATCH 16/31] Add PEP 691 --- .../specifications/simple-repository-api.rst | 181 +++++++++++++++++- 1 file changed, 177 insertions(+), 4 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 024489221..b1e7e4d1b 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -57,9 +57,70 @@ Representations ############### The index server can respond with one of two different representation formats -for each endpoint: `HTML5`_ and JSON. The format can be selected by the -client through the use of `HTTP content negotiation -`_. +for each endpoint: + +* `HTML5`_: content-type ``application/vnd.pypi.simple.v1+html`` + + * Clients can also request ``application/vnd.pypi.simple.latest+json`` + +* `JSON`_: content-type ``application/vnd.pypi.simple.v1+html`` + + * Clients can also request ``application/vnd.pypi.simple.latest+html`` + + * ``text/html`` is an alias for this content-type + +See :pep:`691` for the details of how these values should be updated for new +versions of the API. + +Content negotiation +------------------- + +The representation can be selected by the client through the use of `HTTP +content negotiation +`_. + +To request one of these representations, the `Accept +`_ header should be +set in requests to one of the above content-type values. If this header is +missing, the index server assumes it to be ``*/*``. + +Responses from the index server should have the corresponding `Content-Type +`_ header for the +representation provided. If ``latest`` was requested by the client, then ``v1`` +must be returned by the index server. + +If a representation can't be selected from the client's request, then the index +server can do one of: + +* Select a default content type other than what the client has requested and + return a response with that. + +* Return a HTTP 406 Not Acceptable response to indicate that none of the + requested content types were available, and the server was unable or + unwilling to select a default content type to respond with. + +* Return a HTTP 300 Multiple Choices response that contains a list of all of + the possible responses that could have been chosen. This option is not + encouraged as there is no standard format for this list. + +Clients should be prepared to handle all possible responses. + +URL parameter +------------- + +This method takes precedence over content negotiation. Index servers may +optionally support this method, or respond with an error if present, and +clients should not rely on it. + +A ``format`` URL parameter can be specified, with value equal to one of the +above content-type values. If the index server does not support the value, it +may fall back to content negotiation. + +Endpoint configuration +---------------------- + +This method is simply a suggestion, and is not standardised. Servers could +configure different base URLs to serve the different representations. Endpoints ######### @@ -84,7 +145,8 @@ Projects list URL: ``/``, the root URL This endpoint returns a list of all of the :term:`projects ` provided -by the index, with each list item containing the project's name. +by the index, with each list item containing the project's name. This list is +not necessarily ordered. HTML representation ^^^^^^^^^^^^^^^^^^^ @@ -122,6 +184,40 @@ An example response page: +JSON representation +^^^^^^^^^^^^^^^^^^^ + +The response from the index is a valid `JSON`_ document. This document +represents an object with properties: + +* ``meta`` (object, required) - response metadata; has properties: + + * ``api-version`` (string, required) - the API version the response + implements. + +* ``projects`` (array of objects, required) - projects list. Each project + provided by the index corresponds to an element in this array, and vice + versa. Objects have properties: + + * ``name`` (required) - the project's name (not necessarily :ref:`normalized + `), as a string. + +Unknown JSON object keys must be ignored. + +An example response document: + +.. code-block:: json + + { + "meta": { + "api-version": "1.0" + }, + "projects": [ + {"name": "Frob"}, + {"name": "spamspamspam"} + ] + } + .. _repo_api_project_details: Project details @@ -240,6 +336,81 @@ An example response page: +JSON representation +^^^^^^^^^^^^^^^^^^^ + +The response from the index is a valid `JSON`_ document. This document +represents an object with properties: + +* ``meta`` (object, required) - response metadata; has properties: + + * ``api-version`` (string, required) - the API version the response + implements. + +* ``name`` (string, required) - the :ref:`normalized ` name + of the project. + +* ``files`` (array of objects, required) - files list. Each file provided by + the index for the project corresponds to an element in this array, and vice + versa. Objects have properties: + + * ``filename`` (string, required) - the file's filename + + * ``url`` (string, required) - the file's URL + + * ``hashes`` (object, required) - the file's hashes. Its keys are the hash + names, and the values are the corresponding hash values. Should contain at + least one hash. + + * ``gpg-sig`` (boolean, optional) - indicates whether the index provides the + file's GPG signature. + + If this key is missing, the signature may or may not be available. + + * ``requires-python`` (string, optional) - the + :ref:`core-metadata-requires-python` metadata field for the file's release. + + * ``yanked`` (boolean or string, optional) - indicates whether the file + should be considered :ref:`yanked ` (if truthy, + using Python :external+python:ref:`truthiness `) or not (if + falsy). + + If this is a string, then it specifies the reason for being yanked. + + * ``core-metadata`` (boolean or object, optional) - indicates whether the + index provide's the file's :ref:`Core Metadata ` (if truthy, + using Python :external+python:ref:`truthiness `) or + not (if falsy). + + If this is an object, then it contains hashes of the metadata, in the same + form as the ``hashes`` file-object key. + + If this key is missing, the metadata may or may not be available. + +Unknown JSON object keys must be ignored. + +An example response document: + +.. code-block:: json + + { + "meta": { + "api-version": "1.0" + }, + "name": "foo", + "files": [ + {"filename": "foo-1.0.0.tar.gz", "url": "/foo/foo-1.0.0.tar.gz"}, + { + "filename": "foo-1.0.1.tar.gz", + "url": "/foo/foo-1.0.1.tar.gz", + "gpg-sig": true, + "requires-python": ">=3.12", + "yanked": "Too much bar", + "core-metadata": {"sha256": "abcd1234"} + } + ] + } + .. _simple_repo_api_yanked: Yanked files @@ -308,6 +479,8 @@ History .. _HTML5: https://html.spec.whatwg.org/ +.. _JSON: https://www.rfc-editor.org/rfc/rfc8259 + .. _anchor element: https://html.spec.whatwg.org/#the-a-element .. _data attribute: https://html.spec.whatwg.org/#attr-data-* From 8d727fec8664c458636e9fc4e9d1a4d58088053d Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 21:45:47 +1000 Subject: [PATCH 17/31] Add PEP 700 --- .../specifications/simple-repository-api.rst | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index b1e7e4d1b..c4dcc05bf 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -350,6 +350,14 @@ represents an object with properties: * ``name`` (string, required) - the :ref:`normalized ` name of the project. +* ``versions`` (array of strings, required) - all of the project versions + uploaded for this project. It must not contain duplicates, and the order is + not significant. All files must be associated with a version in this array, + but not all versions need files associated. These versions should be + :ref:`normalized `. + + *New in API v1.1* + * ``files`` (array of objects, required) - files list. Each file provided by the index for the project corresponds to an element in this array, and vice versa. Objects have properties: @@ -387,6 +395,17 @@ represents an object with properties: If this key is missing, the metadata may or may not be available. + * ``size`` (number, required) - file size in integer bytes. + + *New in API v1.1* + + * ``upload-time`` (string, optional) - file upload time, as an ISO 8601 + date/time string in the UTC timezone using a ``Z`` suffix with precision + between seconds and microseconds: in the format + ``YYYY-mm-ddTHH:MM:SS.ffffffZ`` (number of ``f``'s variable). + + *New in API v1.1* + Unknown JSON object keys must be ignored. An example response document: From 9810653039137314aaac3f93900e31b7e1ceda72 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 21:46:01 +1000 Subject: [PATCH 18/31] Document files-list unordered --- source/specifications/simple-repository-api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index c4dcc05bf..7591d1d67 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -228,7 +228,7 @@ name ` of the project. This endpoint returns some metadata of the :term:`project `, along with a list of all :term:`package files ` provided by the -index for the project. +index for the project. This list of files is not necessarily ordered. If a client uses an unnormalized name for ````, the index server may redirect to the URL with the normalized name. Conformant client must always From 602d0b4c8b7b08a094671eef97cc43e82b29f0ac Mon Sep 17 00:00:00 2001 From: Laurie O Date: Fri, 8 Dec 2023 21:46:22 +1000 Subject: [PATCH 19/31] Hashlib references with intersphinx --- source/specifications/simple-repository-api.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 7591d1d67..69bc02833 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -243,8 +243,9 @@ API file-related features: * Hashes of the file's contents are optional but recommended. The hash name is the name of the hash algorithm's function, and the value is the hex-encoded - digest hash. The function should be one in the standard-library ``hashlib`` - module, and ``sha256`` is preferred. + digest hash. The function should be one in the standard-library + :external+python:mod:`hashlib` module, and + :external+python:func:`hashlib.sha256` is preferred. * A `GPG signature `_ for the file can be accessed at the same URL as the file but with ``.asc`` From c6cec4eac23ad7632062082709a7b51b3abac13f Mon Sep 17 00:00:00 2001 From: Laurie O Date: Mon, 11 Dec 2023 11:01:09 +1000 Subject: [PATCH 20/31] Single-underscore OpenAPI spec reference --- source/specifications/simple-repository-api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 69bc02833..0b0a9ca16 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -26,7 +26,7 @@ Specification .. collapse:: OpenAPI - An `OpenAPI `__ document which + An `OpenAPI `_ document which specifies the full API. It declares both representations, but only details the JSON format's schema. From c53b9a9755c89b5a8bd7747ddc04b29f408d4286 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Mon, 11 Dec 2023 11:03:39 +1000 Subject: [PATCH 21/31] Make reference names consistent --- .../specifications/simple-repository-api.rst | 26 +++++++++---------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 0b0a9ca16..5ccd6f6c7 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -127,8 +127,8 @@ Endpoints The API consists of two metadata endpoints: -* :ref:`repo_api_projects_list` -* :ref:`repo_api_project_details` +* :ref:`simple-repository-api-projects-list` +* :ref:`simple-repository-api-project-details` The root URL ``/`` represents the base URL, where it would be prefixed with the index's URL to construct the full URL which tools make the request for. @@ -137,7 +137,7 @@ If a client makes a request to a URL without a trailing forward-slash ``/``, then the index server should redirect the client to the same URL with the ``/`` appended. -.. _repo_api_projects_list: +.. _simple-repository-api-projects-list: Projects list ------------- @@ -165,8 +165,8 @@ Each project provided by the index has a corresponding `anchor element`_ :ref:`normalized `). * Its ``href`` attribute must exist and is a URL to the :ref:`project details - ` page for the project. This URL must end with a - forward-slash ``/``, but may be absolute or relative. + ` page for the project. This URL must + end with a forward-slash ``/``, but may be absolute or relative. An example response page: @@ -218,7 +218,7 @@ An example response document: ] } -.. _repo_api_project_details: +.. _simple-repository-api-project-details: Project details --------------- @@ -257,7 +257,7 @@ API file-related features: be provided. Clients should ignore the file when installing to an environment for a version of Python which doesn't satisfy the requirement. -* Files may be marked as :ref:`yanked `. +* Files may be marked as :ref:`yanked `. * The file's :ref:`Core Metadata ` must be provided if its existence is indicated. In addition, the file must contain this metadata @@ -303,8 +303,8 @@ corresponding `anchor element`_ ````: greater-than ``>`` becomes the string ``>``). * A ``data-yanked`` `data attribute`_ may exist to indicate the file was - :ref:`yanked `. The attribute may have a value which - specifies the reason the file is yanked. + :ref:`yanked `. The attribute may have a value + which specifies the reason the file is yanked. * A ``data-core-metadata`` `data attribute`_ may exist to indicate the index provides the file's core-metadata. The attribute's value should be of the @@ -380,9 +380,9 @@ represents an object with properties: :ref:`core-metadata-requires-python` metadata field for the file's release. * ``yanked`` (boolean or string, optional) - indicates whether the file - should be considered :ref:`yanked ` (if truthy, - using Python :external+python:ref:`truthiness `) or not (if - falsy). + should be considered :ref:`yanked ` (if + truthy, using Python :external+python:ref:`truthiness `) or not + (if falsy). If this is a string, then it specifies the reason for being yanked. @@ -431,7 +431,7 @@ An example response document: ] } -.. _simple_repo_api_yanked: +.. _simple-repository-api-yanked: Yanked files ############ From 2bc32e286c3ae0ae10f8bad1037173bc055dc535 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Mon, 11 Dec 2023 11:08:10 +1000 Subject: [PATCH 22/31] Fix grammar --- source/specifications/simple-repository-api.rst | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 5ccd6f6c7..281c85b1b 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -8,8 +8,9 @@ Simple repository API The simple repository API is an HTTP-based protocol for tools to list and download Python :term:`packages `. It is the API which :term:`package indexes ` implement to provide package managers -(eg :ref:`pip`) enough information to determine what to install for a given set -of :term:`requirements `, then go on to install those packages. +(for example, :ref:`pip`) enough information to determine what to install for a +given set of :term:`requirements `, then go on to install those +packages. There is one version series for the API: version 1. Minor versions add optional features, and are described below. @@ -239,7 +240,7 @@ API file-related features: * The file can be hosted anywhere, not necessarily by the index server. * The file's URL in the list-item is a URL to fetch the file. It may be - absolute or relative. It's last path segment must be the file's filename. + absolute or relative. Its last path segment must be the file's filename. * Hashes of the file's contents are optional but recommended. The hash name is the name of the hash algorithm's function, and the value is the hex-encoded @@ -437,7 +438,7 @@ Yanked files ############ A yanked :term:`package file ` is one intended to be -now-unavailable for installation from the index. The file's yank-status can be +now-unavailable for installation from the index. The file's yank status can be changed at anypoint (to be unyanked, or even yanked again). Indexes may provide a textual reason for why the file has been yanked, and @@ -458,8 +459,8 @@ to install a yanked release and not install anything. Installers should follow the spirit of the intention quoted above and prevent new dependencies on yanked releases and files. -Installers should emit a warning when it does decide to install a yanked file. -That warning may utilize the reason for the yanking. +Installers should emit a warning if they decide to install a yanked file. That +warning may utilize the reason for the yanking. What this means is left up to the specific installer, to decide how to best fit into the overall usage of their installer. However, there are two suggested @@ -476,8 +477,8 @@ approaches to take: installed. In this case, a yanked file SHOULD not be used when creating or updating a lock file from some input file or command. -Mirror indexes may omit list-items for yanked files in their responses to -clients, or may include list-items for yanked files along with their +Mirror indexes may omit list items for yanked files in their responses to +clients, or may include list items for yanked files along with their yank-status (this status must be present for yanked files). History From e016b328158daa7f05bb2d25dd60672c6000dbaa Mon Sep 17 00:00:00 2001 From: Laurie O Date: Mon, 11 Dec 2023 11:24:31 +1000 Subject: [PATCH 23/31] Replace PEP 440 reference with guide ref --- source/specifications/simple-repository-api.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 281c85b1b..b2d86aea3 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -469,8 +469,9 @@ approaches to take: * Yanked files are always ignored, unless they are the only file that matches a version specifier that “pins” to an exact version using either ``==`` (without any modifiers that make it a range, such as ``.*``) or ``===``. - Matching this version specifier should otherwise be done as per :pep:`440` - for things like local versions, zero padding, etc. + Matching this version specifier should otherwise be done as per the] + :ref:`version specifier specification ` for things + like local versions, zero padding, etc. * Yanked files are always ignored, unless they are the only file that matches what a lock file (such as Pipfile.lock or poetry.lock) specifies to be From 2154e29cecd472d598dc5852b13a3637d07dd94e Mon Sep 17 00:00:00 2001 From: Laurie O Date: Thu, 14 Dec 2023 20:19:55 +1000 Subject: [PATCH 24/31] Remove license from OpenAPI doc Inherit from the rest of the guide instead --- source/specifications/simple-repository-api.openapi.yml | 3 --- 1 file changed, 3 deletions(-) diff --git a/source/specifications/simple-repository-api.openapi.yml b/source/specifications/simple-repository-api.openapi.yml index 118d2f468..c6b8fe33d 100644 --- a/source/specifications/simple-repository-api.openapi.yml +++ b/source/specifications/simple-repository-api.openapi.yml @@ -4,9 +4,6 @@ info: title: Simple Repository API version: '1.1' summary: A protocol for tools to list and and download Python packages - license: - name: Creative Commons Attribution-ShareAlike - url: https://creativecommons.org/licenses/by-sa/3.0 paths: /: From f6ff4524e2da9fc2f34d2e7702f1892011824fd9 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Thu, 14 Dec 2023 20:25:48 +1000 Subject: [PATCH 25/31] Move 'yanked' intention to footnote --- .../specifications/simple-repository-api.rst | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index b2d86aea3..6f7c4770a 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -444,20 +444,11 @@ changed at anypoint (to be unyanked, or even yanked again). Indexes may provide a textual reason for why the file has been yanked, and clients may display that reason to end-users. -From :pep:`592`, the intention for the behaviour with yanked files is: - - The desirable experience for users is that once a file is yanked, when a - human being is currently trying to directly install a yanked file, that it - fails as if that file had been deleted. However, when a human did that - awhile ago, and now a computer is just continuing to mechanically follow the - original order to install the now yanked file, then it acts as if it had not - been yanked. - Installers must ignore yanked :term:`releases ` if a non-yanked release satisfies the :term:`requirement `. Installers may refuse to install a yanked release and not install anything. Installers should follow -the spirit of the intention quoted above and prevent new dependencies on yanked -releases and files. +the spirit of the intention of yanked files [#f1]_ and prevent new dependencies +on yanked releases and files. Installers should emit a warning if they decide to install a yanked file. That warning may utilize the reason for the yanking. @@ -499,6 +490,15 @@ History * June 2023: renaming the field which provides package metadata independently from a package, in :pep:`714` +.. rubric:: Footnotes + +.. [#f1] The desirable experience for users is that once a file is yanked, when + a human being is currently trying to directly install a yanked file, that it + fails as if that file had been deleted. However, when a human did that + awhile ago, and now a computer is just continuing to mechanically follow the + original order to install the now yanked file, then it acts as if it had not + been yanked. + .. _HTML5: https://html.spec.whatwg.org/ .. _JSON: https://www.rfc-editor.org/rfc/rfc8259 From 1d3c63e6c1fd67d367fafd4296ef0392fb3697bd Mon Sep 17 00:00:00 2001 From: Laurie O Date: Thu, 14 Dec 2023 20:30:53 +1000 Subject: [PATCH 26/31] Add content-type format description as a footnote --- source/specifications/simple-repository-api.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 6f7c4770a..c085b8e03 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -58,7 +58,7 @@ Representations ############### The index server can respond with one of two different representation formats -for each endpoint: +for each endpoint [#f2]_: * `HTML5`_: content-type ``application/vnd.pypi.simple.v1+html`` @@ -70,9 +70,6 @@ for each endpoint: * ``text/html`` is an alias for this content-type -See :pep:`691` for the details of how these values should be updated for new -versions of the API. - Content negotiation ------------------- @@ -492,6 +489,10 @@ History .. rubric:: Footnotes +.. [#f2] The format of this content type follows + ``application/vnd.pypi.simple.$version+format``, where ``$version`` is the + major version of the API prefixed with ``v``. + .. [#f1] The desirable experience for users is that once a file is yanked, when a human being is currently trying to directly install a yanked file, that it fails as if that file had been deleted. However, when a human did that From 62fd78eda1618f1857335b323f39e55158c3d949 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Mon, 18 Dec 2023 12:35:32 +1000 Subject: [PATCH 27/31] Remove redundant adjective --- source/specifications/simple-repository-api.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index c085b8e03..8bb7debb7 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -251,8 +251,8 @@ API file-related features: ``/packages/HolyGrail-1.0.tar.gz`` may have a signature at ``/packages/HolyGrail-1.0.tar.gz.asc``. -* The file's release's :ref:`core-metadata-requires-python` metadata field may - be provided. Clients should ignore the file when installing to an environment +* The file's :ref:`core-metadata-requires-python` metadata field may be + provided. Clients should ignore the file when installing to an environment for a version of Python which doesn't satisfy the requirement. * Files may be marked as :ref:`yanked `. From f26cb155e652d37c4e291780fecb47c47725c69b Mon Sep 17 00:00:00 2001 From: Laurie O Date: Mon, 18 Dec 2023 12:40:10 +1000 Subject: [PATCH 28/31] Use 'versionadded' directive --- source/specifications/simple-repository-api.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 8bb7debb7..6248f87e7 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -355,7 +355,7 @@ represents an object with properties: but not all versions need files associated. These versions should be :ref:`normalized `. - *New in API v1.1* + .. versionadded:: 1.1 * ``files`` (array of objects, required) - files list. Each file provided by the index for the project corresponds to an element in this array, and vice @@ -396,14 +396,14 @@ represents an object with properties: * ``size`` (number, required) - file size in integer bytes. - *New in API v1.1* + .. versionadded:: 1.1 * ``upload-time`` (string, optional) - file upload time, as an ISO 8601 date/time string in the UTC timezone using a ``Z`` suffix with precision between seconds and microseconds: in the format ``YYYY-mm-ddTHH:MM:SS.ffffffZ`` (number of ``f``'s variable). - *New in API v1.1* + .. versionadded:: 1.1 Unknown JSON object keys must be ignored. From ec1068a3fbf2b5971780ee6c630fbc225388194a Mon Sep 17 00:00:00 2001 From: Laurie O Date: Tue, 2 Jan 2024 14:33:46 +1000 Subject: [PATCH 29/31] Remove unnecessary hyphen --- source/specifications/simple-repository-api.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index 6248f87e7..e51b0b484 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -468,7 +468,7 @@ approaches to take: Mirror indexes may omit list items for yanked files in their responses to clients, or may include list items for yanked files along with their -yank-status (this status must be present for yanked files). +yank status (this status must be present for yanked files). History ======= From 7e583aa4b1a028c7138f064699a97097942ead7e Mon Sep 17 00:00:00 2001 From: Laurie O Date: Tue, 2 Jan 2024 14:34:12 +1000 Subject: [PATCH 30/31] Fix representation-specific content-types --- source/specifications/simple-repository-api.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index e51b0b484..b693eb7ec 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -62,11 +62,11 @@ for each endpoint [#f2]_: * `HTML5`_: content-type ``application/vnd.pypi.simple.v1+html`` - * Clients can also request ``application/vnd.pypi.simple.latest+json`` + * Clients can also request ``application/vnd.pypi.simple.latest+html`` -* `JSON`_: content-type ``application/vnd.pypi.simple.v1+html`` +* `JSON`_: content-type ``application/vnd.pypi.simple.v1+json`` - * Clients can also request ``application/vnd.pypi.simple.latest+html`` + * Clients can also request ``application/vnd.pypi.simple.latest+json`` * ``text/html`` is an alias for this content-type From f622a7588b2fe0e62a98778d266ddde8c52ec0d3 Mon Sep 17 00:00:00 2001 From: Laurie O Date: Tue, 2 Jan 2024 14:34:38 +1000 Subject: [PATCH 31/31] Refer to representation-specific details for core-metadata file existence --- source/specifications/simple-repository-api.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/specifications/simple-repository-api.rst b/source/specifications/simple-repository-api.rst index b693eb7ec..856b21195 100644 --- a/source/specifications/simple-repository-api.rst +++ b/source/specifications/simple-repository-api.rst @@ -258,9 +258,9 @@ API file-related features: * Files may be marked as :ref:`yanked `. * The file's :ref:`Core Metadata ` must be provided if its - existence is indicated. In addition, the file must contain this metadata - which will not be modified when the distribution is processed and/or - installed. + existence is indicated (by one of the representation-specific mechanisms + described below). In addition, the file must contain this metadata which will + not be modified when the distribution is processed and/or installed. The metadata must be accessed at the same URL as the file but with ``.metadata`` appended. For example, the file at