Skip to content

Commit 2b46bad

Browse files
woodruffwwebknjaz
andcommitted
OIDC beta support
Co-authored-by: Sviatoslav Sydorenko <[email protected]>
1 parent 22b4d1f commit 2b46bad

8 files changed

+234
-2
lines changed

.pre-commit-config.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,11 +105,15 @@ repos:
105105
name: flake8 WPS-only
106106
args:
107107
- --ignore
108+
# NOTE: WPS326: Found implicit string concatenation
109+
# NOTE: WPS332: Found walrus operator
108110
- >-
109111
WPS102,
110112
WPS110,
111113
WPS111,
112114
WPS305,
115+
WPS326,
116+
WPS332,
113117
WPS347,
114118
WPS360,
115119
WPS421,

Dockerfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ WORKDIR /app
2525
COPY LICENSE.md .
2626
COPY twine-upload.sh .
2727
COPY print-hash.py .
28+
COPY oidc-exchange.py .
2829

2930
RUN chmod +x twine-upload.sh
3031
ENTRYPOINT ["/app/twine-upload.sh"]

README.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,51 @@ The secret used in `${{ secrets.PYPI_API_TOKEN }}` needs to be created on the
6262
settings page of your project on GitHub. See [Creating & using secrets].
6363

6464

65+
### Publishing with OpenID Connect
66+
67+
> **IMPORTANT**: This functionality is in beta, and will not work for you
68+
> unless you're a member of the PyPI OIDC beta testers' group. For more
69+
> information, see [warehouse#12965].
70+
71+
This action supports PyPI's [OpenID Connect publishing]
72+
implementation, which allows authentication to PyPI without a manually
73+
configured API token or username/password combination. To perform
74+
[OIDC publishing][OpenID Connect Publishing] with this action, your project's
75+
OIDC publisher must already be configured on PyPI.
76+
77+
To enter the OIDC flow, configure this action's job with the `id-token: write`
78+
permission and **without** an explicit username or password:
79+
80+
```yaml
81+
jobs:
82+
pypi-publish:
83+
name: Upload release to PyPI
84+
runs-on: ubuntu-latest
85+
permissions:
86+
id-token: write # IMPORTANT: this permission is mandatory for OIDC publishing
87+
steps:
88+
# retrieve your distributions here
89+
90+
- name: Publish package distributions to PyPI
91+
uses: pypa/gh-action-pypi-publish@release/v1
92+
```
93+
94+
Other indices that support OIDC publishing can also be used, like TestPyPI:
95+
96+
```yaml
97+
- name: Publish package distributions to TestPyPI
98+
uses: pypa/gh-action-pypi-publish@release/v1
99+
with:
100+
repository-url: https://test.pypi.org/legacy/
101+
```
102+
103+
> **Pro tip**: only set the `id-token: write` permission in the job that does
104+
> publishing, not globally. Also, try to separate building from publishing
105+
> — this makes sure that any scripts maliciously injected into the build
106+
> or test environment won't be able to elevate privileges while flying under
107+
> the radar.
108+
109+
65110
## Non-goals
66111

67112
This GitHub Action [has nothing to do with _building package
@@ -221,3 +266,6 @@ https://packaging.python.org/glossary/#term-Distribution-Package
221266
https://raw.githubusercontent.com/vshymanskyy/StandWithUkraine/main/banner-direct-single.svg
222267
[SWUdocs]:
223268
https://github.com/vshymanskyy/StandWithUkraine/blob/main/docs/README.md
269+
270+
[warehouse#12965]: https://github.com/pypi/warehouse/issues/12965
271+
[OpenID Connect Publishing]: https://pypi.org/help/#openid-connect

action.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ inputs:
2020
The inputs have been normalized to use kebab-case.
2121
Use `repository-url` instead.
2222
required: false
23+
default: https://pypi.org/legacy/
2324
packages-dir: # Canonical alias for `packages_dir`
2425
description: The target directory for distribution
2526
required: false

oidc-exchange.py

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
import os
2+
import sys
3+
from http import HTTPStatus
4+
from pathlib import Path
5+
from typing import NoReturn
6+
from urllib.parse import urlparse
7+
8+
import id # pylint: disable=redefined-builtin
9+
import requests
10+
11+
_GITHUB_STEP_SUMMARY = Path(os.getenv("GITHUB_STEP_SUMMARY"))
12+
13+
# Rendered if OIDC identity token retrieval fails for any reason.
14+
_TOKEN_RETRIEVAL_FAILED_MESSAGE = """
15+
OIDC token retrieval failed: {identity_error}
16+
17+
This generally indicates a workflow configuration error, such as insufficient
18+
permissions. Make sure that your workflow has `id-token: write` configured
19+
at the job level, e.g.:
20+
21+
```yaml
22+
permissions:
23+
id-token: write
24+
```
25+
26+
Learn more at https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#adding-permissions-settings.
27+
"""
28+
29+
# Rendered if the package index refuses the given OIDC token.
30+
_SERVER_REFUSED_TOKEN_EXCHANGE_MESSAGE = """
31+
Token request failed: the server refused the request for the following reasons:
32+
33+
{reasons}
34+
"""
35+
36+
# Rendered if the package index's token response isn't valid JSON.
37+
_SERVER_TOKEN_RESPONSE_MALFORMED_JSON = """
38+
Token request failed: the index produced an unexpected
39+
{status_code} response.
40+
41+
This strongly suggests a server configuration or downtime issue; wait
42+
a few minutes and try again.
43+
"""
44+
45+
# Rendered if the package index's token response isn't a valid API token payload.
46+
_SERVER_TOKEN_RESPONSE_MALFORMED_MESSAGE = """
47+
Token response error: the index gave us an invalid response.
48+
49+
This strongly suggests a server configuration or downtime issue; wait
50+
a few minutes and try again.
51+
"""
52+
53+
54+
def die(msg: str) -> NoReturn:
55+
with _GITHUB_STEP_SUMMARY.open("a", encoding="utf-8") as io:
56+
print(msg, file=io)
57+
58+
# NOTE: `msg` is Markdown formatted, so we emit only the header line to
59+
# avoid clogging the console log with a full Markdown formatted document.
60+
header = msg.splitlines()[0]
61+
print(f"::error::OIDC exchange failure: {header}", file=sys.stderr)
62+
sys.exit(1)
63+
64+
65+
def debug(msg: str):
66+
print(f"::debug::{msg.title()}", file=sys.stderr)
67+
68+
69+
def get_normalized_input(name: str) -> str | None:
70+
name = f"INPUT_{name.upper()}"
71+
if val := os.getenv(name):
72+
return val
73+
return os.getenv(name.replace("-", "_"))
74+
75+
76+
def assert_successful_audience_call(resp: requests.Response, domain: str):
77+
if resp.ok:
78+
return
79+
80+
match resp.status_code:
81+
case HTTPStatus.FORBIDDEN:
82+
# This index supports OIDC, but forbids the client from using
83+
# it (either because it's disabled, limited to a beta group, etc.)
84+
die(f"audience retrieval failed: repository at {domain} has OIDC disabled")
85+
case HTTPStatus.NOT_FOUND:
86+
# This index does not support OIDC.
87+
die(
88+
"audience retrieval failed: repository at "
89+
f"{domain} does not indicate OIDC support",
90+
)
91+
case other:
92+
status = HTTPStatus(other)
93+
# Unknown: the index may or may not support OIDC, but didn't respond with
94+
# something we expect. This can happen if the index is broken, in maintenance mode,
95+
# misconfigured, etc.
96+
die(
97+
"audience retrieval failed: repository at "
98+
f"{domain} responded with unexpected {other}: {status.phrase}",
99+
)
100+
101+
102+
repository_url = get_normalized_input("repository-url")
103+
repository_domain = urlparse(repository_url).netloc
104+
token_exchange_url = f"https://{repository_domain}/_/oidc/github/mint-token"
105+
106+
# Indices are expected to support `https://{domain}/_/oidc/audience`,
107+
# which tells OIDC exchange clients which audience to use.
108+
audience_url = f"https://{repository_domain}/_/oidc/audience"
109+
audience_resp = requests.get(audience_url)
110+
assert_successful_audience_call(audience_resp, repository_domain)
111+
112+
oidc_audience = audience_resp.json()["audience"]
113+
114+
debug(f"selected OIDC token exchange endpoint: {token_exchange_url}")
115+
116+
try:
117+
oidc_token = id.detect_credential(audience=oidc_audience)
118+
except id.IdentityError as identity_error:
119+
die(_TOKEN_RETRIEVAL_FAILED_MESSAGE.format(identity_error=identity_error))
120+
121+
# Now we can do the actual token exchange.
122+
mint_token_resp = requests.post(
123+
token_exchange_url,
124+
json={"token": oidc_token},
125+
)
126+
127+
try:
128+
mint_token_payload = mint_token_resp.json()
129+
except requests.JSONDecodeError:
130+
# Token exchange failure normally produces a JSON error response, but
131+
# we might have hit a server error instead.
132+
die(
133+
_SERVER_TOKEN_RESPONSE_MALFORMED_JSON.format(
134+
status_code=mint_token_resp.status_code,
135+
),
136+
)
137+
138+
# On failure, the JSON response includes the list of errors that
139+
# occurred during minting.
140+
if not mint_token_resp.ok:
141+
reasons = "\n".join(
142+
f"* `{error['code']}`: {error['description']}"
143+
for error in mint_token_payload["errors"]
144+
)
145+
146+
die(_SERVER_REFUSED_TOKEN_EXCHANGE_MESSAGE.format(reasons=reasons))
147+
148+
pypi_token = mint_token_payload.get("token")
149+
if pypi_token is None:
150+
die(_SERVER_TOKEN_RESPONSE_MALFORMED_MESSAGE)
151+
152+
# Mask the newly minted PyPI token, so that we don't accidentally leak it in logs.
153+
print(f"::add-mask::{pypi_token}", file=sys.stderr)
154+
155+
# This final print will be captured by the subshell in `twine-upload.sh`.
156+
print(pypi_token)

requirements/runtime.in

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,13 @@
11
twine
22

3+
# NOTE: Used to detect an ambient OIDC credential for OIDC publishing.
4+
id ~= 1.0
5+
6+
# NOTE: This is pulled in transitively through `twine`, but we also declare
7+
# NOTE: it explicitly here because `oidc-exchange.py` uses it.
8+
# Ref: https://github.com/di/id
9+
requests
10+
311
# NOTE: `pkginfo` is a transitive dependency for us that is coming from Twine.
412
# NOTE: It is declared here only to avoid installing a broken combination of
513
# NOTE: the distribution packages. This should be removed once a fixed version

requirements/runtime.txt

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,8 @@ cryptography==39.0.1
1818
# via secretstorage
1919
docutils==0.19
2020
# via readme-renderer
21+
id==1.0.0
22+
# via -r requirements/runtime.in
2123
idna==3.4
2224
# via requests
2325
importlib-metadata==5.1.0
@@ -36,10 +38,12 @@ more-itertools==9.0.0
3638
# via jaraco-classes
3739
pkginfo==1.9.2
3840
# via
39-
# -r runtime.in
41+
# -r requirements/runtime.in
4042
# twine
4143
pycparser==2.21
4244
# via cffi
45+
pydantic==1.10.6
46+
# via id
4347
pygments==2.13.0
4448
# via
4549
# readme-renderer
@@ -48,6 +52,8 @@ readme-renderer==37.3
4852
# via twine
4953
requests==2.28.1
5054
# via
55+
# -r requirements/runtime.in
56+
# id
5157
# requests-toolbelt
5258
# twine
5359
requests-toolbelt==0.10.1
@@ -61,7 +67,9 @@ secretstorage==3.3.3
6167
six==1.16.0
6268
# via bleach
6369
twine==4.0.1
64-
# via -r runtime.in
70+
# via -r requirements/runtime.in
71+
typing-extensions==4.5.0
72+
# via pydantic
6573
urllib3==1.26.13
6674
# via
6775
# requests

twine-upload.sh

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,12 @@ INPUT_VERIFY_METADATA="$(get-normalized-input 'verify-metadata')"
4040
INPUT_SKIP_EXISTING="$(get-normalized-input 'skip-existing')"
4141
INPUT_PRINT_HASH="$(get-normalized-input 'print-hash')"
4242

43+
if [[ "${INPUT_USER}" == "__token__" && -z "${INPUT_PASSWORD}" ]] ; then
44+
# No password supplied by the user implies that we're in the OIDC flow;
45+
# retrieve the OIDC credential and exchange it for a PyPI API token.
46+
echo "::notice::In OIDC flow"
47+
INPUT_PASSWORD="$(python /app/oidc-exchange.py)"
48+
fi
4349

4450
if [[
4551
"$INPUT_USER" == "__token__" &&

0 commit comments

Comments
 (0)