Skip to content

Brittle TSA tests #1378

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ramonpetgrave64 opened this issue May 9, 2025 · 1 comment
Closed

Brittle TSA tests #1378

ramonpetgrave64 opened this issue May 9, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@ramonpetgrave64
Copy link
Contributor

Description

Version

When the sigstore/timestamp-authority has cut a new tag, but not a new release, the tests fail.

https://github.com/sigstore/sigstore-python/actions/runs/14933726057/job/41956125514?pr=1377#step:7:1

Run # Fetch the latest sigstore/timestamp-authority build
  # Fetch the latest sigstore/timestamp-authority build
  SIGSTORE_TIMESTAMP_VERSION=$(gh api /repos/sigstore/timestamp-authority/tags --jq '.[0].name')
  wget https://github.com/sigstore/timestamp-authority/releases/download/${SIGSTORE_TIMESTAMP_VERSION}/timestamp-server-linux-amd64 -O /tmp/timestamp-server
  chmod +x /tmp/timestamp-server
  
  # Run the TSA in background
  /tmp/timestamp-server serve --port 3000 --disable-ntp-monitoring &
  export TEST_SIGSTORE_TIMESTAMP_AUTHORITY_URL="http://localhost:3000/api/v1/timestamp"
  
  # Ensure Timestamp Authority tests are not skipped by
  # having pytest show skipped tests and verifying ours are running
  make test TEST_ARGS="-m timestamp_authority -rs" | tee output
  ! grep -q "skipping test that requires a Timestamp Authority" output || (echo "ERROR: Found skip message" && exit 1)
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.1[2](https://github.com/sigstore/sigstore-python/actions/runs/14933726057/job/41956125514?pr=1377#step:7:2).10/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/[3](https://github.com/sigstore/sigstore-python/actions/runs/14933726057/job/41956125514?pr=1377#step:7:3).12.10/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.10/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.10/x6[4](https://github.com/sigstore/sigstore-python/actions/runs/14933726057/job/41956125514?pr=1377#step:7:4)
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.10/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.10/x64/lib
    GH_TOKEN: ***
--202[5](https://github.com/sigstore/sigstore-python/actions/runs/14933726057/job/41956125514?pr=1377#step:7:5)-05-09 16:49:59--  https://github.com/sigstore/timestamp-authority/releases/download/v1.2.7/timestamp-server-linux-amd[6](https://github.com/sigstore/sigstore-python/actions/runs/14933726057/job/41956125514?pr=1377#step:7:6)4
Resolving github.com (github.com)... 140.[8](https://github.com/sigstore/sigstore-python/actions/runs/14933726057/job/41956125514?pr=1377#step:7:8)2.112.3
Connecting to github.com (github.com)|140.82.112.3|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2025-05-0[9](https://github.com/sigstore/sigstore-python/actions/runs/14933726057/job/41956125514?pr=1377#step:7:9) 16:49:59 ERROR 404: Not Found.

We should edit the test to pull from the latest release instead, by changing

gh api /repos/sigstore/timestamp-authority/tags --jq '.[0].name'

to

gh api /repos/sigstore/timestamp-authority/releases --jq '.[0].tag_name
@ramonpetgrave64 ramonpetgrave64 added the bug Something isn't working label May 9, 2025
@jku
Copy link
Member

jku commented May 14, 2025

fixed in #1377

@jku jku closed this as completed May 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants