Worklow to publish a package from a CODEOWNER tag #598

NathanielRN · 2021-07-21T01:06:30Z

Description

Adds a new workflow (.github/publish-a-package.yml) and a new build script (scripts/build_a_package.sh) to allow releases of contrib repo packages from a push of a tag.

The hope is that once we approve CODEOWNERS who own contrib packages, they can release packages independent of the maintainers by simply pushing a tag with a package name and package version that matches a package in this repo.

The format of the tag should be:

opentelemetry-<PKG_NAME>==<PKG_VERSION>

NOTE: You can only release a package that is >=1.0! This is to allow beta packages to be released by the regular release workflow when the maintainers have a chance to do it.

I also updated the original script to skip .tar.gz files which are >=1.0 for this very reason.

This is very similar to the setup over on opentelemtry-dotnet-contrib where they have a separate workflow for every package. That workflow gets triggered on a push to a relevant tag. However, my goal here is to one have one workflow for each package.

Also see the .NET PR that added this functionality: open-telemetry/opentelemetry-dotnet-contrib#64

The PRO is less code duplication.

The CON is less control by CODEOWNERS as to what the workflow does for their specific package (although we can address those later on a case-by-case need)

Type of change

Please delete options that are not relevant.

New feature (non-breaking change which adds functionality)
~~- [ ] This change requires a documentation update~~ No?

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

You can run ./scripts/build_a_package.sh locally, you just need to:
- Set GITHUB_REF= to something like refs/tags/opentelemetry-sdk-extension-aws==0.23b0
- Update the BASE_DIR variable to the location of your contrib repo
To test the actual workflow, I'll try to create my own repo sometime this week to validate everything works.

Does This PR Require a Core Repo Change?

No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

Followed the style guidelines of this project
~~- [ ] Changelogs have been updated~~
~~- [ ] Unit tests have been added~~ Didn't see any for the previous release flow?
~~- [ ] Documentation has been updated~~

.github/workflows/publish-a-package.yml

NathanielRN · 2021-07-21T01:09:24Z

.github/workflows/publish-a-package.yml

+
+jobs:
+  publish:
+    name: Publish package from tag ${{ env.GITHUB_REF }}


This variable is crucial to make this whole setup work. The variable is defined on GitHub's docs and I took inspiration from this SO post on how to get the currently pushed tag in GitHub actions for the rest of the PR.

.github/workflows/publish-a-package.yml

scripts/build_a_package.sh

aabmass · 2021-07-21T15:50:24Z

Nice this is pretty neat

Adds a new workflow (.github/publish-a-package.yml) and a new build script (scripts/build_a_package.sh) to allow releases of contrib repo packages from a push of a tag.

How would a CODEOWNER push to a tag? Wouldn't this still require the CODEOWNER to have push access?

NathanielRN · 2021-07-21T18:58:05Z

@aabmass Thanks for the review! It prompted me to test the results out on my own repo and validate that it works 🙂

Here is a workflow where the publish fails because the tag package name does not correspond to any package.

Here is another workflow where the publish fails because the tag matches a package but does not match the current version of the package in the repo.

Let me know if you have feedback on these?

scripts/build_a_package.sh

aabmass · 2021-07-22T23:35:34Z

scripts/build_a_package.sh

+  exit -1
+fi
+
+PKG_NAME_AND_VERSION=${GITHUB_REF#refs/tags/*}


One other bash nit, i'd use lowercase variable names for non-environment variables.

That's a good idea! I was following ./scripts/build.sh but we should probably update that one instead. Updated!

ocelotl · 2021-07-27T15:10:45Z

scripts/build_a_package.sh

@@ -0,0 +1,68 @@
+#!/bin/sh


Not asking you to translate this, but maybe we should consider writing these tools in Python.

I think we definitely could! I just wrote it like this for now so people could compare it against the ./build.sh in the same repo and see the difference 🙂 I think it's a small enough script to change to Python in the near future if we want!

scripts/build_a_package.sh

aabmass · 2021-08-02T13:31:11Z

Code LGTM but I'm not sure if we've agreed on the approach enough to merge yet. My understanding is this isn't needed for a while until that checklist is complete, right?

lzchen · 2021-08-05T16:27:10Z

LGTM.

@codeboten
Thoughts on this?

owais

Would suggest to run a test with a test/local pypi server and a fake package.

owais · 2021-08-19T12:26:51Z

.github/workflows/publish-a-package.yml

+      run: |
+        pip install twine
+    # We don't need to publish to TestPyPI because we only publish 1 package.
+    # If it fails no other work needs to be reversed.


This PR is fine but I think we should add this and then verify the package by installing it with pip. Once everything works as expected, only then we publish to real pypi service.

Not asking we do it in this PR. Can we create a ticket for it?

Sounds good! I made #622 🙂

NathanielRN requested review from a team, ocelotl and lzchen and removed request for a team July 21, 2021 01:06

NathanielRN commented Jul 21, 2021

View reviewed changes

.github/workflows/publish-a-package.yml Outdated Show resolved Hide resolved

NathanielRN commented Jul 21, 2021

View reviewed changes

aabmass reviewed Jul 21, 2021

View reviewed changes

NathanielRN requested a review from aabmass July 21, 2021 19:05

aabmass reviewed Jul 21, 2021

View reviewed changes

scripts/build_a_package.sh Outdated Show resolved Hide resolved

NathanielRN requested a review from aabmass July 21, 2021 21:40

aabmass reviewed Jul 22, 2021

View reviewed changes

NathanielRN requested a review from aabmass July 23, 2021 03:46

ocelotl approved these changes Jul 27, 2021

View reviewed changes

ocelotl added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Aug 5, 2021

NathanielRN force-pushed the separate-package-releases branch from 4729936 to 9b24978 Compare August 5, 2021 16:07

lzchen assigned codeboten Aug 5, 2021

NathanielRN closed this Aug 6, 2021

NathanielRN reopened this Aug 6, 2021

NathanielRN added 6 commits August 17, 2021 09:29

Worklow to publish a package from a CODEOWNER tag

35cec35

Prevent main script from release >=1.0 packages

9e0780c

Fixes to build package workflow and script

d9d40e8

Better naming and remove redundant ls flag

abeb606

Add err msg for missing setup.py ls match

4cd3fce

Use lowercase for all non env variable vars

56ebd0e

NathanielRN force-pushed the separate-package-releases branch from 9b24978 to 56ebd0e Compare August 17, 2021 16:29

NathanielRN mentioned this pull request Aug 17, 2021

README includes 1.0 release checklist + CODEOWNER description #596

Merged

4 tasks

aabmass approved these changes Aug 18, 2021

View reviewed changes

owais approved these changes Aug 19, 2021

View reviewed changes

NathanielRN mentioned this pull request Aug 19, 2021

Before we publish a pkg to PyPi, install release candidate package with pip to verify it #622

Open

owais merged commit d9c0116 into open-telemetry:main Aug 19, 2021

NathanielRN deleted the separate-package-releases branch August 19, 2021 18:33

NathanielRN mentioned this pull request Sep 8, 2021

Release AWS Python SDK Extension as 1.0 #667

Merged

3 tasks

NathanielRN mentioned this pull request Oct 14, 2021

Fix build script to not release stable pkgs #743

Merged

3 tasks

lzchen mentioned this pull request Oct 19, 2021

Prepare release v1.6.2 and 0.25b2 #762

Merged

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Worklow to publish a package from a CODEOWNER tag #598

Worklow to publish a package from a CODEOWNER tag #598

NathanielRN commented Jul 21, 2021 •

edited

Loading

NathanielRN Jul 21, 2021

aabmass commented Jul 21, 2021 •

edited

Loading

NathanielRN commented Jul 21, 2021

aabmass Jul 22, 2021

NathanielRN Jul 23, 2021

ocelotl Jul 27, 2021

NathanielRN Jul 27, 2021

aabmass commented Aug 2, 2021

lzchen commented Aug 5, 2021

owais left a comment

owais Aug 19, 2021

owais Aug 19, 2021

NathanielRN Aug 19, 2021

Worklow to publish a package from a CODEOWNER tag #598

Worklow to publish a package from a CODEOWNER tag #598

Conversation

NathanielRN commented Jul 21, 2021 • edited Loading

Description

Type of change

How Has This Been Tested?

Does This PR Require a Core Repo Change?

Checklist:

Choose a reason for hiding this comment

aabmass commented Jul 21, 2021 • edited Loading

NathanielRN commented Jul 21, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aabmass commented Aug 2, 2021

lzchen commented Aug 5, 2021

owais left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NathanielRN commented Jul 21, 2021 •

edited

Loading

aabmass commented Jul 21, 2021 •

edited

Loading