Skip to content

Eric/cus 10 add automation for deploying to test pypi and prod #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
e2d31f6
feat: add github actions for deployment automation
flowstate Nov 6, 2024
01370a7
Packaging moved to main dependencies to support version check code
flowstate Nov 6, 2024
c7b4914
Added dockerhub login steps where needed
flowstate Nov 6, 2024
75b80cc
added the commit sha to the generated preview version to ensure uniqu…
flowstate Nov 6, 2024
e9b9919
Actually setting preview version in init
flowstate Nov 6, 2024
ac18fa2
Still working on the version handling
flowstate Nov 6, 2024
55e0b41
Local versions are not allowed in pypi as per PEP440
flowstate Nov 6, 2024
c738a85
More versioning fun
flowstate Nov 6, 2024
abfc210
PEP 440 specifier shenanigans
flowstate Nov 6, 2024
34ebc53
fixing pr commenting step
flowstate Nov 6, 2024
dc440a1
Forcing the repo name to populate in comment pr step
flowstate Nov 6, 2024
12976ab
better version checking
flowstate Nov 6, 2024
175e962
added a step to wait for new package avail before docker build
flowstate Nov 6, 2024
d14dcc7
querying pypi metadata directly
flowstate Nov 6, 2024
6495d97
waiting longer for the package to appear
flowstate Nov 6, 2024
fbbdfc1
forgot to add the env var to the steps
flowstate Nov 6, 2024
b9b0d25
I hate pypi.
flowstate Nov 7, 2024
923f9cc
resolving weird pip issues, remove argparse as its built-in
flowstate Nov 7, 2024
7a389af
trying a more deterministic approach to package version verification
flowstate Nov 7, 2024
d925f1f
cleaned up comment behavior, updated release and stable
flowstate Nov 7, 2024
fc23da7
made release workflow idempotent
flowstate Nov 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/docker-stable.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Mark Release as Stable
on:
workflow_dispatch:
inputs:
version:
description: 'Version to mark as stable (e.g., 1.2.3)'
required: true

jobs:
stable:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Check if version exists in PyPI
id: version_check
run: |
if ! curl -s -f https://pypi.org/pypi/socketsecurity/${{ inputs.version }}/json > /dev/null; then
echo "Error: Version ${{ inputs.version }} not found on PyPI"
exit 1
fi
echo "Version ${{ inputs.version }} found on PyPI - proceeding with release"

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build & Push Stable Docker
uses: docker/build-push-action@v5
with:
push: true
platforms: linux/amd64,linux/arm64
tags: socketdev/cli:stable
build-args: |
CLI_VERSION=${{ inputs.version }}
154 changes: 154 additions & 0 deletions .github/workflows/pr-preview.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
name: PR Preview
on:
pull_request:
types: [opened, synchronize]

jobs:
preview:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.x'

- name: Set preview version
run: |
BASE_VERSION=$(grep -o "__version__.*" socketsecurity/__init__.py | awk '{print $3}' | tr -d "'")
PREVIEW_VERSION="${BASE_VERSION}.dev${{ github.event.pull_request.number }}${{ github.event.pull_request.commits }}"
echo "VERSION=${PREVIEW_VERSION}" >> $GITHUB_ENV

# Update version in __init__.py
echo "__version__ = \"${PREVIEW_VERSION}\"" > socketsecurity/__init__.py.tmp
cat socketsecurity/__init__.py | grep -v "__version__" >> socketsecurity/__init__.py.tmp
mv socketsecurity/__init__.py.tmp socketsecurity/__init__.py

# Verify the change
echo "Updated version in __init__.py:"
cat socketsecurity/__init__.py | grep "__version__"

- name: Check if version exists on Test PyPI
id: version_check
env:
VERSION: ${{ env.VERSION }}
run: |
if curl -s -f https://test.pypi.org/pypi/socketsecurity/$VERSION/json > /dev/null; then
echo "Version ${VERSION} already exists on Test PyPI"
echo "exists=true" >> $GITHUB_OUTPUT
else
echo "Version ${VERSION} not found on Test PyPI"
echo "exists=false" >> $GITHUB_OUTPUT
fi

- name: Build package
if: steps.version_check.outputs.exists != 'true'
run: |
pip install build
python -m build

- name: Restore original version
if: always()
run: |
BASE_VERSION=$(echo $VERSION | cut -d'.' -f1-3)
echo "__version__ = \"${BASE_VERSION}\"" > socketsecurity/__init__.py.tmp
cat socketsecurity/__init__.py | grep -v "__version__" >> socketsecurity/__init__.py.tmp
mv socketsecurity/__init__.py.tmp socketsecurity/__init__.py

- name: Publish to Test PyPI
if: steps.version_check.outputs.exists != 'true'
uses: pypa/[email protected]
with:
repository-url: https://test.pypi.org/legacy/
password: ${{ secrets.TEST_PYPI_TOKEN }}
verbose: true

- name: Comment on PR
if: steps.version_check.outputs.exists != 'true'
uses: actions/github-script@v7
env:
VERSION: ${{ env.VERSION }}
with:
script: |
const version = process.env.VERSION;
const prNumber = context.payload.pull_request.number;
const owner = context.repo.owner;
const repo = context.repo.repo;
// Find existing bot comments
const comments = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
});

const botComment = comments.data.find(comment =>
comment.user.type === 'Bot' &&
comment.body.includes('🚀 Preview package published!')
);

const comment = `
🚀 Preview package published!

Install with:
\`\`\`bash
pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple socketsecurity==${version}
\`\`\`

Docker image: \`socketdev/cli:pr-${prNumber}\`
`;

if (botComment) {
// Update existing comment
await github.rest.issues.updateComment({
owner: owner,
repo: repo,
comment_id: botComment.id,
body: comment
});
} else {
// Create new comment
await github.rest.issues.createComment({
owner: owner,
repo: repo,
issue_number: prNumber,
body: comment
});
}

- name: Verify package is available
if: steps.version_check.outputs.exists != 'true'
id: verify_package
env:
VERSION: ${{ env.VERSION }}
run: |
for i in {1..30}; do
if pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple socketsecurity==${VERSION}; then
echo "Package ${VERSION} is now available and installable on Test PyPI"
pip uninstall -y socketsecurity
echo "success=true" >> $GITHUB_OUTPUT
exit 0
fi
echo "Attempt $i: Package not yet installable, waiting 20s... (${i}/30)"
sleep 20
done
echo "success=false" >> $GITHUB_OUTPUT
exit 1

- name: Login to Docker Hub
if: steps.verify_package.outputs.success == 'true'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build & Push Docker Preview
if: steps.verify_package.outputs.success == 'true'
uses: docker/build-push-action@v5
env:
VERSION: ${{ env.VERSION }}
with:
push: true
tags: socketdev/cli:pr-${{ github.event.pull_request.number }}
build-args: |
CLI_VERSION=${{ env.VERSION }}
PIP_INDEX_URL=https://test.pypi.org/simple
PIP_EXTRA_INDEX_URL=https://pypi.org/simple
99 changes: 99 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
name: Release
on:
push:
tags:
- 'v*'

jobs:
release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.x'

- name: Get Version
id: version
run: |
RAW_VERSION=$(python -c "from socketsecurity import __version__; print(__version__)")
echo "VERSION=$RAW_VERSION" >> $GITHUB_ENV
if [ "v$RAW_VERSION" != "${{ github.ref_name }}" ]; then
echo "Error: Git tag (${{ github.ref_name }}) does not match package version (v$RAW_VERSION)"
exit 1
fi

- name: Check if version exists on PyPI
id: version_check
env:
VERSION: ${{ env.VERSION }}
run: |
if curl -s -f https://pypi.org/pypi/socketsecurity/$VERSION/json > /dev/null; then
echo "Version ${VERSION} already exists on PyPI"
echo "pypi_exists=true" >> $GITHUB_OUTPUT
else
echo "Version ${VERSION} not found on PyPI - proceeding with PyPI deployment"
echo "pypi_exists=false" >> $GITHUB_OUTPUT
fi

- name: Check Docker image existence
id: docker_check
env:
VERSION: ${{ env.VERSION }}
run: |
if curl -s -f "https://hub.docker.com/v2/repositories/socketdev/cli/tags/${{ env.VERSION }}" > /dev/null; then
echo "Docker image socketdev/cli:${VERSION} already exists"
echo "docker_exists=true" >> $GITHUB_OUTPUT
else
echo "docker_exists=false" >> $GITHUB_OUTPUT
fi

- name: Build package
if: steps.version_check.outputs.pypi_exists != 'true'
run: |
pip install build
python -m build

- name: Publish to PyPI
if: steps.version_check.outputs.pypi_exists != 'true'
uses: pypa/[email protected]
with:
password: ${{ secrets.PYPI_TOKEN }}

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Verify package is installable
id: verify_package
env:
VERSION: ${{ env.VERSION }}
run: |
for i in {1..30}; do
if pip install socketsecurity==${VERSION}; then
echo "Package ${VERSION} is now available and installable on PyPI"
pip uninstall -y socketsecurity
echo "success=true" >> $GITHUB_OUTPUT
exit 0
fi
echo "Attempt $i: Package not yet installable, waiting 20s... (${i}/30)"
sleep 20
done
echo "success=false" >> $GITHUB_OUTPUT
exit 1

- name: Build & Push Docker
if: |
steps.verify_package.outputs.success == 'true' &&
steps.docker_check.outputs.docker_exists != 'true'
uses: docker/build-push-action@v5
env:
VERSION: ${{ env.VERSION }}
with:
push: true
platforms: linux/amd64,linux/arm64
tags: |
socketdev/cli:latest
socketdev/cli:${{ env.VERSION }}
90 changes: 90 additions & 0 deletions .github/workflows/version-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
name: Version Check
on:
pull_request:
types: [opened, synchronize]
paths:
- 'socketsecurity/**'
- 'setup.py'
- 'pyproject.toml'

jobs:
check_version:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch all history for all branches

- name: Check version increment
id: version_check
run: |
# Get version from current PR
PR_VERSION=$(grep -o "__version__.*" socketsecurity/__init__.py | awk '{print $3}' | tr -d "'")
echo "PR_VERSION=$PR_VERSION" >> $GITHUB_ENV

# Get version from main branch
git checkout origin/main
MAIN_VERSION=$(grep -o "__version__.*" socketsecurity/__init__.py | awk '{print $3}' | tr -d "'")
echo "MAIN_VERSION=$MAIN_VERSION" >> $GITHUB_ENV

# Compare versions using Python
python3 -c "
from packaging import version
pr_ver = version.parse('${PR_VERSION}')
main_ver = version.parse('${MAIN_VERSION}')
if pr_ver <= main_ver:
print(f'❌ Version must be incremented! Main: {main_ver}, PR: {pr_ver}')
exit(1)
print(f'✅ Version properly incremented from {main_ver} to {pr_ver}')
"

- name: Manage PR Comment
uses: actions/github-script@v7
if: always()
env:
MAIN_VERSION: ${{ env.MAIN_VERSION }}
PR_VERSION: ${{ env.PR_VERSION }}
CHECK_RESULT: ${{ steps.version_check.outcome }}
with:
script: |
const success = process.env.CHECK_RESULT === 'success';
const prNumber = context.payload.pull_request.number;
const owner = context.repo.owner;
const repo = context.repo.repo;
const comments = await github.rest.issues.listComments({
owner: owner,
repo: repo,
issue_number: prNumber,
});

const versionComment = comments.data.find(comment =>
comment.user.type === 'Bot' &&
comment.body.includes('Version Check')
);

if (versionComment) {
if (success) {
// Delete the warning comment if check passes
await github.rest.issues.deleteComment({
owner: owner,
repo: repo,
comment_id: versionComment.id
});
} else {
// Update existing warning
await github.rest.issues.updateComment({
owner: owner,
repo: repo,
comment_id: versionComment.id,
body: `❌ **Version Check Failed**\n\nPlease increment...`
});
}
} else if (!success) {
// Create new warning comment only if check fails
await github.rest.issues.createComment({
owner: owner,
repo: repo,
issue_number: prNumber,
body: `❌ **Version Check Failed**\n\nPlease increment...`
});
}
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,5 @@ markdown_security_temp.md
*.pyc
test.py
*.cpython-312.pyc`
file_generator.py
file_generator.py
.env.local
Loading
Loading