Skip to content

action: support github commands for the tav modules #3246

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
May 10, 2023
5 changes: 0 additions & 5 deletions .ci/generate-github-tav-matrix.sh

This file was deleted.

103 changes: 103 additions & 0 deletions .github/workflows/tav-command.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
name: tav-command

on:
pull_request_review:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pull_request_review worked like a charm but pull_request_review_comment didn't get triggered. In any case pull_request_review is the one linked to a particular sha commit, hence github.event.pull_request.head.sha is the correct data to be used when checking out the source code.

types: [submitted]

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}

permissions:
contents: read

jobs:
command-validation:
if: startsWith(github.event.review.body, '/test tav')
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
pull-requests: write
outputs:
versions: ${{ steps.transform.outputs.versions }}
modules: ${{ steps.transform.outputs.modules }}
steps:
- name: Is comment allowed?
uses: actions/github-script@v6
with:
script: |
const actorPermission = (await github.rest.repos.getCollaboratorPermissionLevel({
...context.repo,
username: context.actor
})).data.permission
const isPermitted = ['write', 'admin'].includes(actorPermission)
if (!isPermitted) {
const errorMessage = 'Only users with write permission to the repository can run GitHub commands'
await github.rest.issues.createComment({
...context.repo,
issue_number: context.issue.number,
body: errorMessage,
})
core.setFailed(errorMessage)
return;
}

- uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}

- id: transform
name: Transform comment to the supported matrix
uses: actions/github-script@v6
with:
script: |
const fs = require('fs')

let modules, versions
try {
const matrix = JSON.parse(fs.readFileSync('./.ci/tav.json'))
versions = matrix.versions
modules = matrix.modules
} catch (err) {
core.setFailed(`Error loading './.ci/tav.json': ${err}`)
return
}
const comment = context.payload.review.body
if (comment !== '/test tav') {
const regex = /\/test tav ([^\s]+)(\s*)([^\s]*)/
const match = comment.match(regex)
if (!match) {
core.setFailed(`Incorrect comment, please use /test tav(\\s(module1,...,moduleN)?(\\s)?(node1,...,nodeN)?)?'`)
return
}
if (match[1]) {
if (match[1] !== 'all') {
modules = match[1].split(',')
}
}
if (match[3]) {
versions = match[3].split(',')
}
}
core.setOutput('modules', modules)
core.setOutput('versions', versions)

test-tav:
needs: command-validation
runs-on: ubuntu-latest
timeout-minutes: 40
strategy:
max-parallel: 15
fail-fast: false
matrix:
node: ${{ fromJSON(needs.command-validation.outputs.versions) }}
module: ${{ fromJSON(needs.command-validation.outputs.modules) }}
steps:

- uses: actions/checkout@v3
with:
ref: ${{ github.event.pull_request.head.sha }}

- run: .ci/scripts/test.sh -b "release" -t "${{ matrix.module }}" "${{ matrix.node }}"
env:
ELASTIC_APM_CONTEXT_MANAGER: ''
29 changes: 21 additions & 8 deletions .github/workflows/tav.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,21 +24,34 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
versions: ${{ steps.generate-matrix.outputs.versions }}
modules: ${{ steps.generate-matrix.outputs.modules }}
versions: ${{ steps.transform.outputs.versions }}
modules: ${{ steps.transform.outputs.modules }}
steps:
- uses: actions/checkout@v3
## TODO: use elastic/apm-pipeline-library/.github/actions/version-framework
## as soon as https://github.com/elastic/apm-pipeline-library/issues/2171 is done.
- id: generate-matrix
run: .ci/generate-github-tav-matrix.sh

- uses: actions/checkout@v3

- id: transform
name: Load matrix from tav.json
uses: actions/github-script@v6
with:
script: |
const fs = require('fs')
let matrix
try {
matrix = JSON.parse(fs.readFileSync('./.ci/tav.json'))
} catch (err) {
core.setFailed(`Error loading './.ci/tav.json': ${err}`)
return
}
core.setOutput('modules', matrix.modules)
core.setOutput('versions', matrix.versions)

test-tav:
needs: prepare-matrix
runs-on: ubuntu-latest
timeout-minutes: 40
strategy:
max-parallel: 30
max-parallel: 15
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need to run in parallel more since it's a merge build, so it's not happening on PRs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what you mean here. Our test matrix is approximately 250 jobs now. If we run only 15 in parallel it may take twice as long to complete them all. Currently (see some examples at https://github.com/elastic/apm-agent-nodejs/actions/workflows/tav.yml) a TAV run is taking 1.5h to complete. It would be unfortunate if it takes 3h to complete.

Is there a reason to want lower max-parallel? Is there added cost?

fail-fast: false
matrix:
# A job matrix limit is 256. We do some grouping of TAV modules to
Expand Down