test automatically prevent future merges when a backport fails

[Mikaayenson]

Issues

Resolves #1882
Related to elastic/ia-trade-team/issues/26

Summary

• Automatically prevent future merges when a backport fails
• Moves unit testing earlier in the workflow
• Sample test: https://github.com/elastic/detection-rules/actions/runs/2221981329

BLUF Recommendation / Solution

• Get a list from a file that can be used in multiple workflows
• Break out the test to a seperate workflow: Branch Version Status Checks
• Check the branch versions in parallel against the Unit Test workflow to make sure they're in a passing state.
• Update other workflows to pull from the target-branches.yml which is generated via get_stack_versions during the CI workflow.

Details

GH Status API

• We could get the checks per branch described in Statue API, but the statuses field will need to be populated (e.g. by some external service).
• When you look at https://api.github.com/repos/elastic/detection-rules/commits/7.13/status, or https://api.github.com/repos/elastic/detection-rules/commits/7.13/statuses you'll see the statuses is an empty list.
• On the other hand, the statuses information is readily available for PRs (e.g. with commit sha https://api.github.com/repos/elastic/detection-rules/commits/0643a0596b0c1f041ac2a30afccee8063a0e5b01/statuses or via branch https://api.github.com/repos/elastic/detection-rules/commits/issue-1934/statuses)

GH Actions API

• Other than this Status API, we have the Actions API which will provide information on workflows.
• Currently the backport workflow only runs againt the main branch so other branches with the backport workflow will not provide status results.
• We can leverage the pythonpackage.yml / Unit Tests workflow to provide a CI status for the branch, but I'm not sure if it will help address the original issue described here. (e.g. https://api.github.com/repos/elastic/detection-rules/actions/workflows/pythonpackage.yml/runs?per_page=1&branch=7.16)

Testing

• Getting the status of all supported branches would be nice if we can use the matrix defined in the backport.yml workflow.
• It doesn't appear that we can share the matrix across workflows via environment variable as-is since it's a list. Also we probably don't want the unit test to rely on environment variables from backport.yml
• As highlighted by Justin, we do not want to have to maintain multiple lists so we can either store and load a list from a file, or store the list in a workflow and share the list as an artifact between workflows.
• Loading from a file would require a job that outputs a matrix from the file as a dependency to the rest of the workflow. This way the workflow strategy runs correctly against the version branches.

[Mikaayenson]

Currently configured to test manually but will add it to trigger on pull request later.

[terrancedejesus]

LGTM

[Mikaayenson]

This is currently was failing the test because cavokz manually skipped the last backport workflow.
The fail is a good thing. We just need to delete the workflow and rerun the unit test.

[brokensound77]

https://github.com/elastic/detection-rules/pull/1909/files#r855654245

[Mikaayenson]

https://github.com/elastic/detection-rules/pull/1909/files#r855654245

Updated the description with the proposed solution.

[brokensound77]

Once this has been E2E tested with all of the recent changes, then I think it is good to go. Thanks! 🎉