TRT-1854: translate test annotations to use: include/exclude CEL, labels, filtering

[smg247]

openshift-tests-extension introduces a new way to skip upstream tests based on environment flags and include/exclude fields present on the listed tests. This PR translates the existing annotations to that functionality, but leaves them in place. They will be removed in a later PR.

[openshift-ci-robot]

@smg247: This pull request references TRT-1854 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

In response to this:

openshift-tests-extension introduces a new way to skip upstream tests based on environment flags and include/exclude fields present on the listed tests. This PR translates the existing test skips to that functionality.

There will be a followup to this to remove the other annotations, and eventually the framework itself.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@smg247: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 4abe9c8|UPSTREAM: : remove "SKIPPED" annotations, and replace with include/exclude CEL to make use of environment flags: does not specify an upstream backport in the commit message
• e26bc4a|UPSTREAM: : updated generated annotations from removing skips: does not specify an upstream backport in the commit message
• efdd8cd|UPSTREAM: : ./hack/update-vendor.sh for openshift-tests-extension: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[smg247]

/hold to merge after #2214

[openshift-ci-robot]

@smg247: This pull request references TRT-1854 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

In response to this:

openshift-tests-extension introduces a new way to skip upstream tests based on environment flags and include/exclude fields present on the listed tests. This PR translates the existing test skips to that functionality.

I have generated a diff of the output of k8s-tests-ext list prior to, and after this change for verification purposes.

There will be a followup to this to remove the other annotations, and eventually the framework itself.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@smg247: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 1daa2c4|UPSTREAM: : updated generated annotations from removing skips: does not specify an upstream backport in the commit message
• 74647d8|UPSTREAM: : ./hack/update-vendor.sh for openshift-tests-extension: does not specify an upstream backport in the commit message
• db6b8e5|UPSTREAM: : remove "SKIPPED" annotations, and replace with include/exclude CEL to make use of environment flags: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[openshift-ci-robot]

@smg247: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 1daa2c4|UPSTREAM: : updated generated annotations from removing skips: does not specify an upstream backport in the commit message
• 74647d8|UPSTREAM: : ./hack/update-vendor.sh for openshift-tests-extension: does not specify an upstream backport in the commit message
• 77fa839|UPSTREAM: : fix vendoring error: does not specify an upstream backport in the commit message
• db6b8e5|UPSTREAM: : remove "SKIPPED" annotations, and replace with include/exclude CEL to make use of environment flags: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[smg247]

/hold cancel

[smg247]

/hold to figure out test mapping with the name changes prior to merge

[openshift-ci-robot]

@smg247: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 0e529d9|UPSTREAM: : add environemntSelector include/exclude CEL to make use of environment flags for SKIPPED tests: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[openshift-ci-robot]

@smg247: This pull request references TRT-1854 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.19.0" version, but no target version was set.

In response to this:

openshift-tests-extension introduces a new way to skip upstream tests based on environment flags and include/exclude fields present on the listed tests. This PR translates the existing test skips to that functionality, but leaves the annotations alone. They will be removed in a later PR.

I have generated a diff of the output of k8s-tests-ext list prior to, and after this change for verification purposes. This diff was generated when we were removing the annotations from the test names, that part will no longer be happening with this PR.

There will be a followup to this to translate the other annotations, and eventually the framework itself.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[smg247]

/hold cancel

[openshift-ci-robot]

@smg247: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 54463b0|UPSTREAM: : add environemntSelector include/exclude CEL to make use of environment flags for SKIPPED tests: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[stbenjam]

/lgtm

[openshift-ci-robot]

@smg247: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• 0d48d87|UPSTREAM: : add environemntSelector include/exclude CEL to make use of environment flags for SKIPPED tests: does not specify an upstream backport in the commit message
• 1458e23|UPSTREAM: : refactor environment selectors into their own file and make additional helpers to simplify the logic: does not specify an upstream backport in the commit message
• 34ca7d9|UPSTREAM: : filter out disabled tests and label those that are annotated as: Slow, Flaky, or Serial respectively: does not specify an upstream backport in the commit message
• 4fa2f14|UPSTREAM: : update to translate rules that were changed since this PR has been opened and remove unused disable: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[bertinatto]

/approve

[bertinatto]

Things that I'd like to see in follow-ups:

1. Some mechanism to detect a rule that doesn't match any test
2. Some mechanism to continue running k8s-* jobs in this repository once the annotations are gone

[openshift-ci-robot]

@smg247: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

• d0c1b6d|UPSTREAM: : translate all existing rules to use filtering and the environemntSelector in the specs: does not specify an upstream backport in the commit message

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

[bertinatto]

/remove-label backports/unvalidated-commits
/label backports/validated-commits

[stbenjam]

/lgtm

[stbenjam]

/test verify

[openshift-ci]

@stbenjam: No presubmit jobs available for openshift/kubernetes@master

In response to this:

/test verify

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bertinatto, smg247, stbenjam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• DOWNSTREAM_OWNERS [bertinatto]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[smg247]

/retest-required

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD f4a27c4 and 2 for PR HEAD d0c1b6d in total

[smg247]

/retest-required

[smg247]

/test verify

[smg247]

/retest-required

[smg247]

/retest-required

[openshift-ci]

@smg247: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-csi	d0c1b6d	link	false	/test e2e-aws-csi
ci/prow/okd-scos-e2e-aws-ovn	d0c1b6d	link	false	/test okd-scos-e2e-aws-ovn

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-bot]

[ART PR BUILD NOTIFIER]

Distgit: openshift-enterprise-pod
This PR has been included in build openshift-enterprise-pod-container-v4.19.0-202503180241.p0.g2ed17b4.assembly.stream.el9.
All builds following this will include this PR.

[openshift-bot]

[ART PR BUILD NOTIFIER]

Distgit: openshift-enterprise-hyperkube
This PR has been included in build openshift-enterprise-hyperkube-container-v4.19.0-202503180241.p0.g2ed17b4.assembly.stream.el9.
All builds following this will include this PR.

[openshift-bot]

[ART PR BUILD NOTIFIER]

Distgit: kube-proxy
This PR has been included in build kube-proxy-container-v4.19.0-202503180241.p0.g2ed17b4.assembly.stream.el9.
All builds following this will include this PR.

[openshift-bot]

[ART PR BUILD NOTIFIER]

Distgit: ose-installer-kube-apiserver-artifacts
This PR has been included in build ose-installer-kube-apiserver-artifacts-container-v4.19.0-202503180241.p0.g2ed17b4.assembly.stream.el9.
All builds following this will include this PR.