OPRUN-2913: Sync api, operator-registry, operator-lifecycle-manager downstream #510

tmshort · 2023-07-17T13:42:09Z

This is a big merge of the upstream repositories into the downstream mono-repo.

Due to different levels of activity in each repository, they were at different level of the k8s API, and this had to be resolved upstream before downstreaming. This involved updated quite a number of components in go.mod/go.sum.

In order to do this sync as cleanly as possible, because various bug fixes were downstreamed independently, I updated the sync scripts in the scripts directory. The first change was to interleave (based on the commit timestamp) the syncing of individual commits from the source repositories, in the future hope that it will make large merges like this easier. Doing each source repo one-at-a-time can cause go.mod components to be upgraded by one repo, and then regressed by another repo, before being upgraded again. Interleaving the commits from the repositories based on date helps alleviate this scenario.

The second change was to note if there were any changes to the go.mod files, and if so, give the merger (e.g. me in this case) the opportunity (by pausing) to evaluate if there were any regressions in the go.mod file. Then the merger can open a shell in another window to evaluate the go.mod file to see if it needs to be modified and/or reverted.

The third change was to handle failures of make manifests in a similar way. If make manifests fails, the update scripts will pause to allow the merge to evaluate the results. This was especially useful as make manifests was failing due to the inconsistent versioning of the k8s API in each of the source repositories.

In this merge, the k8s APIs are now at v0.27.2.

openshift-ci-robot · 2023-07-17T13:42:14Z

@tmshort: This pull request references OPRUN-2913 which is a valid jira issue.

In response to this:

This is a big merge of the upstream repositories into the downstream mono-repo.

Due to different levels of activity in each repository, they were at different level of the k8s API, and this had to be resolved upstream before downstreaming. This involved updated quite a number of components in go.mod/go.sum.

In order to do this sync as cleanly as possible, because various bug fixes were downstreamed independently, I updated the sync scripts in the scripts directory. The first change was to interleave (based on the commit timestamp) the syncing of individual commits from the source repositories, in the future hope that it will make large merges like this easier. Doing each source repo one-at-a-time can cause go.mod components to be upgraded by one repo, and then regressed by another repo, before being upgraded again. Interleaving the commits from the repositories based on date helps alleviate this scenario.

The second change was to note if there were any changes to the go.mod files, and if so, give the merger (e.g. me in this case) the opportunity (by pausing) to evaluate if there were any regressions in the go.mod file. Then the merger can open a shell in another window to evaluate the go.mod file to see if it needs to be modified and/or reverted.

The third change was to handle failures of make manifests in a similar way. If make manifests fails, the update scripts will pause to allow the merge to evaluate the results. This was especially useful as make manifests was failing due to the inconsistent versioning of the k8s API in each of the source repositories.

In this merge, the k8s APIs are now at v0.27.2.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci · 2023-07-17T13:42:29Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

tmshort · 2023-07-17T13:42:40Z

/test all

openshift-ci · 2023-07-17T14:12:31Z

@tmshort: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

/test e2e-gcp-console-olm
/test e2e-gcp-olm
/test e2e-gcp-ovn
/test e2e-upgrade
/test images
/test unit-api
/test unit-olm
/test unit-psm
/test unit-registry
/test verify

The following commands are available to trigger optional jobs:

/test e2e-gcp-olm-flaky

Use /test all to run all jobs.

In response to this:

/retest all

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

tmshort · 2023-07-17T20:18:32Z

/retest

perdasilva · 2023-07-18T12:38:26Z

/retest

perdasilva · 2023-07-18T12:38:37Z

/retest

tmshort · 2023-07-18T18:06:21Z

It appears ci/prow/e2e-gcp-ovn/ci/prow/e2e-gcp-olm are failing because oc get packagemanifest (singular) is failing to retrieve any resources.
Creating a cluster with this PR, oc get packagemanifests (plural) works, but oc get packagemanifest (singular) fails. I will need to test a cluster w/o this PR to see which one works.
One of the upstream commits required the addition of GetSingularName() for packagemanifests, which you think would not be a problem (and would certainly not cause this problem).

tmshort · 2023-07-18T18:16:17Z

With this PR

tshort@cube:~/OPRUN-2913$ oc get packagemanifests -A | wc
    432    2279   44063
tshort@cube:~/OPRUN-2913$ oc get packagemanifest -A | wc
error: the server doesn't have a resource type "packagemanifest"
      0       0       0
tshort@cube:~/OPRUN-2913$

tmshort · 2023-07-18T18:58:27Z

Vanilla 4.14:

tshort@cube:~/OPRUN-2913$ oc get packagemanifests | wc
    432    1847   33264
tshort@cube:~/OPRUN-2913$ oc get packagemanifest | wc
    432    1847   33264
tshort@cube:~/OPRUN-2913$

tmshort · 2023-07-18T19:08:38Z

It's a change in the upstream behavior; verified on a vanilla 4.14 cluster, and on operator-framework/operator-lifecycle-manager@11b66ef (tip of master) vs operator-framework/operator-lifecycle-manager@4564b26 (before k8s updates)

tmshort · 2023-07-19T18:57:19Z

/retest

tmshort · 2023-07-19T22:57:17Z

/retest

ankitathomas

/lgtm

tmshort · 2023-07-20T13:19:58Z

The CI complains about the OWNERS files are all about those in the staging directories. Do we care? Should I remove those individuals, or even the OWNERS files themselves?

openshift-ci · 2023-07-20T23:22:12Z

The following users are mentioned in OWNERS file(s) but are untrusted for the following reasons. One way to make the user trusted is to add them as members of the openshift org. You can then trigger verification by writing /verify-owners in a comment.

akihikokuroda
- User is not a member of the org. User is not a collaborator. Satisfy at least one of these conditions to make the user trusted.
asmacdo
- User is not a member of the org. User is not a collaborator. Satisfy at least one of these conditions to make the user trusted.
exdx
- User is not a member of the org. User is not a collaborator. Satisfy at least one of these conditions to make the user trusted.
njhale
- User is not a member of the org. User is not a collaborator. Satisfy at least one of these conditions to make the user trusted.
- staging/operator-registry/OWNERS

tmshort · 2023-07-21T03:03:53Z

/retest

87utierrez · 2023-07-21T21:30:20Z

@tmshort Hi, my name is Hector, I was hoping that you could help and share with me as I'm new with go and development. with run a build to create an image from this repo.

I've cloned this repo and made some changes to the go. mod and run the following command

go mod tidy
go mod verify
go mod vendor
go test ./...

It came back with the program, not an importable package.

what I'm trying to accomplish is to use this registry - within my docker file
FROM registry.redhat.io/openshift4/ose-operator-registry:v4.13.0-202306072029.p0.gce46f5b.assembly.stream
I'm pulling the /bin/opm and /bin/grpc_health_probe into my image. The reason why I made changes to the go mod list is to update the packages that have fixes from the previous version.

The ask is after making those changes and updating the go.sum , how do I rebuild the manifest to have an image made?
I could really use your help --- desperate.

Update pakcage-server-manager to support latest APIs/golang Signed-off-by: Todd Short <[email protected]>

Otherwise, `oc get packagemanifest` (singular) breaks. `oc get packagemanifests` (plural) works. Signed-off-by: Todd Short <[email protected]> Upstream-repository: operator-lifecycle-manager Upstream-commit: b30a3be15c6a1df17292f1b873c2dd85a1fb41c2

tmshort · 2023-07-24T14:23:18Z

@tmshort Hi, my name is Hector, I was hoping that you could help and share with me as I'm new with go and development. with run a build to create an image from this repo.

Hi @87utierrez, this is not an appropriate forum for this type of discussion. I suggest joining the #olm-dev channel in the Kubernetes Slack instance: https://kubernetes.slack.com/archives/C0181L6JYQ2 and asking your questions there.

ankitathomas

/lgtm

openshift-ci · 2023-07-24T14:33:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ankitathomas, tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~DOWNSTREAM_OWNERS~~ [ankitathomas]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tmshort · 2023-07-24T17:41:54Z

/retest

tmshort · 2023-07-24T18:36:32Z

/retest

87utierrez · 2023-07-24T18:41:33Z

I don’t have an account on there ? Hector Gutierrez On Jul 24, 2023, at 9:23 AM, Todd Short ***@***.***> wrote: @tmshort Hi, my name is Hector, I was hoping that you could help and share with me as I'm new with go and development. with run a build to create an image from this repo. Hi @87utierrez, this is not an appropriate forum for this type of discussion. I suggest joining the #olm-dev channel in the Kubernetes Slack instance: https://kubernetes.slack.com/archives/C0181L6JYQ2 —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

tmshort · 2023-07-24T21:02:00Z

/retest

openshift-ci-robot · 2023-07-24T22:45:36Z

/retest-required

Remaining retests: 0 against base HEAD b4ce064 and 2 for PR HEAD 045a139 in total

tmshort · 2023-07-25T01:40:22Z

/retest

m1kola · 2023-07-25T08:38:19Z

/retest

m1kola · 2023-07-25T08:40:17Z

@87utierrez you can create a slack account and joint the community using this link - https://slack.k8s.io/

tmshort · 2023-07-25T12:49:38Z

/retest

tmshort · 2023-07-25T17:06:56Z

/retest

tmshort · 2023-07-25T21:34:51Z

/retest

tmshort · 2023-07-26T02:00:50Z

/retest

tmshort · 2023-07-26T12:31:48Z

/retest

tmshort · 2023-07-26T16:15:07Z

/retest

tmshort · 2023-07-26T19:42:14Z

/retest

openshift-ci · 2023-07-26T22:27:44Z

@tmshort: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jul 17, 2023

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 17, 2023

openshift-ci bot added the do-not-merge/invalid-owners-file Indicates that a PR should not merge because it has an invalid OWNERS file in it. label Jul 17, 2023

This was referenced Jul 17, 2023

Registry bulk sync #506

Closed

OPRUN-2913: Sync 2023 07 11 #507

Closed

tmshort marked this pull request as ready for review July 17, 2023 21:05

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 17, 2023

openshift-ci bot requested review from ankitathomas and benluddy July 17, 2023 21:06

tmshort force-pushed the sync-2023-07-17 branch 2 times, most recently from 648b33b to 0ea0616 Compare July 19, 2023 13:51

ankitathomas approved these changes Jul 20, 2023

View reviewed changes

openshift-ci bot assigned ankitathomas Jul 20, 2023

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 20, 2023

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 20, 2023

tmshort mentioned this pull request Jul 20, 2023

Bump go-restful/v3 to v3.10.2 to address PRISMA-2022-0227 #521

Closed

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Jul 20, 2023

tmshort and others added 2 commits July 24, 2023 10:17

OPRUN-2913: Update package-server-manager

ec11e29

Update pakcage-server-manager to support latest APIs/golang Signed-off-by: Todd Short <[email protected]>

tmshort force-pushed the sync-2023-07-17 branch from 77c458e to 045a139 Compare July 24, 2023 14:17

openshift-ci bot removed the do-not-merge/invalid-owners-file Indicates that a PR should not merge because it has an invalid OWNERS file in it. label Jul 24, 2023

ankitathomas approved these changes Jul 24, 2023

View reviewed changes

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 24, 2023

openshift-merge-robot merged commit 9a977fa into openshift:master Jul 26, 2023

tmshort deleted the sync-2023-07-17 branch July 27, 2023 17:15

OPRUN-2913: Sync api, operator-registry, operator-lifecycle-manager downstream #510

OPRUN-2913: Sync api, operator-registry, operator-lifecycle-manager downstream #510

Conversation

tmshort commented Jul 17, 2023

openshift-ci-robot commented Jul 17, 2023 • edited by openshift-ci bot Loading

openshift-ci bot commented Jul 17, 2023

tmshort commented Jul 17, 2023

openshift-ci bot commented Jul 17, 2023

tmshort commented Jul 17, 2023

perdasilva commented Jul 18, 2023

perdasilva commented Jul 18, 2023

tmshort commented Jul 18, 2023

tmshort commented Jul 18, 2023 • edited Loading

tmshort commented Jul 18, 2023 • edited Loading

tmshort commented Jul 18, 2023

tmshort commented Jul 19, 2023

tmshort commented Jul 19, 2023

ankitathomas left a comment

Choose a reason for hiding this comment

tmshort commented Jul 20, 2023

openshift-ci bot commented Jul 20, 2023

tmshort commented Jul 21, 2023

87utierrez commented Jul 21, 2023

tmshort commented Jul 24, 2023 • edited Loading

ankitathomas left a comment

Choose a reason for hiding this comment

openshift-ci bot commented Jul 24, 2023

tmshort commented Jul 24, 2023

tmshort commented Jul 24, 2023

87utierrez commented Jul 24, 2023 via email

tmshort commented Jul 24, 2023

openshift-ci-robot commented Jul 24, 2023

tmshort commented Jul 25, 2023

m1kola commented Jul 25, 2023

m1kola commented Jul 25, 2023

tmshort commented Jul 25, 2023

tmshort commented Jul 25, 2023

tmshort commented Jul 25, 2023

tmshort commented Jul 26, 2023

tmshort commented Jul 26, 2023

tmshort commented Jul 26, 2023

tmshort commented Jul 26, 2023

openshift-ci bot commented Jul 26, 2023

openshift-ci-robot commented Jul 17, 2023 •

edited by openshift-ci bot

Loading

tmshort commented Jul 18, 2023 •

edited

Loading

tmshort commented Jul 18, 2023 •

edited

Loading

tmshort commented Jul 24, 2023 •

edited

Loading