🌱 Test Registry

[dtfranz]

Test Registry

Adds a bare-bones test registry and a folder tree for our test images.

This PR aims to improve our test infrastructure by removing scripts and external images for registry operations in favor of bare-bones tools built in golang using well-known libraries (go-containerregistry in particular).

Structure

The PR adds two tools:registry and push. It also aggregates all our image-related makefile targets into one: image-registry.

registry

Just about the simplest possible image registry that can be built using the go-containerregistry lib. See README here.

push

Utilizes crane within go-containerregistry to build and push our catalog and bundle images. It uses a directory walker which is intended to make adding future test bundles and catalogs easily. It also (IMO - totally subjective) makes our test image collection more WYSIWYG by clearly showing all the images and tags we use in the folder rather than hiding them in scripts or makefiles.

See README here for more info on the expected folder structure.

make image-registry

Builds the registry and push binaries, bundles them into one image, then deploys them both. registry is deployed essentially as before. push executes as a job, building all the images bundled inside of it then pushing them to the specified registry address.

TODOs / Follow-Ups

PR size is currently huge due to the amount of prometheus manifests being copied for each bundle tag... I'm open to suggestions on how we could address this. Maybe we could remove the prometheus-operator in favor of a test operator with fewer manifests? Addressed this, see below.

Remaining items to address:

• Either: Done - going with option C for now.
• A: Find a better way to organize tags such that we need not have many copies of the same manifests.
• B: Keep the current folder structure for tags but simplify the manifests we're using in test to reduce file count/complexity.
• C: Modify the catalog yaml so that every version of prometheus-operator uses the same bundle image (v1.0.0). We were previously doing this anyway essentially because each tag we created for this bundle had the exact same content.
• Metadata reading for bundle images is using one path instead of reading dynamically (which we can get away with right now because all the bundles are identical). Done.
• Configurable cert/key path for registry (not super critical right now). Done.
• Use mounted secrets for push instead of crane.Insecure (also not critical). Done.

Description

Reviewer Checklist

• API Go Documentation
• Tests: Unit Tests (and E2E Tests, if appropriate)
• Comprehensive Commit Messages
• Links to related GitHub Issue(s)

[netlify]

✅ Deploy Preview for olmv1 ready!

Name	Link
🔨 Latest commit	2f2d220
🔍 Latest deploy log	https://app.netlify.com/sites/olmv1/deploys/672bb2bd23a9f300081148f7
😎 Deploy Preview	https://deploy-preview-1425--olmv1.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.45%. Comparing base (482a66d) to head (2f2d220).
Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1425      +/-   ##
==========================================
+ Coverage   73.42%   73.45%   +0.03%     
==========================================
  Files          42       42              
  Lines        3063     3063              
==========================================
+ Hits         2249     2250       +1     
+ Misses        641      640       -1     
  Partials      173      173              

Flag	Coverage Δ
e2e	55.20% <ø> (-0.14%)	⬇️
unit	52.59% <ø> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[tmshort]

verdiff is complaining about new go.mod files; we should be ok with that.

[tmshort]

I have a wicked simple bundle that can be installed: https://github.com/tmshort/test-image
It consists of just a configmap

[dtfranz]

I have a wicked simple bundle that can be installed: https://github.com/tmshort/test-image It consists of just a configmap

If you're suggesting this to replace the prometheus-operator, I'm happy with that as long as we're OK with not having things like: crd, deployment with runtime, etc. in our test bundle. But I think that having at least a running pod as a result of the ClusterExtension is important for the e2e to check for installation success.

I did also just test that we can definitely get away with just having the test-catalog's bundles all point to the same prometheus-operator version.

[tmshort]

It's a bundle, it can be installed... Not sure how critical it is to actually have something with a CRD or deployment. That being said, it's just an offer. Not making it a requirement.

[bentito]

Suggested change

	go build $(GO_BUILD_FLAGS) -tags '$(GO_BUILD_TAGS)' -ldflags '$(GO_BUILD_LDFLAGS)' -gcflags '$(GO_BUILD_GCFLAGS)' -asmflags '$(GO_BUILD_ASMFLAGS)' -o ./testdata/registry/bin/registry ./testdata/registry/registry.go
	GOOS=linux GOARCH=amd64 go build $(GO_BUILD_FLAGS) -tags '$(GO_BUILD_TAGS)' -ldflags '$(GO_BUILD_LDFLAGS)' -gcflags '$(GO_BUILD_GCFLAGS)' -asmflags '$(GO_BUILD_ASMFLAGS)' -o ./testdata/registry/bin/registry ./testdata/registry/registry.go

[tmshort]

Since the test-e2e target has a dependency on the image-registry target, the GOOS and GOARCH variables should be set from the test-e2e target, and don't need to be set explicitly here. This way, you can build image-registry locally, and get the correct (local) images when no running the e2e.

[camilamacedo86]

Hi @tmshort and @bentito,

Would it make more sense to use GOOS=$(go env GOOS) GOARCH=$(go env GOARCH) instead?

This approach would make the target compatible with both macOS and Linux environments, allowing us to run it locally as well. If we explicitly set GOARCH=amd64, for example, it won’t run on macOS systems with a different architecture, where the expected result would be darwin/arm64. Is targeting linux/amd64 specifically what we're aiming for here?

[bentito]

yep. current changes take this into account.

[tmshort]

Not sure that's the right approach. Doing the export should work when doing the image-target, and setting the variable equal to the go env ought to be a noop.

[camilamacedo86]

Hi @tmshort,

Not sure that's the right approach. Doing the export should work when doing the image-target, and setting the variable equal to the go env ought to be a noop.

I’m not sure if I fully follow your comment here. The export has fixed values as well:

image-registry: export GOOS=linux
image-registry: export GOARCH=amd64

Either way, this suggests fixed values for go build.

So, how would GOOS=linux GOARCH=amd64 go build build a binary for darwin/arm64 (my local env, for example)?

[tmshort]

Right, but doing this:

GOOS=$(go env GOOS)

Is basically setting the GOOS variable to the environment value that the go command already has available to it. It's equivalent to doing:

GOOS=${GOOS}

Which is a no-op.

[tmshort]

@camilamacedo86 I didn't answer your question.

So, how would GOOS=linux GOARCH=amd64 go build build a binary for darwin/arm64 (my local env, for example)?

It wouldn't. In this particular case, it's building executables for the docker container, which have to be linux/amd64. These two executables are never expected to run locally.

My point is that doing GOOS=$(go env GODS) is pointless.

[dtfranz]

This build is specifically for the image-registry image to be run in kind; comparing to how we build the operator-controller's manager binary, we use the same GOOS=linux GOARCH=amd64 ENV when building (make run uses docker-build which then uses build-linux). I'm just following that same convention.

If you feel it's important, what we could potentially do is split up the image-registry target so that GOOS=linux GOARCH=amd64 is only set when actually building the image. But I think this is unnecessary or at least non-critical enough that we could address it later in a follow-up.

[camilamacedo86]

These two executables are never expected to run locally.

If that is the case, (we never run it locally and would not make sense run it locally) then I am OK with fixed values 👍

[tmshort]

/lgtm

[tmshort]

/lgtm
After the squash, it's still the same.