From 483e3ca244a8614b66e660b63cb02ac0856ee5b3 Mon Sep 17 00:00:00 2001 From: Andrew Pantuso Date: Fri, 14 Oct 2022 17:05:32 -0400 Subject: [PATCH] Boilerplate: Update to a6c27570590858412f03ce4b84e7d6bb1ecfdc7a MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Conventions: - openshift/golang-osd-operator: Update --- https://github.com/openshift/boilerplate/compare/b0a20637c47d7d94d6bdadbb2660b4f081526015...a6c27570590858412f03ce4b84e7d6bb1ecfdc7a commit: 08bf780089af601a3554931e1342d81238286396 author: klin update ubi image tag commit: 93bb8c3b6f9636c582f4b1e642f259cf569283ab author: klin update ubi image commit: efe22eed1a95a5820f9011c979e8bc25933f2587 author: Supreeth Basabattini Add container-make targets commit: b8febb30962c92e9406143e24292249d38bc5064 author: Michael Shen Make env var optional for operator containers Signed-off-by: Michael Shen commit: 6f0a5c1385f7b48ff30f7ae49cfbddee775ab88a author: Ravi Trivedi Ignoring autogenerated zz files for codecov commit: b2b57ed9f0d2ebe75dfeea3bb13d360aba460d8a author: Ravi Trivedi Ignoring mock clients for codecov commit: 9520d29ded3d9dda08172165e6e15bc31a72ce4c author: Matt Bargenquast Set main package to base dir in new SDK commit: 350f8631ecf20a852b82a0f90b7bcfea8ff19845 author: Antony Natale OSD-12367: update to fix skopeo version dependency (#236) * update to fix skopeo version dependency * typo fix * typo fix commit: 4c70ca1b4f70da2a3a4606e37bf0d2caa23dc120 author: Antony Natale fix quoting commit: b6c8caca3763c7d3b85783b327f29afbaecaaffb author: Antony Natale added more error checking commit: f6c47f83a2fff43c027d22ff2c93e56496f8e27f author: Antony Natale added other commented fixes from last PR commit: b71e2da17b3d1eb344b57ce6dad8637ac71561ff author: Antony Natale fixes error check on opm command commit: 7db0538e630305a5ba047aa7ceef7bd78adb8f86 author: Antony Natale OSD-11742, OSD-12367 - bug and CVE fixes for catalog operators (#234) * base changes and logic set, needs polishing * adds printout of first run, undos changes made for local testing * update custom catalog makefile to match golang and better handle podman vs docker * fix typos and clean up * made suggested changes * remove unwanted testing values commit: 79cb8136e506524e740d78aff414e419415017ea author: Alex Vulaj Only remove for darwin/mac commit: 21b4ed75091749567ff9f53367e4303f850a0ef5 author: Alex Vulaj Remove :Z mount option for podman in container-make commit: fe734d5a42331418c0630cfe1e86221e8e995320 author: Eric Fried Fix golang-lint README It touted the wrong `make` target name. commit: 7c5112a0a8e3d187f56384efac222200b9b10244 author: Benjamin Dematteo Fixing the variable assignment commit: 1a05b3e6572eb37bd2098267287afc3dc35dead9 author: Supreeth Basabattini Automate migration to an extent commit: 8e0880fc695a2421cdd400249690a997ce50ff11 author: Antony Natale changes source of operator name as it is not compatible with jenkins pipeline commit: 9744d76fc22d4b838dadee9fb229f5e591e837ac author: John Roche Change to a template file commit: ec6e7a3e70bb23401df5f7e333569fdef2535c4e author: Supreeth Basabattini Remove support for CRDv1beta1 in boilerplate Removed op-generate-crd-fixup test case commit: 16924d60224e73f7d5b82c652fc9e28fb18794a5 author: Ravi Trivedi Adding initializer for debugging purpose commit: ffc15d3682ce1f84f37499fd65c472dc60623ecf author: Haoran Wang generate CRD v1 by default commit: 1e947d2b7daee575dbc0283c647b9945a8081c8f author: Haoran Wang 1. Set a home dir that are writable when do go test to workaround the (#224) issue we have in openshift ci 2. Set the test binary bin dir to a tmp dir commit: 8f3dfee52954fecbb9b834676f3bfc435619d9d7 author: Supreeth Basabattini Remove explicit configuration of envtest bin directory commit: 95509602ef9a3fe37a23f3b15d0f013918d2cebb author: Supreeth Basabattini Fix openapi-gen versioning for new o-sdk commit: 39cc9e896380f04987f92d5a5aeec331966e2d1e author: Supreeth Basabattini Include openapi-gen and setup-envtest in the backing image commit: 1d6d39398ec892deec3add7f9b0c3d616af1a60a author: Supreeth Basabattini Fix incorrect bash syntax commit: 1f0fa3dc4630b115897dcadf1cf2f25edfa3a731 author: Ravi Trivedi Rebuild registry image from 4.10.0 into a ubi-micro image commit: bac2488ea130d26848df3cba7ccc1ef98496b436 author: Haoran Wang Update boilerplate to support latest osdk use controller-gen v0.3.0 when it's using old osdk Address some comments use controller-gen in the baking image commit: 210292d58116f6c5981e1ab43b5d6543fb8a070d author: Haoran Wang install v0.8.0 controller-gen in the backing image commit: 9fa3022a5f8c00b7f5e1e10f6f935162c1741a21 author: John Roche make target for fips on osd operators commit: e35d0f5c1aa9ef8b22ede5e2bd7f674d2f07b3ba author: Michael Shen Initial prow-config addition Signed-off-by: Michael Shen commit: a078e1ce4ffd8607b13f398e7bcab5cc759fa864 author: Michael Shen Unexport GOFLAGS to fix bug when using container-make Signed-off-by: Michael Shen commit: b99a046991521f41600c897354009b93dee15e33 author: Benjamin Dematteo Fixing errors from golang-lint (in standard.mk) commit: 3560610f126217211ad9663f8e5729ffee2735c6 author: Benson Ngoy USER 1001 doesn't have pip install permissions + updating pip commit: bbf4703f01903064bf1d2f79a0e893f8535ce048 author: Wesley Hearn [OSD-10491] Bump urllib3 version (#209) * [OSD-10491] Bump urllib3 version * Update catalog-build.sh Remove the --upgrade from the pip3 install commit: c06911de4e086ff74e40cc31f018a71f72b9b408 author: Michael Shen Initial commit for osd-container-image convention Signed-off-by: Michael Shen commit: b924e51f50330cd4e7279acc102b7e1adc29f338 author: Benjamin Dematteo update README for new conventions commit: 7788244648e463fac115937799297d6eab204179 author: Benjamin Dematteo updating with PR Review comments commit: 1c13a6d23ef92603bbae32d5fb8f543789fc579c author: Christoph Blecker Update image tag in test files commit: 4d1f8a2678f70f7c64c0ae16800049e3feee64cc author: Christoph Blecker Update build_image script to work with RHEL8/go1.17 commit: 35820443749339ecb2f9d5d5ff7df0f43b159776 author: Christoph Blecker Switch to RHEL8 builder image commit: 03deceeda6b7b39b22e4eae06d59de78e5fdbbf0 author: Eric Fried Update README for image tagging quirks Clarify a couple of things in the README wrt image tagging: - You have to push the new tag to your `origin` as well as `upstream`. - You have to edit the tag in a couple of in-repo files as well. commit: b1c28646013edcecdcc7ab0631bd43c3183184b4 author: Eric Fried Fix broken link in README The release repo moved the file where we import our backing image. commit: 31bf3b7e35fc1f900aa7e1d8ae61e7c209182181 author: Ron Green feat(golangci): add gosec per ticket OSD-10161, this change should be running via CI on all osd operators. this way we are always compliant to the gosec (as we have done one audit a forever ago and cleaned issues this change should get approval before merging as this might cause initial work to upgrade to this version of boilerplate commit: 7d81a9d4ba6ed1b17ed0f0ceee85cb9def9884d4 author: Benjamin Dematteo Initial commit for golang-codecov and golang-lint conventions commit: 35c3064256d88136a6eaa75ced0660a7426d3c70 author: Eric Fried Remove support for hack/generate-operator-bundle.py Support for using a "local" (within the consuming repo) hack/generate-operator-bundle.py script was included as a bridge while consumers were cutting over to use the common bundle generator. That has happened. Get rid of the old script, make targets, and support. commit: 5f249984fba668d090b3ee2194dd0e98e506d43c author: Ron Green fix(CONTAINER_ENGINE): allow setting the env after #201 didn't work, I updated it a bit now you can set the env and it's not overriden commit: 304c86f779a2b8cb3daf8f02ae06fc0afe158bca author: Matt Bargenquast Support empty SaaS repositories commit: aa424cb414d16f5227ad15815487932f3ea7276c author: Ron Green fix(CONTAINER_ENGINE): allow setting the env this allows setting the image from an external source (additional checks might be required) commit: 2c52f542c9e5baf4bd77ed126e70c0c207829cdc author: Christoph Blecker Always re-pull the FROM images on container build commit: e2466bb12ad455410516761fdc90e9ea2381439d author: Matt Bargenquast App-SRE testing docs should suggest to rebase saas fork commit: 4cc462fca11eecd6e25b3ffca74a7b966470d623 author: Eric Fried golang-osd-operator appsre: Fix docker login When stealing (ahem, "reappropriating") podman/docker-accommodating App-SRE pipeline code when hive switched to a rhel8 jenkins node, we found a latent bug on the docker side: the `REGISTRY_AUTH_FILE` variable needs to point to an actual file, not just the directory in which the file is located. Contributing the fix back "upstream". commit: eefc1c43ec3a9717f6017fd747134568192e634d author: Candace Sheremeta Add OPERATOR_IMAGE as an env var for OSD operators commit: 1e7de2c16f088bbb18c7a83835870c3e6a35318b author: Dustin Row Add --pull to build command for operator-registry build for security fixes commit: 14bb7be131c97bb6271e78c4c211224f6d9a775d author: Dustin Row Revert "Update operator-registry base image to 4.9.0" commit: 3af04216b99e713d24a3ba527c283d9f45c3e7df author: Dustin Row Update operator-registry base image to 4.9.0 commit: 8f2bc55518f69bd5cd6cd9c73cba4b689fce5858 author: Karthik Perumal more loose ends being fixed commit: 4ab8860af2acc2eebc38ce96247134df4a432a83 author: Karthik Perumal Fix a few small issues with custom catalog convention commit: 253f82a45b2f5f57302e59ca6f355201ff9860bb author: Karthik Perumal Apply suggestions from code review Co-authored-by: Eric Fried <2uasimojo@users.noreply.github.com> commit: cc0ae4d07927030d527849d0834b84ba0fd51b2e author: Karthik Perumal fix make default target's grep Co-authored-by: Eric Fried <2uasimojo@users.noreply.github.com> commit: 9d78a55603eeb6f312e18ce1a8e23b41b5e7e6bf author: Karthik Perumal clean-up custom catalog convention as per review commit: b8892e15fd675c2f137a1b20df604c6a95a5b1d3 author: Karthik Perumal fix typo Co-authored-by: Eric Fried <2uasimojo@users.noreply.github.com> commit: 6c8a1d587dd975b0bced42560c85399cf3b1500d author: Karthik Perumal Add custom-catalog-osd-operator convention to boilerplate upstream [OSD-7284] commit: e26f6048109a5403a55f3008fee7c66e29e93c82 author: Benson Ngoy OSD-7604 - CRDs don't have a spec.version field in v1 commit: 5ad097e2fd2103bfe9bd007a68c5d3bc46e95fbe author: Eric Fried podman enablement - Podman and docker use different mechanisms to override the default path to the credentials cache. Accommodate both. - Podman and docker use different transport prefixes to reference locally built images via skopeo. Accommodate both. - Expand addition of `--userns keep-id` and `-v ...:Z` to more places. - Resolve OSD-6941 by detecting the container engine in catalog-build.sh. Co-Authored-By: @dofinn commit: 2ceeef142460be66ac28fb9f9fbcd42d64c89d9d author: Eric Fried Support CRD v1 (optional) By default, `make op-generate` will now generate CRD v1. Setting the `make` variable `CRD_VERSION=v1beta1` will override this behavior and build v1beta1 as before. OSD-5869 commit: fd46dd207919143b42db39ea73b20a0ae530aa5c author: Eric Fried Remove codecov secret mapping stuff This is no longer applicable since the move to self-service vault. commit: deb98557f3fe39ba318e9890e93aac5d6a72a739 author: Eric Fried Skip update/revert test case if at master If we're already at master (which happens e.g. during rehearsals on CI config updates), the 04-update-from-master-and-revert test is a) silly, and b) going to fail when attempting `boilerplate-commit` because there's nothing to update and therefore nothing to commit. Add logic to short out of this test case in this scenario. commit: 7ac2a583973b4799c685d701d784624bb815b729 author: Dominic Finn Update boilerplate/_lib/container-make commit: 42e4047700fbf04e5ee984a092562208ed33bc14 author: Dominic Finn Update boilerplate/_lib/container-make commit: 7dd43153c50332aa1cc9ba02db6231a3b026edcb author: Dominic Finn enable make generate locally commit: 2714e7c4a11d931ee8272b55c49445437ce2273a author: Eric Fried image-v2.1.0: add `gh` Add the `gh` (GitHub CLI) executable to the backing image so CICD jobs can do consumer reports. Part of OSD-5962 commit: 8bc4fa86071070d417e81170b1f6c37ea8c4756e author: Eric Fried Add `make subscriber-report` Create a `make` target that runs subscriber reports. The intent is to run this in a postsubmit prow job, the results of which will (eventually) be posted somewhere/somehow. Currently only runs `onboarding` and `release` subcommands, as the `pr` subcommand requires the `gh` CLI to be installed and authenticated. Part of OSD-5962 commit: 32bbc81aae1388f638cdecc29f0457344a369bef author: Eric Fried image-v2.0.1: Ratchet base to image-v2.0.0 To speed up boilerplate CI, ratchet prow's Dockerfile to build FROM `image-v2.0.0`. commit: ac77c2b43941a96dcd9b133bbc57772814c3a2da author: Eric Fried image-v2.0.0: Get rid of operator-sdk generate Remove invocations of and support for `operator-sdk generate`, replacing these with the corresponding `controller-gen` calls in the `op-generate` target of openshift/golang-osd-operator's standard.mk. The operator-sdk-generate.sh helper script is removed. We invoke controller-gen directly from the `make` target. We need to preserve and add to the post-CRD-gen `yq` hacks to produce CRDs compatible with both v3 and v4. These should be able to go away once 3.11 is dead. (At that time we'll also need to flag controller-gen to produce CRD v1 instead of v1beta1.) As written, this will work for consumers whether their APIs are packaged separately (as in e.g. https://github.com/openshift/aws-account-operator/pull/580) or not. With this commit, we produce a fresh backing image that omits the operator-sdk binaries, but is otherwise the same as image-v1.0.1. With this commit, we're rebuilding the backing image from scratch, so the prow and jenkins Dockerfiles are the same. A subsequent commit will ratchet the former to be based on image-v2.0.0 to speed up builds, in a spirit similar to #164. OSD-7352 OSD-7353 commit: 6b7309ab8cb31ca402961f832346fdafea06574b author: Eric Fried Document picking up fixes in the backing image commit: f156e7bf09eabdd45fccd1a898c92f53a8061ae5 author: Eric Fried image-v1.0.1: Ratchet base image to 1.0.0 To make presumbit CI faster, this commit ratchets up the CI image build to "start" from the previously-released image, `image-v1.0.0`. The original build.sh is renamed and a fresh build.sh is introduced. The latter is currently a no-op; subsequent image releases can add to it. An app-sre-specific Dockerfile combines all the build scripts to do a full build from scratch in the appsre pipeline. There is no functional change to the image itself. OSD-7253 commit: 26e72e939a3d0efd4492a94c890ed8148d659654 author: Sebastian Łaskawiec Unbound error fix for the new operators --- .ci-operator.yaml | 2 +- .codecov.yml | 4 + boilerplate/_data/backing-image-tag | 2 +- boilerplate/_data/last-boilerplate-commit | 2 +- boilerplate/_lib/container-make | 15 +- boilerplate/_lib/subscriber-propose | 1 - .../golang-osd-operator/.codecov.yml | 4 + .../openshift/golang-osd-operator/README.md | 61 +++-- .../app-sre-build-deploy.sh | 2 +- .../openshift/golang-osd-operator/app-sre.md | 11 + .../golang-osd-operator/build-opm-catalog.sh | 8 + .../codecov-secret-mapping | 62 ----- .../golang-osd-operator/configure-fips.sh | 18 ++ .../csv-generate/catalog-build.sh | 27 ++- .../csv-generate/catalog-publish.sh | 15 +- .../common-generate-operator-bundle.py | 42 +++- .../csv-generate/csv-generate.mk | 66 ++---- .../csv-generate/csv-generate.sh | 222 ++++++++---------- .../openshift/golang-osd-operator/ensure.sh | 53 ----- .../golang-osd-operator/fips.go.tmplt | 15 ++ .../golang-osd-operator/golangci.yml | 1 + .../operator-sdk-generate.sh | 47 ---- .../openshift/golang-osd-operator/standard.mk | 178 ++++++++++++-- .../openshift/golang-osd-operator/update | 5 +- build/Dockerfile | 2 +- 25 files changed, 455 insertions(+), 410 deletions(-) delete mode 100755 boilerplate/openshift/golang-osd-operator/codecov-secret-mapping create mode 100755 boilerplate/openshift/golang-osd-operator/configure-fips.sh create mode 100644 boilerplate/openshift/golang-osd-operator/fips.go.tmplt delete mode 100755 boilerplate/openshift/golang-osd-operator/operator-sdk-generate.sh diff --git a/.ci-operator.yaml b/.ci-operator.yaml index 5061986f..22d2bf05 100644 --- a/.ci-operator.yaml +++ b/.ci-operator.yaml @@ -1,4 +1,4 @@ build_root_image: name: boilerplate namespace: openshift - tag: image-v1.0.0 + tag: image-v2.3.2 diff --git a/.codecov.yml b/.codecov.yml index 844b447e..ba05647a 100644 --- a/.codecov.yml +++ b/.codecov.yml @@ -24,3 +24,7 @@ comment: layout: "reach,diff,flags,tree" behavior: default require_changes: no + +ignore: + - "**/mocks" + - "**/zz_generated*.go" diff --git a/boilerplate/_data/backing-image-tag b/boilerplate/_data/backing-image-tag index 69c23d74..bb65150a 100644 --- a/boilerplate/_data/backing-image-tag +++ b/boilerplate/_data/backing-image-tag @@ -1 +1 @@ -image-v1.0.0 +image-v2.3.2 diff --git a/boilerplate/_data/last-boilerplate-commit b/boilerplate/_data/last-boilerplate-commit index 06f06555..60a5d960 100644 --- a/boilerplate/_data/last-boilerplate-commit +++ b/boilerplate/_data/last-boilerplate-commit @@ -1 +1 @@ -b0a20637c47d7d94d6bdadbb2660b4f081526015 +a6c27570590858412f03ce4b84e7d6bb1ecfdc7a diff --git a/boilerplate/_lib/container-make b/boilerplate/_lib/container-make index 7ef0f1bf..ddcfedb5 100755 --- a/boilerplate/_lib/container-make +++ b/boilerplate/_lib/container-make @@ -9,11 +9,9 @@ fi source ${0%/*}/common.sh -CONTAINER_ENGINE=$(command -v podman || command -v docker) +CONTAINER_ENGINE="${CONTAINER_ENGINE:-$(command -v podman || command -v docker)}" [[ -n "$CONTAINER_ENGINE" ]] || err "Couldn't find a container engine. Are you already in a container?" -CONTAINER_ENGINE_SHORT=${CONTAINER_ENGINE##*/} - # Make sure the mount inside the container is named in such a way that # - openapi-gen (which relies on GOPATH) produces absolute paths; and # - other go-ish paths are writeable, e.g. for `go mod download`. @@ -21,11 +19,16 @@ CONTAINER_MOUNT=/go/src/$(repo_import $REPO_ROOT) # First set up a detached container with the repo mounted. banner "Starting the container" -if [[ $CONTAINER_ENGINE_SHORT == "podman" ]]; then - container_id=$($CONTAINER_ENGINE run --userns keep-id -d -v "$REPO_ROOT":"$CONTAINER_MOUNT":Z $IMAGE_PULL_PATH tail -f /dev/null) +if [[ "${CONTAINER_ENGINE##*/}" == "podman" ]]; then + if [[ $OSTYPE == *"darwin"* ]]; then + CE_OPTS="--userns keep-id -v $REPO_ROOT:$CONTAINER_MOUNT" + else + CE_OPTS="--userns keep-id -v $REPO_ROOT:$CONTAINER_MOUNT:Z" + fi else - container_id=$($CONTAINER_ENGINE run -d -v "$REPO_ROOT":"$CONTAINER_MOUNT" $IMAGE_PULL_PATH tail -f /dev/null) + CE_OPTS="-v $REPO_ROOT:$CONTAINER_MOUNT" fi +container_id=$($CONTAINER_ENGINE run -d ${CE_OPTS} $IMAGE_PULL_PATH sleep infinity) if [[ $? -ne 0 ]] || [[ -z "$container_id" ]]; then err "Couldn't start detached container" diff --git a/boilerplate/_lib/subscriber-propose b/boilerplate/_lib/subscriber-propose index cc57b961..8fb05768 100755 --- a/boilerplate/_lib/subscriber-propose +++ b/boilerplate/_lib/subscriber-propose @@ -7,7 +7,6 @@ declare -A SUBCOMMANDS SUBCOMMANDS=( # TODO: # [bootstrap]='Bootstrap a new subscriber' - # [codecov-secret-mapping]='Propose codecov secret mapping to openshift/release' # [prow-config]='Propose standardized prow configuration to openshift/release' [update]='Update an already-onboarded subscriber' ) diff --git a/boilerplate/openshift/golang-osd-operator/.codecov.yml b/boilerplate/openshift/golang-osd-operator/.codecov.yml index 844b447e..ba05647a 100644 --- a/boilerplate/openshift/golang-osd-operator/.codecov.yml +++ b/boilerplate/openshift/golang-osd-operator/.codecov.yml @@ -24,3 +24,7 @@ comment: layout: "reach,diff,flags,tree" behavior: default require_changes: no + +ignore: + - "**/mocks" + - "**/zz_generated*.go" diff --git a/boilerplate/openshift/golang-osd-operator/README.md b/boilerplate/openshift/golang-osd-operator/README.md index 4f6d5777..edee6acb 100644 --- a/boilerplate/openshift/golang-osd-operator/README.md +++ b/boilerplate/openshift/golang-osd-operator/README.md @@ -14,6 +14,7 @@ This convention is suitable for both cluster- and hive-deployed operators. The following components are included: ## `make` targets and functions. + **Note:** Your repository's main `Makefile` needs to be edited to include the "nexus makefile include": @@ -28,7 +29,7 @@ following: ### Prow | Test name / `make` target | Purpose | -|---------------------------|-----------------------------------------------------------------------------------------------------------------| +| ------------------------- | --------------------------------------------------------------------------------------------------------------- | | `validate` | Ensure code generation has not been forgotten; and ensure generated and boilerplate code has not been modified. | | `lint` | Perform static analysis. | | `test` | "Local" unit and functional testing. | @@ -48,18 +49,26 @@ $ make RELEASE_CLONE=/home/me/github/openshift/release prow-config ``` This will generate a delta configuring prow to: + - Build your `build/Dockerfile`. - Run the above targets in presubmit tests. - Run the `coverage` target in a postsubmit. This is the step that updates your coverage report in codecov.io. #### Local Testing + You can run these `make` targets locally during development to test your code changes. However, differences in platforms and environments may lead to unpredictable results. Therefore boilerplate provides a utility to run targets in a container environment that is designed to be as similar as possible to CI: +```shell +$ make container-{target} +``` + +or + ```shell $ ./boilerplate/_lib/container-make {target} ``` @@ -72,27 +81,15 @@ By default it is configured to be run from the app-sre jenkins pipelines. Consult [this doc](app-sre.md) for information on local execution/testing. ## Code coverage + - A `codecov.sh` script, referenced by the `coverage` `make` target, to -run code coverage analysis per [this SOP](https://github.com/openshift/ops-sop/blob/93d100347746ce04ad552591136818f82043c648/services/codecov.md). + run code coverage analysis per [this SOP](https://github.com/openshift/ops-sop/blob/93d100347746ce04ad552591136818f82043c648/services/codecov.md). - A `.codecov.yml` configuration file for [codecov.io](https://docs.codecov.io/docs/codecov-yaml). Note that this is copied into the repository root, because that's [where codecov.io expects it](https://docs.codecov.io/docs/codecov-yaml#can-i-name-the-file-codecovyml). -- A `make` target to [request the secret mapping in openshift/release](https://github.com/openshift/ops-sop/blob/be43125239deb1f2bbc1ef54f010410e97ff6146/services/codecov.md#openshiftrelease-pr-1---secret-mapping): - -```shell -$ make codecov-secret-mapping -``` - -If you already have the openshift/release repository cloned locally, you -may specify its path via `$RELEASE_CLONE`: - -```shell -$ make RELEASE_CLONE=/home/me/github/openshift/release codecov-secret-mapping -``` - ## Linting and other static analysis with `golangci-lint` - A `go-check` `make` target, which @@ -107,13 +104,35 @@ The convention embeds default checks to ensure generated code generation is curr To trigger the check, you can use `make generate-check` provided your Makefile properly includes the boilerplate-generated include `boilerplate/generated-includes.mk`. Checks consist of: -* Checking all files are committed to ensure a safe point to revert to in case of error -* Running the `make generate` command (see below) to regenerate the needed code -* Checking if this results in any new uncommitted files in the git project or if all is clean. + +- Checking all files are committed to ensure a safe point to revert to in case of error +- Running the `make generate` command (see below) to regenerate the needed code +- Checking if this results in any new uncommitted files in the git project or if all is clean. `make generate` does the following: -* `operator-sdk generate crds` and `k8s`. This is a no-op if your + +- generate crds and deepcopy via controller-gen. This is a no-op if your operator has no APIs. -* `openapi-gen`. This is a no-op if your operator has no APIs. -* `go generate`. This is a no-op if you have no `//go:generate` +- `openapi-gen`. This is a no-op if your operator has no APIs. +- `go generate`. This is a no-op if you have no `//go:generate` directives in your code. + +## FIPS (Federal Information Processing Standards) + +To enable FIPS in your build there is a `make ensure-fips` target. + +Add `FIPS_ENABLED=true` to your repos Makefile. Please ensure that this variable is added **before** including boilerplate Makefiles. + +e.g. + +```.mk +FIPS_ENABLED=true + +include boilerplate/generated-includes.mk +``` + +`ensure-fips` will add a [fips.go](./fips.go) file in the same directory as the `main.go` file. (Please commit this file as normal) + +`fips.go` will import the necessary packages to restrict all TLS configuration to FIPS-approved settings. + +With `FIPS_ENABLED=true`, `ensure-fips` is always run before `make go-build` diff --git a/boilerplate/openshift/golang-osd-operator/app-sre-build-deploy.sh b/boilerplate/openshift/golang-osd-operator/app-sre-build-deploy.sh index 1af16793..0aa6dd79 100755 --- a/boilerplate/openshift/golang-osd-operator/app-sre-build-deploy.sh +++ b/boilerplate/openshift/golang-osd-operator/app-sre-build-deploy.sh @@ -68,6 +68,6 @@ for channel in staging production; do echo "properly. Nothing to do!" else # build the CSV and create & push image catalog for the appropriate channel - make ${channel}-common-csv-build ${channel}-catalog-build ${channel}-catalog-publish + make ${channel}-csv-build ${channel}-catalog-build ${channel}-catalog-publish fi done diff --git a/boilerplate/openshift/golang-osd-operator/app-sre.md b/boilerplate/openshift/golang-osd-operator/app-sre.md index 2c4d59cc..dfd9ede7 100644 --- a/boilerplate/openshift/golang-osd-operator/app-sre.md +++ b/boilerplate/openshift/golang-osd-operator/app-sre.md @@ -21,6 +21,17 @@ If not, you will need to set the `IMAGE_REGISTRY` environment variable (see [bel The SaaS bundle repository for `$OPERATOR_NAME` should be located at `https://gitlab.cee.redhat.com/service/saas-{operator}-bundle`, e.g. https://gitlab.cee.redhat.com/service/saas-deadmanssnitch-operator-bundle. Fork it to your personal namespace. +If you have already forked it to your personal namespace and/or used your fork for testing app-sre scripts at some time in the past, it is recommended that you bring your fork in sync with how upstream appears, or else the catalog you test with may not work correctly when deployed. + +An example of how to do this for the `staging` branch is below (`production` steps are the same): + +``` +git checkout staging +git pull upstream staging +git reset --hard upstream/staging +git push origin staging --force +``` + ## Set environment variables ```bash # The process creates artifacts in your git clone. Some of the make targets diff --git a/boilerplate/openshift/golang-osd-operator/build-opm-catalog.sh b/boilerplate/openshift/golang-osd-operator/build-opm-catalog.sh index f856aae9..196d07a7 100755 --- a/boilerplate/openshift/golang-osd-operator/build-opm-catalog.sh +++ b/boilerplate/openshift/golang-osd-operator/build-opm-catalog.sh @@ -320,6 +320,14 @@ function main() { local versions # shellcheck disable=SC2207 versions=($(get_prev_operator_version "$bundle_versions_file")) + # This condition is triggered when an operator is built for the first time. In such case the + # get_prev_operator_version returns an empty string and causes undefined variables failures + # in a few lines below. + if [ -z ${versions+x} ] + then + versions[0]="" + versions[1]="" + fi local prev_operator_version="${versions[0]}" local prev_good_operator_version="${versions[1]}" local skip_versions=("${versions[@]:2}") diff --git a/boilerplate/openshift/golang-osd-operator/codecov-secret-mapping b/boilerplate/openshift/golang-osd-operator/codecov-secret-mapping deleted file mode 100755 index d75dcbeb..00000000 --- a/boilerplate/openshift/golang-osd-operator/codecov-secret-mapping +++ /dev/null @@ -1,62 +0,0 @@ -#!/usr/bin/env bash - -set -e - -REPO_ROOT=$(git rev-parse --show-toplevel) -source $REPO_ROOT/boilerplate/_lib/common.sh -source $REPO_ROOT/boilerplate/_lib/release.sh - -cmd=${0##*/} - -usage() { - cat <> $mapping_file -- from: - namespace: sd-sre-secrets - name: $secret_name - to: - namespace: ci - name: $secret_name -EOF - -release_branch=$CONSUMER_ORG-$CONSUMER_NAME-$DEFAULT_BRANCH-boilerplate-$cmd - -release_done_msg $release_branch diff --git a/boilerplate/openshift/golang-osd-operator/configure-fips.sh b/boilerplate/openshift/golang-osd-operator/configure-fips.sh new file mode 100755 index 00000000..e506a00d --- /dev/null +++ b/boilerplate/openshift/golang-osd-operator/configure-fips.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash + +set -e + +REPO_ROOT=$(git rev-parse --show-toplevel) +CONVENTION_DIR="$REPO_ROOT/boilerplate/openshift/golang-osd-operator" +PRE_V1_SDK_MANAGER_DIR="$REPO_ROOT/cmd/manager" + +if [[ -d "$PRE_V1_SDK_MANAGER_DIR" ]] +then + MAIN_DIR=$PRE_V1_SDK_MANAGER_DIR +else + MAIN_DIR=$REPO_ROOT +fi + +echo "Writing fips file at $MAIN_DIR/fips.go" + +cp $CONVENTION_DIR/fips.go.tmplt "$MAIN_DIR/fips.go" \ No newline at end of file diff --git a/boilerplate/openshift/golang-osd-operator/csv-generate/catalog-build.sh b/boilerplate/openshift/golang-osd-operator/csv-generate/catalog-build.sh index 8012940c..8b5f1d52 100755 --- a/boilerplate/openshift/golang-osd-operator/csv-generate/catalog-build.sh +++ b/boilerplate/openshift/golang-osd-operator/csv-generate/catalog-build.sh @@ -23,6 +23,13 @@ while getopts "o:c:r:" option; do esac done +# Detect the container engine to use, allowing override from the env +CONTAINER_ENGINE=${CONTAINER_ENGINE:-$(command -v podman || command -v docker || true)} +if [[ -z "$CONTAINER_ENGINE" ]]; then + echo "WARNING: Couldn't find a container engine! Defaulting to docker." + CONTAINER_ENGINE=docker +fi + # Checking parameters check_mandatory_params operator_channel operator_name @@ -56,13 +63,29 @@ EOF # Build registry cat < $DOCKERFILE_REGISTRY -FROM quay.io/openshift/origin-operator-registry:4.8.0 +FROM quay.io/openshift/origin-operator-registry:4.10.0 AS builder COPY $SAAS_OPERATOR_DIR manifests RUN initializer --permissive + +FROM registry.access.redhat.com/ubi8/ubi-micro:8.6-484 + +COPY --from=builder /bin/registry-server /bin/registry-server +COPY --from=builder /bin/grpc_health_probe /bin/grpc_health_probe +COPY --from=builder /bin/initializer /bin/initializer + +WORKDIR /registry +RUN chgrp -R 0 /registry && chmod -R g+rwx /registry + +USER 1001 + +COPY --from=builder /registry /registry + +EXPOSE 50051 + CMD ["registry-server", "-t", "/tmp/terminate.log"] EOF -docker build -f $DOCKERFILE_REGISTRY --tag "${registry_image}:${operator_channel}-latest" . +${CONTAINER_ENGINE} build --pull -f $DOCKERFILE_REGISTRY --tag "${registry_image}:${operator_channel}-latest" . if [ $? -ne 0 ] ; then echo "docker build failed, exiting..." diff --git a/boilerplate/openshift/golang-osd-operator/csv-generate/catalog-publish.sh b/boilerplate/openshift/golang-osd-operator/csv-generate/catalog-publish.sh index b5665f8a..16205295 100755 --- a/boilerplate/openshift/golang-osd-operator/csv-generate/catalog-publish.sh +++ b/boilerplate/openshift/golang-osd-operator/csv-generate/catalog-publish.sh @@ -41,6 +41,17 @@ BUNDLE_DIR="${SAAS_OPERATOR_DIR}/${operator_name}" OPERATOR_NEW_VERSION=$(ls "${BUNDLE_DIR}" | sort -t . -k 3 -g | tail -n 1) OPERATOR_PREV_VERSION=$(ls "${BUNDLE_DIR}" | sort -t . -k 3 -g | tail -n 2 | head -n 1) +# Get container engine +CONTAINER_ENGINE=$(command -v podman || command -v docker || true) +[[ -n "$CONTAINER_ENGINE" ]] || echo "WARNING: Couldn't find a container engine. Assuming you already in a container, running unit tests." >&2 + +# Set SRC container transport based on container engine +if [[ "${CONTAINER_ENGINE##*/}" == "podman" ]]; then + SRC_CONTAINER_TRANSPORT="containers-storage" +else + SRC_CONTAINER_TRANSPORT="docker-daemon" +fi + # Checking SAAS_OPERATOR_DIR exist if [ ! -d "${SAAS_OPERATOR_DIR}/.git" ] ; then echo "${SAAS_OPERATOR_DIR} should exist and be a git repository" @@ -85,7 +96,7 @@ popd if [ "$push_catalog" = true ] ; then # push image skopeo copy --dest-creds "${QUAY_USER}:${QUAY_TOKEN}" \ - "docker-daemon:${registry_image}:${operator_channel}-latest" \ + "${SRC_CONTAINER_TRANSPORT}:${registry_image}:${operator_channel}-latest" \ "docker://${registry_image}:${operator_channel}-latest" if [ $? -ne 0 ] ; then @@ -94,7 +105,7 @@ if [ "$push_catalog" = true ] ; then fi skopeo copy --dest-creds "${QUAY_USER}:${QUAY_TOKEN}" \ - "docker-daemon:${registry_image}:${operator_channel}-latest" \ + "${SRC_CONTAINER_TRANSPORT}:${registry_image}:${operator_channel}-latest" \ "docker://${registry_image}:${operator_channel}-${operator_commit_hash}" if [ $? -ne 0 ] ; then diff --git a/boilerplate/openshift/golang-osd-operator/csv-generate/common-generate-operator-bundle.py b/boilerplate/openshift/golang-osd-operator/csv-generate/common-generate-operator-bundle.py index d27fc736..3bf6c0a1 100755 --- a/boilerplate/openshift/golang-osd-operator/csv-generate/common-generate-operator-bundle.py +++ b/boilerplate/openshift/golang-osd-operator/csv-generate/common-generate-operator-bundle.py @@ -44,7 +44,7 @@ parser = argparse.ArgumentParser() parser.add_argument("-o", "--operator-name", type=str, help="Name of the operator", required=True) parser.add_argument("-d", "--output-dir", type=str, help="Directory for the CSV generation", required=True) -parser.add_argument("-p", "--previous-version", type=str, help="Semver of the version being replaced", required=True) +parser.add_argument("-p", "--previous-version", type=str, help="Semver of the version being replaced", required=False) parser.add_argument("-i", "--operator-image", type=str, help="Base index image to be used", required=True) parser.add_argument("-V", "--operator-version", type=str, help="The full version of the operator (without the leading `v`): {major}.{minor}.{commit-number}-{hash}", required=True) args = parser.parse_args() @@ -240,16 +240,18 @@ def trim_index(index, kind, item): csv['spec']['customresourcedefinitions'] = {'owned': []} for crd in by_kind.get('CustomResourceDefinition', []): log_resource(crd) + # And register the CRD as "owned" in the CSV - csv['spec']['customresourcedefinitions']['owned'].append( - { - "name": crd["metadata"]["name"], - "description": crd["spec"]["names"]["kind"], - "displayName": crd["spec"]["names"]["kind"], - "kind": crd["spec"]["names"]["kind"], - "version": crd["spec"]["version"] - } - ) + for version in crd["spec"]["versions"]: + csv['spec']['customresourcedefinitions']['owned'].append( + { + "name": crd["metadata"]["name"], + "description": crd["spec"]["names"]["kind"], + "displayName": crd["spec"]["names"]["kind"], + "kind": crd["spec"]["names"]["kind"], + "version": version["name"] + } + ) # These will be written to the bundle at the end along with generic resources ## Process [Cluster]Role[Binding]s (TODO: Match up ServiceAccounts) @@ -305,7 +307,7 @@ def trim_index(index, kind, item): csv['spec']['install']['spec']['permissions'].append( { 'rules': role['rules'], - 'serviceAccountName': role_binding['subjects'][0]['name'] + 'serviceAccountName': role_binding['subjects'][0]['name'] } ) trim_index(by_kind, 'Role', role) @@ -316,6 +318,21 @@ def trim_index(index, kind, item): deploy = by_kind['Deployment'][0] # Use the operator image pull spec we were passed deploy['spec']['template']['spec']['containers'][0]['image'] = operator_image +# Add or replace OPERATOR_IMAGE env var +env = deploy['spec']['template']['spec']['containers'][0].get('env') +if env: + # Does OPERATOR_IMAGE key already exist in spec? If so, update value + for entry in env: + if entry['name'] == 'OPERATOR_IMAGE': + entry['value'] = operator_image + break + # If not, add it + else: + env.append(dict(name='OPERATOR_IMAGE', value=operator_image)) +else: + # The container has no environment variables, so just set this one + env = dict(name='OPERATOR_IMAGE', value=operator_image) + csv['spec']['install']['spec']['deployments'] = [ { 'name': deploy['metadata']['name'], @@ -346,7 +363,8 @@ def trim_index(index, kind, item): # Update the versions to include git hash: csv['metadata']['name'] = f"{OPERATOR_NAME}.v{full_version}" csv['spec']['version'] = full_version -csv['spec']['replaces'] = f"{OPERATOR_NAME}.v{prev_version}" +if prev_version: + csv['spec']['replaces'] = f"{OPERATOR_NAME}.v{prev_version}" # Set the CSV createdAt annotation: now = datetime.datetime.now() diff --git a/boilerplate/openshift/golang-osd-operator/csv-generate/csv-generate.mk b/boilerplate/openshift/golang-osd-operator/csv-generate/csv-generate.mk index 8aeff43b..21e9ea43 100644 --- a/boilerplate/openshift/golang-osd-operator/csv-generate/csv-generate.mk +++ b/boilerplate/openshift/golang-osd-operator/csv-generate/csv-generate.mk @@ -1,67 +1,47 @@ - - -.PHONY: staging-hack-csv-build -staging-hack-csv-build: - @${CONVENTION_DIR}/csv-generate/csv-generate.sh -o $(OPERATOR_NAME) -i $(OPERATOR_IMAGE) -V $(OPERATOR_VERSION) -c staging -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) -g hack - -.PHONY: staging-common-csv-build -staging-common-csv-build: - @${CONVENTION_DIR}/csv-generate/csv-generate.sh -o $(OPERATOR_NAME) -i $(OPERATOR_IMAGE) -V $(OPERATOR_VERSION) -c staging -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) -g common - .PHONY: staging-csv-build -staging-csv-build: staging-hack-csv-build - -.PHONY: staging-common-csv-build-and-diff -staging-common-csv-build-and-diff: - @${CONVENTION_DIR}/csv-generate/csv-generate.sh -o $(OPERATOR_NAME) -i $(OPERATOR_IMAGE) -V $(OPERATOR_VERSION) -c staging -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) -g common -d +staging-csv-build: + @${CONVENTION_DIR}/csv-generate/csv-generate.sh -o $(OPERATOR_NAME) -i $(OPERATOR_IMAGE) -V $(OPERATOR_VERSION) -c staging -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) .PHONY: staging-catalog-build -staging-catalog-build: +staging-catalog-build: @${CONVENTION_DIR}/csv-generate/catalog-build.sh -o $(OPERATOR_NAME) -c staging -r ${REGISTRY_IMAGE} - + .PHONY: staging-saas-bundle-push -staging-saas-bundle-push: +staging-saas-bundle-push: @${CONVENTION_DIR}/csv-generate/catalog-publish.sh -o $(OPERATOR_NAME) -c staging -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) -r ${REGISTRY_IMAGE} - + .PHONY: staging-catalog-publish -staging-catalog-publish: +staging-catalog-publish: @${CONVENTION_DIR}/csv-generate/catalog-publish.sh -o $(OPERATOR_NAME) -c staging -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) -p -r ${REGISTRY_IMAGE} - + .PHONY: staging-catalog-build-and-publish -staging-catalog-build-and-publish: +staging-catalog-build-and-publish: @$(MAKE) -s staging-csv-build --no-print-directory @$(MAKE) -s staging-catalog-build --no-print-directory - @$(MAKE) -s staging-catalog-publish --no-print-directory - + @$(MAKE) -s staging-catalog-publish --no-print-directory + .PHONY: production-hack-csv-build -production-hack-csv-build: +production-hack-csv-build: @${CONVENTION_DIR}/csv-generate/csv-generate.sh -o $(OPERATOR_NAME) -i $(OPERATOR_IMAGE) -V $(OPERATOR_VERSION) -c production -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) -g hack - -.PHONY: production-common-csv-build -production-common-csv-build: - @${CONVENTION_DIR}/csv-generate/csv-generate.sh -o $(OPERATOR_NAME) -i $(OPERATOR_IMAGE) -V $(OPERATOR_VERSION) -c production -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) -g common - -.PHONY: production-csv-build -production-csv-build: production-hack-csv-build -.PHONY: production-common-csv-build-and-diff -production-common-csv-build-and-diff: - @${CONVENTION_DIR}/csv-generate/csv-generate.sh -o $(OPERATOR_NAME) -i $(OPERATOR_IMAGE) -V $(OPERATOR_VERSION) -c production -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) -g common -d +.PHONY: production-csv-build +production-csv-build: + @${CONVENTION_DIR}/csv-generate/csv-generate.sh -o $(OPERATOR_NAME) -i $(OPERATOR_IMAGE) -V $(OPERATOR_VERSION) -c production -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) .PHONY: production-catalog-build -production-catalog-build: +production-catalog-build: @${CONVENTION_DIR}/csv-generate/catalog-build.sh -o $(OPERATOR_NAME) -c production -r ${REGISTRY_IMAGE} - + .PHONY: production-saas-bundle-push -production-saas-bundle-push: +production-saas-bundle-push: @${CONVENTION_DIR}/csv-generate/catalog-publish.sh -o $(OPERATOR_NAME) -c production -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) -r ${REGISTRY_IMAGE} - + .PHONY: production-catalog-publish -production-catalog-publish: +production-catalog-publish: @${CONVENTION_DIR}/csv-generate/catalog-publish.sh -o $(OPERATOR_NAME) -c production -H $(CURRENT_COMMIT) -n $(COMMIT_NUMBER) -p -r ${REGISTRY_IMAGE} - + .PHONY: production-catalog-build-and-publish -production-catalog-build-and-publish: +production-catalog-build-and-publish: @$(MAKE) -s production-csv-build --no-print-directory @$(MAKE) -s production-catalog-build --no-print-directory - @$(MAKE) -s production-catalog-publish --no-print-directory + @$(MAKE) -s production-catalog-publish --no-print-directory diff --git a/boilerplate/openshift/golang-osd-operator/csv-generate/csv-generate.sh b/boilerplate/openshift/golang-osd-operator/csv-generate/csv-generate.sh index 668e1fa5..5e7a7104 100755 --- a/boilerplate/openshift/golang-osd-operator/csv-generate/csv-generate.sh +++ b/boilerplate/openshift/golang-osd-operator/csv-generate/csv-generate.sh @@ -4,7 +4,7 @@ set -e source `dirname $0`/common.sh -usage() { echo "Usage: $0 -o operator-name -c saas-repository-channel -H operator-commit-hash -n operator-commit-number -i operator-image -V operator-version -g [hack|common][-d]" 1>&2; exit 1; } +usage() { echo "Usage: $0 -o operator-name -c saas-repository-channel -H operator-commit-hash -n operator-commit-number -i operator-image -V operator-version" 1>&2; exit 1; } # TODO : Add support of long-options while getopts "c:dg:H:i:n:o:V:" option; do @@ -12,17 +12,6 @@ while getopts "c:dg:H:i:n:o:V:" option; do c) operator_channel=${OPTARG} ;; - d) - diff_generate=true - ;; - g) - if [ "${OPTARG}" = "hack" ] || [ "${OPTARG}" = "common" ] ; then - generate_script=${OPTARG} - else - # TODO : Case to be tested - echo "Incorrect value for '-g'. Expecting 'hack' or 'common'. Got ${OPTARG}" - fi - ;; H) operator_commit_hash=${OPTARG} ;; @@ -49,7 +38,7 @@ while getopts "c:dg:H:i:n:o:V:" option; do done # Checking parameters -check_mandatory_params operator_channel operator_image operator_version operator_name operator_commit_hash operator_commit_number generate_script +check_mandatory_params operator_channel operator_image operator_version operator_name operator_commit_hash operator_commit_number # Use set container engine or select one from available binaries if [[ -z "$CONTAINER_ENGINE" ]]; then @@ -79,135 +68,112 @@ fi SAAS_OPERATOR_DIR="saas-${operator_name}-bundle" BUNDLE_DIR="$SAAS_OPERATOR_DIR/${operator_name}/" -if [ "$diff_generate" = true ] ; then - OPERATOR_NEW_VERSION=$(ls "$BUNDLE_DIR" | sort -t . -k 3 -g | tail -n 1) - OPERATOR_PREV_VERSION=$(ls "${BUNDLE_DIR}" | sort -t . -k 3 -g | tail -n 2 | head -n 1) - OUTPUT_DIR="output-comparison" +rm -rf "$SAAS_OPERATOR_DIR" +git clone --branch "$operator_channel" ${GIT_PATH} "$SAAS_OPERATOR_DIR" - # For diff usecase, checking there is already a generated CSV - if [ ! -f ${BUNDLE_DIR}/${OPERATOR_NEW_VERSION}/*.clusterserviceversion.yaml ] ; then - echo "You need to generate CSV with your legacy script before trying to run the diff option" +# If this is a brand new SaaS setup, then set up accordingly +if [[ ! -d "${BUNDLE_DIR}" ]]; then + echo "Setting up new SaaS operator dir: ${BUNDLE_DIR}" + mkdir "${BUNDLE_DIR}" +fi + +# For testing purposes, support disabling anything that relies on +# querying the saas file in app-interface. This includes pruning +# undeployed commits in production. +# FIXME -- This should go away when we're querying app-interface via +# graphql. +if [[ -z "$SKIP_SAAS_FILE_CHECKS" ]]; then + # PATH to saas file in app-interface + SAAS_FILE_URL="https://gitlab.cee.redhat.com/service/app-interface/raw/master/data/services/osd-operators/cicd/saas/saas-${operator_name}.yaml" + + # MANAGED_RESOURCE_TYPE + # SAAS files contain the type of resources managed within the OC templates that + # are being applied to hive. + # For customer cluster resources this should always be of type "SelectorSyncSet" resources otherwise + # can't be sync'd to the customer cluster. We're explicity selecting the first element in the array. + # We can safely assume anything that is not of type "SelectorSyncSet" is being applied to hive only + # since it matches ClusterDeployment resources. + # From this we'll assume that the namespace reference in resourceTemplates to be: + # For customer clusters: /services/osd-operators/namespace//namespaces/cluster-scope.yaml + # For hive clusters: /services/osd-operators/namespace//namespaces/.yaml + MANAGED_RESOURCE_TYPE=$(curl -s "${SAAS_FILE_URL}" | \ + $YQ_CMD r - "managedResourceTypes[0]" + ) + if [[ "${MANAGED_RESOURCE_TYPE}" == "" ]]; then + echo "Unabled to determine if SAAS file managed resource type" exit 1 fi -else - rm -rf "$SAAS_OPERATOR_DIR" - git clone --branch "$operator_channel" ${GIT_PATH} "$SAAS_OPERATOR_DIR" - - # For testing purposes, support disabling anything that relies on - # querying the saas file in app-interface. This includes pruning - # undeployed commits in production. - # FIXME -- This should go away when we're querying app-interface via - # graphql. - if [[ -z "$SKIP_SAAS_FILE_CHECKS" ]]; then - # PATH to saas file in app-interface - SAAS_FILE_URL="https://gitlab.cee.redhat.com/service/app-interface/raw/master/data/services/osd-operators/cicd/saas/saas-${operator_name}.yaml" - - # MANAGED_RESOURCE_TYPE - # SAAS files contain the type of resources managed within the OC templates that - # are being applied to hive. - # For customer cluster resources this should always be of type "SelectorSyncSet" resources otherwise - # can't be sync'd to the customer cluster. We're explicity selecting the first element in the array. - # We can safely assume anything that is not of type "SelectorSyncSet" is being applied to hive only - # since it matches ClusterDeployment resources. - # From this we'll assume that the namespace reference in resourceTemplates to be: - # For customer clusters: /services/osd-operators/namespace//namespaces/cluster-scope.yaml - # For hive clusters: /services/osd-operators/namespace//namespaces/.yaml - MANAGED_RESOURCE_TYPE=$(curl -s "${SAAS_FILE_URL}" | \ - $YQ_CMD r - "managedResourceTypes[0]" - ) - - if [[ "${MANAGED_RESOURCE_TYPE}" == "" ]]; then - echo "Unabled to determine if SAAS file managed resource type" - exit 1 - fi - # Determine namespace reference path, output resource type - if [[ "${MANAGED_RESOURCE_TYPE}" == "SelectorSyncSet" ]]; then - echo "SAAS file is NOT applied to Hive, MANAGED_RESOURCE_TYPE=$MANAGED_RESOURCE_TYPE" - resource_template_ns_path="/services/osd-operators/namespaces/hivep01ue1/cluster-scope.yml" - else - echo "SAAS file is applied to Hive, MANAGED_RESOURCE_TYPE=$MANAGED_RESOURCE_TYPE" - resource_template_ns_path="/services/osd-operators/namespaces/hivep01ue1/${operator_name}.yml" - fi + # Determine namespace reference path, output resource type + if [[ "${MANAGED_RESOURCE_TYPE}" == "SelectorSyncSet" ]]; then + echo "SAAS file is NOT applied to Hive, MANAGED_RESOURCE_TYPE=$MANAGED_RESOURCE_TYPE" + resource_template_ns_path="/services/osd-operators/namespaces/hivep01ue1/cluster-scope.yml" + else + echo "SAAS file is applied to Hive, MANAGED_RESOURCE_TYPE=$MANAGED_RESOURCE_TYPE" + resource_template_ns_path="/services/osd-operators/namespaces/hivep01ue1/${operator_name}.yml" + fi - # remove any versions more recent than deployed hash - if [[ "$operator_channel" == "production" ]]; then - if [ -z "$DEPLOYED_HASH" ] ; then - DEPLOYED_HASH=$( - curl -s "${SAAS_FILE_URL}" | \ - $YQ_CMD r - "resourceTemplates[*].targets(namespace.\$ref==${resource_template_ns_path}).ref" - ) - fi + # remove any versions more recent than deployed hash + if [[ "$operator_channel" == "production" ]]; then + if [ -z "$DEPLOYED_HASH" ] ; then + DEPLOYED_HASH=$( + curl -s "${SAAS_FILE_URL}" | \ + $YQ_CMD r - "resourceTemplates[*].targets(namespace.\$ref==${resource_template_ns_path}).ref" + ) + fi - # Ensure that our query for the current deployed hash worked - # Validate that our DEPLOYED_HASH var isn't empty. - # Although we have `set -e` defined the docker container isn't returning - # an error and allowing the script to continue - echo "Current deployed production HASH: $DEPLOYED_HASH" + # Ensure that our query for the current deployed hash worked + # Validate that our DEPLOYED_HASH var isn't empty. + # Although we have `set -e` defined the docker container isn't returning + # an error and allowing the script to continue + echo "Current deployed production HASH: $DEPLOYED_HASH" - if [[ ! "${DEPLOYED_HASH}" =~ [0-9a-f]{40} ]]; then - echo "Error discovering current production deployed HASH" - exit 1 - fi + if [[ ! "${DEPLOYED_HASH}" =~ [0-9a-f]{40} ]]; then + echo "Error discovering current production deployed HASH" + exit 1 + fi - delete=false - # Sort based on commit number - for version in $(ls $BUNDLE_DIR | sort -t . -k 3 -g); do - # skip if not directory - [ -d "$BUNDLE_DIR/$version" ] || continue + delete=false + # Sort based on commit number + for version in $(ls $BUNDLE_DIR | sort -t . -k 3 -g); do + # skip if not directory + [ -d "$BUNDLE_DIR/$version" ] || continue - if [[ "$delete" == false ]]; then - short_hash=$(echo "$version" | cut -d- -f2) + if [[ "$delete" == false ]]; then + short_hash=$(echo "$version" | cut -d- -f2) - if [[ "$DEPLOYED_HASH" == "${short_hash}"* ]]; then - delete=true - fi - else - rm -rf "${BUNDLE_DIR:?BUNDLE_DIR var not set}/$version" + if [[ "$DEPLOYED_HASH" == "${short_hash}"* ]]; then + delete=true fi - done - fi - fi # End of SKIP_SAAS_FILE_CHECKS granny switch - - OPERATOR_PREV_VERSION=$(ls "$BUNDLE_DIR" | sort -t . -k 3 -g | tail -n 1) - OPERATOR_NEW_VERSION="${operator_version}" - OUTPUT_DIR=${BUNDLE_DIR} -fi - -if [[ "$generate_script" = "common" ]] ; then - # Jenkins can't be relied upon to have py3, so run the generator in - # a container. - # ...Unless we're already in a container, which is how boilerplate - # CI runs. We have py3 there, so run natively in that case. - if [[ -z "$CONTAINER_ENGINE" ]]; then - ./boilerplate/openshift/golang-osd-operator/csv-generate/common-generate-operator-bundle.py -o ${operator_name} -d ${OUTPUT_DIR} -p ${OPERATOR_PREV_VERSION} -i ${REPO_DIGEST} -V ${operator_version} - else - $CONTAINER_ENGINE run --rm -v `pwd`:`pwd` -u `id -u`:0 -w `pwd` registry.access.redhat.com/ubi8/python-36:1-134 /bin/bash -c "python -m pip install oyaml; python ./boilerplate/openshift/golang-osd-operator/csv-generate/common-generate-operator-bundle.py -o ${operator_name} -d ${OUTPUT_DIR} -p ${OPERATOR_PREV_VERSION} -i ${REPO_DIGEST} -V ${operator_version}" - fi -elif [[ "$generate_script" = "hack" ]] ; then - if [ -z "$OPERATOR_PREV_VERSION" ] ; then - OPERATOR_PREV_VERSION="no-version" - DELETE_REPLACE=true + else + rm -rf "${BUNDLE_DIR:?BUNDLE_DIR var not set}/$version" + fi + done fi +fi # End of SKIP_SAAS_FILE_CHECKS granny switch - ./hack/generate-operator-bundle.py ${OUTPUT_DIR} ${OPERATOR_PREV_VERSION} ${operator_commit_number} ${operator_commit_hash} ${REPO_DIGEST} +OPERATOR_PREV_VERSION=$(ls "$BUNDLE_DIR" | sort -t . -k 3 -g | tail -n 1) +OPERATOR_NEW_VERSION="${operator_version}" +OUTPUT_DIR=${BUNDLE_DIR} - if [ ! -z "${DELETE_REPLACE}" ] ; then - yq d -i output-comparison/${OPERATOR_NEW_VERSION}/*.clusterserviceversion.yaml 'spec.replaces' - fi +# If setting up a new SaaS repo, there is no previous version when building a bundle +# Optionally pass it to the bundle generator in that case. +if [[ -z "${OPERATOR_PREV_VERSION}" ]]; then + PREV_VERSION_OPTS="" +else + PREV_VERSION_OPTS="-p ${OPERATOR_PREV_VERSION}" fi - -if [ "$diff_generate" = true ] ; then - # TODO : Current hack script does not allow to generate the CSV for the comparison (it will generate a different version that the common one because there is 1 extra version in the history) - if [[ "$generate_script" = "hack" ]] ; then - echo "Generating with the common script and after, generating with the hack script is not supported yet. For comparison, please first generate with hack script, and then build/compare with the common script" - exit 1 - # Preparing yamls for the diff by removing the creation timestamp - elif [ -f ${BUNDLE_DIR}/${OPERATOR_NEW_VERSION}/*.clusterserviceversion.yaml ] ; then - yq d ${BUNDLE_DIR}/${OPERATOR_NEW_VERSION}/*.clusterserviceversion.yaml 'metadata.annotations.createdAt' > output-comparison/hack_generate.yaml - yq d output-comparison/${OPERATOR_NEW_VERSION}/*.clusterserviceversion.yaml 'metadata.annotations.createdAt' > output-comparison/common_generate.yaml - # Diff on the filtered files - diff output-comparison/hack_generate.yaml output-comparison/common_generate.yaml +# Jenkins can't be relied upon to have py3, so run the generator in +# a container. +# ...Unless we're already in a container, which is how boilerplate +# CI runs. We have py3 there, so run natively in that case. +if [[ -z "$CONTAINER_ENGINE" ]]; then + ./boilerplate/openshift/golang-osd-operator/csv-generate/common-generate-operator-bundle.py -o ${operator_name} -d ${OUTPUT_DIR} ${PREV_VERSION_OPTS} -i ${REPO_DIGEST} -V ${operator_version} +else + if [[ ${CONTAINER_ENGINE##*/} == "podman" ]]; then + CE_OPTS="--userns keep-id -v `pwd`:`pwd`:Z" + else + CE_OPTS="-v `pwd`:`pwd`" fi + $CONTAINER_ENGINE run --rm ${CE_OPTS} -u `id -u`:0 -w `pwd` registry.access.redhat.com/ubi8/python-36:1-134 /bin/bash -c "python -m pip install oyaml; python ./boilerplate/openshift/golang-osd-operator/csv-generate/common-generate-operator-bundle.py -o ${operator_name} -d ${OUTPUT_DIR} ${PREV_VERSION_OPTS} -i ${REPO_DIGEST} -V ${operator_version}" fi - diff --git a/boilerplate/openshift/golang-osd-operator/ensure.sh b/boilerplate/openshift/golang-osd-operator/ensure.sh index 36e24e47..a0a97575 100755 --- a/boilerplate/openshift/golang-osd-operator/ensure.sh +++ b/boilerplate/openshift/golang-osd-operator/ensure.sh @@ -30,59 +30,6 @@ golangci-lint) fi ;; -operator-sdk) - ######################################################### - # Ensure operator-sdk is installed at the desired version - # When done, ./.operator-sdk/bin/operator-sdk will be a - # symlink to the appropriate executable. - ######################################################### - # First discover the desired version from go.mod - # The following properly takes `replace` directives into account. - wantver=$(go list -json -m github.com/operator-framework/operator-sdk | jq -r 'if .Replace != null then .Replace.Version else .Version end') - echo "go.mod says you want operator-sdk $wantver" - # Where we'll put our (binary and) symlink - mkdir -p .operator-sdk/bin - cd .operator-sdk/bin - # Discover existing, giving preference to one already installed in - # this path, because that has a higher probability of being the - # right one. - if [[ -x ./operator-sdk ]] && [[ "$(osdk_version ./operator-sdk)" == "$wantver" ]]; then - echo "operator-sdk $wantver already installed" - exit 0 - fi - # Is there one in $PATH? - if which operator-sdk && [[ $(osdk_version $(which operator-sdk)) == "$wantver" ]]; then - osdk=$(realpath $(which operator-sdk)) - echo "Found at $osdk" - else - case "$(uname -s)" in - Linux*) - binary="operator-sdk-${wantver}-x86_64-linux-gnu" - ;; - Darwin*) - binary="operator-sdk-${wantver}-x86_64-apple-darwin" - ;; - *) - echo "OS unsupported" - exit 1 - ;; - esac - # The boilerplate backing image sets up binaries with the full - # name in /usr/local/bin, so look for the right one of those - if which $binary; then - osdk=$(realpath $(which $binary)) - else - echo "Downloading $binary" - curl -OJL https://github.com/operator-framework/operator-sdk/releases/download/${wantver}/${binary} - chmod +x ${binary} - osdk=${binary} - fi - fi - # Create (or overwrite) the symlink to the binary we discovered or - # downloaded above. - ln -sf $osdk operator-sdk - ;; - opm) mkdir -p .opm/bin cd .opm/bin diff --git a/boilerplate/openshift/golang-osd-operator/fips.go.tmplt b/boilerplate/openshift/golang-osd-operator/fips.go.tmplt new file mode 100644 index 00000000..bc0d4547 --- /dev/null +++ b/boilerplate/openshift/golang-osd-operator/fips.go.tmplt @@ -0,0 +1,15 @@ +// +build fips_enabled + +// BOILERPLATE GENERATED -- DO NOT EDIT +// Run 'make ensure-fips' to regenerate + +package main + +import ( + _ "crypto/tls/fipsonly" + "fmt" +) + +func init() { + fmt.Println("***** Starting with FIPS crypto enabled *****") +} diff --git a/boilerplate/openshift/golang-osd-operator/golangci.yml b/boilerplate/openshift/golang-osd-operator/golangci.yml index 77ff6450..e4a3f2d7 100644 --- a/boilerplate/openshift/golang-osd-operator/golangci.yml +++ b/boilerplate/openshift/golang-osd-operator/golangci.yml @@ -18,6 +18,7 @@ linters: enable: - deadcode - errcheck + - gosec - gosimple - govet - ineffassign diff --git a/boilerplate/openshift/golang-osd-operator/operator-sdk-generate.sh b/boilerplate/openshift/golang-osd-operator/operator-sdk-generate.sh deleted file mode 100755 index 6aadb666..00000000 --- a/boilerplate/openshift/golang-osd-operator/operator-sdk-generate.sh +++ /dev/null @@ -1,47 +0,0 @@ -#!/usr/bin/env bash -set -eo pipefail - -### -# Run operator-sdk generate commands appropriate to the version of -# operator-sdk configured in the consuming repository. -### - -REPO_ROOT=$(git rev-parse --show-toplevel) - -source $REPO_ROOT/boilerplate/_lib/common.sh - -# There's nothing to generate if pkg/apis is empty (other than apis.go). -# And instead of succeeding gracefully, `operator-sdk generate` will -# fail if you try. So do our own check. -if ! /bin/ls -1 pkg/apis | grep -Fqv apis.go; then - echo "No APIs! Skipping operator-sdk generate." - exit 0 -fi - -$HERE/ensure.sh operator-sdk - -# Symlink to operator-sdk binary set up by `ensure.sh operator-sdk`: -OSDK=$REPO_ROOT/.operator-sdk/bin/operator-sdk - -VER=$(osdk_version $OSDK) - -# This explicitly lists the versions we know about. We don't support -# anything outside of that. -# NOTE: We are gluing to CRD v1beta1 for the moment. Support for v1 -# needs to be considered carefully in the context of -# - Hive v3 (which doesn't support v1) -# - When OCP will remove support for v1beta1 (currently we know it's -# deprecated in 4.6, but don't know when it's actually removed). -case $VER in - 'v0.15.1'|'v0.16.0') - # No-op: just declare support for these osdk versions. - ;; - 'v0.17.0'|'v0.17.1'|'v0.17.2'|'v0.18.2') - # The --crd-version flag was introduced in 0.17. v1beta1 is the - # default until 0.18, but let's be explicit. - _osdk_generate_crds_flags='--crd-version v1beta1' - ;; - *) err "Unsupported operator-sdk version $VER" ;; -esac -$OSDK generate crds $_osdk_generate_crds_flags -$OSDK generate k8s diff --git a/boilerplate/openshift/golang-osd-operator/standard.mk b/boilerplate/openshift/golang-osd-operator/standard.mk index 802109e7..69dcb8b8 100644 --- a/boilerplate/openshift/golang-osd-operator/standard.mk +++ b/boilerplate/openshift/golang-osd-operator/standard.mk @@ -15,8 +15,24 @@ ifndef VERSION_MINOR $(error VERSION_MINOR is not set; check project.mk file) endif -# Accommodate docker or podman -CONTAINER_ENGINE=$(shell command -v podman 2>/dev/null || command -v docker 2>/dev/null) +### Accommodate docker or podman +# +# The docker/podman creds cache needs to be in a location unique to this +# invocation; otherwise it could collide across jenkins jobs. We'll use +# a .docker folder relative to pwd (the repo root). +CONTAINER_ENGINE_CONFIG_DIR = .docker +# But docker and podman use different options to configure it :eyeroll: +# ==> Podman uses --authfile=PATH *after* the `login` subcommand; but +# also accepts REGISTRY_AUTH_FILE from the env. See +# https://www.mankier.com/1/podman-login#Options---authfile=path +export REGISTRY_AUTH_FILE = ${CONTAINER_ENGINE_CONFIG_DIR}/config.json +# ==> Docker uses --config=PATH *before* (any) subcommand; so we'll glue +# that to the CONTAINER_ENGINE variable itself. (NOTE: I tried half a +# dozen other ways to do this. This was the least ugly one that actually +# works.) +ifndef CONTAINER_ENGINE +CONTAINER_ENGINE=$(shell command -v podman 2>/dev/null || echo docker --config=$(CONTAINER_ENGINE_CONFIG_DIR)) +endif # Generate version and tag information from inputs COMMIT_NUMBER=$(shell git rev-list `git rev-list --parents HEAD | egrep "^[a-f0-9]{40}$$"`..HEAD --count) @@ -30,6 +46,9 @@ OPERATOR_IMAGE_URI=${IMG} OPERATOR_IMAGE_URI_LATEST=$(IMAGE_REGISTRY)/$(IMAGE_REPOSITORY)/$(IMAGE_NAME):latest OPERATOR_DOCKERFILE ?=build/Dockerfile REGISTRY_IMAGE=$(IMAGE_REGISTRY)/$(IMAGE_REPOSITORY)/$(IMAGE_NAME)-registry +#The api dir that latest osdk generated +NEW_API_DIR=./api +USE_OLD_SDK=$(shell if [[ -d "$(NEW_API_DIR)" ]];then echo FALSE;else echo TRUE;fi) # Consumer can optionally define ADDITIONAL_IMAGE_SPECS like: # define ADDITIONAL_IMAGE_SPECS @@ -49,18 +68,37 @@ OLM_CHANNEL ?= alpha REGISTRY_USER ?= REGISTRY_TOKEN ?= -CONTAINER_ENGINE_CONFIG_DIR = .docker BINFILE=build/_output/bin/$(OPERATOR_NAME) -MAINPACKAGE ?= ./cmd/manager +MAINPACKAGE = ./ +API_DIR = $(NEW_API_DIR) +ifeq ($(USE_OLD_SDK), TRUE) +MAINPACKAGE = ./cmd/manager +API_DIR = ./pkg/apis +endif GOOS?=$(shell go env GOOS) GOARCH?=$(shell go env GOARCH) +GOBIN?=$(shell go env GOBIN) # Consumers may override GOFLAGS_MOD e.g. to use `-mod=vendor` unexport GOFLAGS GOFLAGS_MOD ?= -GOENV=GOOS=${GOOS} GOARCH=${GOARCH} CGO_ENABLED=0 GOFLAGS=${GOFLAGS_MOD} + +# In openshift ci (Prow), we need to set $HOME to a writable directory else tests will fail +# because they don't have permissions to create /.local or /.cache directories +# as $HOME is set to "/" by default. +ifeq ($(HOME),/) +export HOME=/tmp/home +endif +PWD=$(shell pwd) + +ifeq (${FIPS_ENABLED}, true) +GOFLAGS_MOD+=-tags=fips_enabled +GOFLAGS_MOD:=$(strip ${GOFLAGS_MOD}) +endif + +GOENV=GOOS=${GOOS} GOARCH=${GOARCH} CGO_ENABLED=0 GOFLAGS="${GOFLAGS_MOD}" GOBUILDFLAGS=-gcflags="all=-trimpath=${GOPATH}" -asmflags="all=-trimpath=${GOPATH}" @@ -80,6 +118,7 @@ ALLOW_DIRTY_CHECKOUT?=false # TODO: Figure out how to discover this dynamically CONVENTION_DIR := boilerplate/openshift/golang-osd-operator +BOILERPLATE_CONTAINER_MAKE := boilerplate/_lib/container-make # Set the default goal in a way that works for older & newer versions of `make`: # Older versions (<=3.8.0) will pay attention to the `default` target. @@ -102,20 +141,20 @@ isclean: docker-build-push-one: isclean docker-login @(if [[ -z "${IMAGE_URI}" ]]; then echo "Must specify IMAGE_URI"; exit 1; fi) @(if [[ -z "${DOCKERFILE_PATH}" ]]; then echo "Must specify DOCKERFILE_PATH"; exit 1; fi) - ${CONTAINER_ENGINE} build . -f $(DOCKERFILE_PATH) -t $(IMAGE_URI) - ${CONTAINER_ENGINE} --config=${CONTAINER_ENGINE_CONFIG_DIR} push ${IMAGE_URI} + ${CONTAINER_ENGINE} build --pull -f $(DOCKERFILE_PATH) -t $(IMAGE_URI) . + ${CONTAINER_ENGINE} push ${IMAGE_URI} # TODO: Get rid of docker-build. It's only used by opm-build-push .PHONY: docker-build docker-build: isclean - ${CONTAINER_ENGINE} build . -f $(OPERATOR_DOCKERFILE) -t $(OPERATOR_IMAGE_URI) + ${CONTAINER_ENGINE} build --pull -f $(OPERATOR_DOCKERFILE) -t $(OPERATOR_IMAGE_URI) . ${CONTAINER_ENGINE} tag $(OPERATOR_IMAGE_URI) $(OPERATOR_IMAGE_URI_LATEST) # TODO: Get rid of docker-push. It's only used by opm-build-push .PHONY: docker-push docker-push: docker-login docker-build - ${CONTAINER_ENGINE} --config=${CONTAINER_ENGINE_CONFIG_DIR} push ${OPERATOR_IMAGE_URI} - ${CONTAINER_ENGINE} --config=${CONTAINER_ENGINE_CONFIG_DIR} push ${OPERATOR_IMAGE_URI_LATEST} + ${CONTAINER_ENGINE} push ${OPERATOR_IMAGE_URI} + ${CONTAINER_ENGINE} push ${OPERATOR_IMAGE_URI_LATEST} # TODO: Get rid of push. It's not used. .PHONY: push @@ -125,7 +164,7 @@ push: docker-push docker-login: @test "${REGISTRY_USER}" != "" && test "${REGISTRY_TOKEN}" != "" || (echo "REGISTRY_USER and REGISTRY_TOKEN must be defined" && exit 1) mkdir -p ${CONTAINER_ENGINE_CONFIG_DIR} - @${CONTAINER_ENGINE} --config=${CONTAINER_ENGINE_CONFIG_DIR} login -u="${REGISTRY_USER}" -p="${REGISTRY_TOKEN}" quay.io + @${CONTAINER_ENGINE} login -u="${REGISTRY_USER}" -p="${REGISTRY_TOKEN}" quay.io .PHONY: go-check go-check: ## Golang linting and other static analysis @@ -138,38 +177,87 @@ go-generate: ${GOENV} go generate $(TESTTARGETS) # Don't forget to commit generated files +# go-get-tool will 'go install' any package $2 and install it to $1. +define go-get-tool +@{ \ +set -e ;\ +TMP_DIR=$$(mktemp -d) ;\ +cd $$TMP_DIR ;\ +go mod init tmp ;\ +echo "Downloading $(2)" ;\ +GOBIN=$(shell dirname $(1)) go install $(2) ;\ +echo "Installed in $(1)" ;\ +rm -rf $$TMP_DIR ;\ +} +endef + +# Deciding on the binary versions +CONTROLLER_GEN_VERSION = v0.8.0 +CONTROLLER_GEN = controller-gen-$(CONTROLLER_GEN_VERSION) + +OPENAPI_GEN_VERSION = v0.23.0 +OPENAPI_GEN = openapi-gen-$(OPENAPI_GEN_VERSION) + +ifeq ($(USE_OLD_SDK), TRUE) +#If we are using the old osdk, we use the default controller-gen and openapi-gen versions. +# Default version is 0.3.0 for now. +CONTROLLER_GEN = controller-gen +# Default version is 0.19.4 for now. +OPENAPI_GEN = openapi-gen +endif + .PHONY: op-generate +## CRD v1beta1 is no longer supported. op-generate: - ${CONVENTION_DIR}/operator-sdk-generate.sh - # HACK: Due to an OLM bug in 3.11, we need to remove the - # spec.validation.openAPIV3Schema.type from CRDs. Remove once - # 3.11 is no longer supported. - find deploy/ -name '*_crd.yaml' | xargs -n1 -I{} yq d -i {} spec.validation.openAPIV3Schema.type - # Don't forget to commit generated files + cd $(API_DIR); $(CONTROLLER_GEN) crd:crdVersions=v1 paths=./... output:dir=$(PWD)/deploy/crds + cd $(API_DIR); $(CONTROLLER_GEN) object paths=./... + +API_DIR_MIN_DEPTH = 1 +ifeq ($(USE_OLD_SDK), TRUE) +API_DIR_MIN_DEPTH = 2 +endif .PHONY: openapi-generate openapi-generate: - find ./pkg/apis/ -maxdepth 2 -mindepth 2 -type d | xargs -t -n1 -I% \ - openapi-gen --logtostderr=true \ + find $(API_DIR) -maxdepth 2 -mindepth $(API_DIR_MIN_DEPTH) -type d | xargs -t -I% \ + $(OPENAPI_GEN) --logtostderr=true \ -i % \ -o "" \ -O zz_generated.openapi \ -p % \ -h /dev/null \ -r "-" - + .PHONY: generate generate: op-generate go-generate openapi-generate +ifeq (${FIPS_ENABLED}, true) +go-build: ensure-fips +endif + .PHONY: go-build go-build: ## Build binary # Force GOOS=linux as we may want to build containers in other *nix-like systems (ie darwin). # This is temporary until a better container build method is developed ${GOENV} GOOS=linux go build ${GOBUILDFLAGS} -o ${BINFILE} ${MAINPACKAGE} +# ENVTEST_K8S_VERSION refers to the version of kubebuilder assets to be downloaded by envtest binary. +ENVTEST_K8S_VERSION = 1.23 +SETUP_ENVTEST = setup-envtest + +.PHONY: setup-envtest +setup-envtest: + $(eval KUBEBUILDER_ASSETS := "$(shell $(SETUP_ENVTEST) use $(ENVTEST_K8S_VERSION) -p path --bin-dir /tmp/envtest/bin)") + +# Setting SHELL to bash allows bash commands to be executed by recipes. +# This is a requirement for 'setup-envtest.sh' in the test target. +# Options are set to exit when a recipe line exits non-zero or a piped command fails. +SHELL = /usr/bin/env bash -o pipefail +.SHELLFLAGS = -ec + .PHONY: go-test -go-test: - ${GOENV} go test $(TESTOPTS) $(TESTTARGETS) +go-test: setup-envtest + KUBEBUILDER_ASSETS=$(KUBEBUILDER_ASSETS) go test $(TESTOPTS) $(TESTTARGETS) .PHONY: python-venv python-venv: @@ -195,10 +283,6 @@ olm-deploy-yaml-validate: python-venv prow-config: ${CONVENTION_DIR}/prow-config ${RELEASE_CLONE} -.PHONY: codecov-secret-mapping -codecov-secret-mapping: - ${CONVENTION_DIR}/codecov-secret-mapping ${RELEASE_CLONE} - ###################### # Targets used by prow @@ -246,3 +330,45 @@ opm-build-push: docker-push OPERATOR_IMAGE_TAG="${OPERATOR_IMAGE_TAG}" \ OLM_CHANNEL="${OLM_CHANNEL}" \ ${CONVENTION_DIR}/build-opm-catalog.sh + +.PHONY: ensure-fips +ensure-fips: + ${CONVENTION_DIR}/configure-fips.sh + +# You will need to export the forked/cloned operator repository directory as OLD_SDK_REPO_DIR to make this work. +# Example: export OLD_SDK_REPO_DIR=~/Projects/My-Operator-Fork +.PHONY: migrate-to-osdk1 +migrate-to-osdk1: +ifndef OLD_SDK_REPO_DIR + $(error OLD_SDK_REPO_DIR is not set) +endif + # Copying files & folders from old repository to current project + rm -rf config + rsync -a $(OLD_SDK_REPO_DIR)/deploy . --exclude=crds + rsync -a $(OLD_SDK_REPO_DIR)/pkg . --exclude={'apis','controller'} + rsync -a $(OLD_SDK_REPO_DIR)/Makefile . + rsync -a $(OLD_SDK_REPO_DIR)/.gitignore . + rsync -a $(OLD_SDK_REPO_DIR)/ . --exclude={'cmd','version','boilerplate','deploy','pkg'} --ignore-existing + +# Boilerplate container-make targets. +# Runs 'make' in the boilerplate backing container. +# If the command fails, starts a shell in the container so you can debug. +.PHONY: container-test +container-test: + ${BOILERPLATE_CONTAINER_MAKE} test + +.PHONY: container-generate +container-generate: + ${BOILERPLATE_CONTAINER_MAKE} generate + +.PHONY: container-lint +container-lint: + ${BOILERPLATE_CONTAINER_MAKE} lint + +.PHONY: container-validate +container-validate: + ${BOILERPLATE_CONTAINER_MAKE} validate + +.PHONY: container-coverage +container-coverage: + ${BOILERPLATE_CONTAINER_MAKE} coverage \ No newline at end of file diff --git a/boilerplate/openshift/golang-osd-operator/update b/boilerplate/openshift/golang-osd-operator/update index 5db57dd9..bed4cc8d 100755 --- a/boilerplate/openshift/golang-osd-operator/update +++ b/boilerplate/openshift/golang-osd-operator/update @@ -14,8 +14,9 @@ echo "Copying .codecov.yml to your repository root." cp ${HERE}/.codecov.yml $REPO_ROOT # TODO: boilerplate more of Dockerfile -echo "Overwriting build/Dockerfile's initial FROM with $IMAGE_PULL_PATH" -${SED?} -i "1s,.*,FROM $IMAGE_PULL_PATH AS builder," build/Dockerfile +DOCKERFILE=build/Dockerfile +echo "Overwriting $DOCKERFILE's initial FROM with $IMAGE_PULL_PATH" +${SED?} -i "1s,.*,FROM $IMAGE_PULL_PATH AS builder," $DOCKERFILE echo "Writing .ci-operator.yaml in your repository root with:" echo " namespace: $IMAGE_NAMESPACE" diff --git a/build/Dockerfile b/build/Dockerfile index 29d2c702..56924fad 100644 --- a/build/Dockerfile +++ b/build/Dockerfile @@ -1,4 +1,4 @@ -FROM quay.io/app-sre/boilerplate:image-v1.0.0 AS builder +FROM quay.io/app-sre/boilerplate:image-v2.3.2 AS builder RUN mkdir -p /workdir COPY . /workdir