Skip to content

Apantuso/revert to working build script #207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .ci-operator.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
build_root_image:
name: boilerplate
namespace: openshift
tag: image-v2.3.2
tag: image-v1.0.0
4 changes: 0 additions & 4 deletions .codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,3 @@ comment:
layout: "reach,diff,flags,tree"
behavior: default
require_changes: no

ignore:
- "**/mocks"
- "**/zz_generated*.go"
2 changes: 1 addition & 1 deletion boilerplate/_data/backing-image-tag
Original file line number Diff line number Diff line change
@@ -1 +1 @@
image-v2.3.2
image-v1.0.0
2 changes: 1 addition & 1 deletion boilerplate/_data/last-boilerplate-commit
Original file line number Diff line number Diff line change
@@ -1 +1 @@
a6c27570590858412f03ce4b84e7d6bb1ecfdc7a
b0a20637c47d7d94d6bdadbb2660b4f081526015
15 changes: 6 additions & 9 deletions boilerplate/_lib/container-make
Original file line number Diff line number Diff line change
Expand Up @@ -9,26 +9,23 @@ fi

source ${0%/*}/common.sh

CONTAINER_ENGINE="${CONTAINER_ENGINE:-$(command -v podman || command -v docker)}"
CONTAINER_ENGINE=$(command -v podman || command -v docker)
[[ -n "$CONTAINER_ENGINE" ]] || err "Couldn't find a container engine. Are you already in a container?"

CONTAINER_ENGINE_SHORT=${CONTAINER_ENGINE##*/}

# Make sure the mount inside the container is named in such a way that
# - openapi-gen (which relies on GOPATH) produces absolute paths; and
# - other go-ish paths are writeable, e.g. for `go mod download`.
CONTAINER_MOUNT=/go/src/$(repo_import $REPO_ROOT)

# First set up a detached container with the repo mounted.
banner "Starting the container"
if [[ "${CONTAINER_ENGINE##*/}" == "podman" ]]; then
if [[ $OSTYPE == *"darwin"* ]]; then
CE_OPTS="--userns keep-id -v $REPO_ROOT:$CONTAINER_MOUNT"
else
CE_OPTS="--userns keep-id -v $REPO_ROOT:$CONTAINER_MOUNT:Z"
fi
if [[ $CONTAINER_ENGINE_SHORT == "podman" ]]; then
container_id=$($CONTAINER_ENGINE run --userns keep-id -d -v "$REPO_ROOT":"$CONTAINER_MOUNT":Z $IMAGE_PULL_PATH tail -f /dev/null)
else
CE_OPTS="-v $REPO_ROOT:$CONTAINER_MOUNT"
container_id=$($CONTAINER_ENGINE run -d -v "$REPO_ROOT":"$CONTAINER_MOUNT" $IMAGE_PULL_PATH tail -f /dev/null)
fi
container_id=$($CONTAINER_ENGINE run -d ${CE_OPTS} $IMAGE_PULL_PATH sleep infinity)

if [[ $? -ne 0 ]] || [[ -z "$container_id" ]]; then
err "Couldn't start detached container"
Expand Down
1 change: 1 addition & 0 deletions boilerplate/_lib/subscriber-propose
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ declare -A SUBCOMMANDS
SUBCOMMANDS=(
# TODO:
# [bootstrap]='Bootstrap a new subscriber'
# [codecov-secret-mapping]='Propose codecov secret mapping to openshift/release'
# [prow-config]='Propose standardized prow configuration to openshift/release'
[update]='Update an already-onboarded subscriber'
)
Expand Down
4 changes: 0 additions & 4 deletions boilerplate/openshift/golang-osd-operator/.codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,3 @@ comment:
layout: "reach,diff,flags,tree"
behavior: default
require_changes: no

ignore:
- "**/mocks"
- "**/zz_generated*.go"
61 changes: 21 additions & 40 deletions boilerplate/openshift/golang-osd-operator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ This convention is suitable for both cluster- and hive-deployed operators.
The following components are included:

## `make` targets and functions.

**Note:** Your repository's main `Makefile` needs to be edited to include the
"nexus makefile include":

Expand All @@ -29,7 +28,7 @@ following:
### Prow

| Test name / `make` target | Purpose |
| ------------------------- | --------------------------------------------------------------------------------------------------------------- |
|---------------------------|-----------------------------------------------------------------------------------------------------------------|
| `validate` | Ensure code generation has not been forgotten; and ensure generated and boilerplate code has not been modified. |
| `lint` | Perform static analysis. |
| `test` | "Local" unit and functional testing. |
Expand All @@ -49,26 +48,18 @@ $ make RELEASE_CLONE=/home/me/github/openshift/release prow-config
```

This will generate a delta configuring prow to:

- Build your `build/Dockerfile`.
- Run the above targets in presubmit tests.
- Run the `coverage` target in a postsubmit. This is the step that
updates your coverage report in codecov.io.

#### Local Testing

You can run these `make` targets locally during development to test your
code changes. However, differences in platforms and environments may
lead to unpredictable results. Therefore boilerplate provides a utility
to run targets in a container environment that is designed to be as
similar as possible to CI:

```shell
$ make container-{target}
```

or

```shell
$ ./boilerplate/_lib/container-make {target}
```
Expand All @@ -81,15 +72,27 @@ By default it is configured to be run from the app-sre jenkins pipelines.
Consult [this doc](app-sre.md) for information on local execution/testing.

## Code coverage

- A `codecov.sh` script, referenced by the `coverage` `make` target, to
run code coverage analysis per [this SOP](https://github.com/openshift/ops-sop/blob/93d100347746ce04ad552591136818f82043c648/services/codecov.md).
run code coverage analysis per [this SOP](https://github.com/openshift/ops-sop/blob/93d100347746ce04ad552591136818f82043c648/services/codecov.md).

- A `.codecov.yml` configuration file for
[codecov.io](https://docs.codecov.io/docs/codecov-yaml). Note that
this is copied into the repository root, because that's
[where codecov.io expects it](https://docs.codecov.io/docs/codecov-yaml#can-i-name-the-file-codecovyml).

- A `make` target to [request the secret mapping in openshift/release](https://github.com/openshift/ops-sop/blob/be43125239deb1f2bbc1ef54f010410e97ff6146/services/codecov.md#openshiftrelease-pr-1---secret-mapping):

```shell
$ make codecov-secret-mapping
```

If you already have the openshift/release repository cloned locally, you
may specify its path via `$RELEASE_CLONE`:

```shell
$ make RELEASE_CLONE=/home/me/github/openshift/release codecov-secret-mapping
```

## Linting and other static analysis with `golangci-lint`

- A `go-check` `make` target, which
Expand All @@ -104,35 +107,13 @@ The convention embeds default checks to ensure generated code generation is curr
To trigger the check, you can use `make generate-check` provided your Makefile properly includes the boilerplate-generated include `boilerplate/generated-includes.mk`.

Checks consist of:

- Checking all files are committed to ensure a safe point to revert to in case of error
- Running the `make generate` command (see below) to regenerate the needed code
- Checking if this results in any new uncommitted files in the git project or if all is clean.
* Checking all files are committed to ensure a safe point to revert to in case of error
* Running the `make generate` command (see below) to regenerate the needed code
* Checking if this results in any new uncommitted files in the git project or if all is clean.

`make generate` does the following:

- generate crds and deepcopy via controller-gen. This is a no-op if your
* `operator-sdk generate crds` and `k8s`. This is a no-op if your
operator has no APIs.
- `openapi-gen`. This is a no-op if your operator has no APIs.
- `go generate`. This is a no-op if you have no `//go:generate`
* `openapi-gen`. This is a no-op if your operator has no APIs.
* `go generate`. This is a no-op if you have no `//go:generate`
directives in your code.

## FIPS (Federal Information Processing Standards)

To enable FIPS in your build there is a `make ensure-fips` target.

Add `FIPS_ENABLED=true` to your repos Makefile. Please ensure that this variable is added **before** including boilerplate Makefiles.

e.g.

```.mk
FIPS_ENABLED=true

include boilerplate/generated-includes.mk
```

`ensure-fips` will add a [fips.go](./fips.go) file in the same directory as the `main.go` file. (Please commit this file as normal)

`fips.go` will import the necessary packages to restrict all TLS configuration to FIPS-approved settings.

With `FIPS_ENABLED=true`, `ensure-fips` is always run before `make go-build`
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,6 @@ for channel in staging production; do
echo "properly. Nothing to do!"
else
# build the CSV and create & push image catalog for the appropriate channel
make ${channel}-csv-build ${channel}-catalog-build ${channel}-catalog-publish
make ${channel}-common-csv-build ${channel}-catalog-build ${channel}-catalog-publish
fi
done
11 changes: 0 additions & 11 deletions boilerplate/openshift/golang-osd-operator/app-sre.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,17 +21,6 @@ If not, you will need to set the `IMAGE_REGISTRY` environment variable (see [bel
The SaaS bundle repository for `$OPERATOR_NAME` should be located at `https://gitlab.cee.redhat.com/service/saas-{operator}-bundle`, e.g. https://gitlab.cee.redhat.com/service/saas-deadmanssnitch-operator-bundle.
Fork it to your personal namespace.

If you have already forked it to your personal namespace and/or used your fork for testing app-sre scripts at some time in the past, it is recommended that you bring your fork in sync with how upstream appears, or else the catalog you test with may not work correctly when deployed.

An example of how to do this for the `staging` branch is below (`production` steps are the same):

```
git checkout staging
git pull upstream staging
git reset --hard upstream/staging
git push origin staging --force
```

## Set environment variables
```bash
# The process creates artifacts in your git clone. Some of the make targets
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -320,14 +320,6 @@ function main() {
local versions
# shellcheck disable=SC2207
versions=($(get_prev_operator_version "$bundle_versions_file"))
# This condition is triggered when an operator is built for the first time. In such case the
# get_prev_operator_version returns an empty string and causes undefined variables failures
# in a few lines below.
if [ -z ${versions+x} ]
then
versions[0]=""
versions[1]=""
fi
local prev_operator_version="${versions[0]}"
local prev_good_operator_version="${versions[1]}"
local skip_versions=("${versions[@]:2}")
Expand Down
62 changes: 62 additions & 0 deletions boilerplate/openshift/golang-osd-operator/codecov-secret-mapping
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
#!/usr/bin/env bash

set -e

REPO_ROOT=$(git rev-parse --show-toplevel)
source $REPO_ROOT/boilerplate/_lib/common.sh
source $REPO_ROOT/boilerplate/_lib/release.sh

cmd=${0##*/}

usage() {
cat <<EOF
Usage: $cmd [PATH_TO_RELEASE_CLONE]

Creates a delta in $RELEASE_REPO requesting a codecov secret mapping for a
boilerplate consumer. Must be invoked from within a local clone of a repository
already subscribed to the $CONVENTION_NAME convention.

Parameters:
PATH_TO_RELEASE_CLONE File system path to a local clone of
https://github.com/$RELEASE_REPO. If not
specified, the repository will be cloned in a
temporary directory.
EOF
exit -1
}

# Was a release repo clone specified?
release_process_args "$@"

release_validate_invocation

release_prep_clone

cd $RELEASE_CLONE

mapping_file=core-services/secret-mirroring/_mapping.yaml
secret_name=$CONSUMER_NAME-codecov-token

# TODO: Do some proper yaml validation and editing here.

# See if the mapping already exists. This ain't great: it assumes the
# lines are in a particular order, and doesn't check the namespaces. See
# TODO above :)
grep -B2 $secret_name $mapping_file && err "
Found existing mapping in $mapping_file
Nothing to do."

# Append it. This ain't great: it assumes the `secrets` top-level key is
# still in play, that we're matching whitespace, etc. See TODO above :)
cat <<EOF >> $mapping_file
- from:
namespace: sd-sre-secrets
name: $secret_name
to:
namespace: ci
name: $secret_name
EOF

release_branch=$CONSUMER_ORG-$CONSUMER_NAME-$DEFAULT_BRANCH-boilerplate-$cmd

release_done_msg $release_branch
18 changes: 0 additions & 18 deletions boilerplate/openshift/golang-osd-operator/configure-fips.sh

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,6 @@ while getopts "o:c:r:" option; do
esac
done

# Detect the container engine to use, allowing override from the env
CONTAINER_ENGINE=${CONTAINER_ENGINE:-$(command -v podman || command -v docker || true)}
if [[ -z "$CONTAINER_ENGINE" ]]; then
echo "WARNING: Couldn't find a container engine! Defaulting to docker."
CONTAINER_ENGINE=docker
fi

# Checking parameters
check_mandatory_params operator_channel operator_name

Expand Down Expand Up @@ -63,29 +56,13 @@ EOF

# Build registry
cat <<EOF > $DOCKERFILE_REGISTRY
FROM quay.io/openshift/origin-operator-registry:4.10.0 AS builder
FROM quay.io/openshift/origin-operator-registry:4.8.0
COPY $SAAS_OPERATOR_DIR manifests
RUN initializer --permissive

FROM registry.access.redhat.com/ubi8/ubi-micro:8.6-484

COPY --from=builder /bin/registry-server /bin/registry-server
COPY --from=builder /bin/grpc_health_probe /bin/grpc_health_probe
COPY --from=builder /bin/initializer /bin/initializer

WORKDIR /registry
RUN chgrp -R 0 /registry && chmod -R g+rwx /registry

USER 1001

COPY --from=builder /registry /registry

EXPOSE 50051

CMD ["registry-server", "-t", "/tmp/terminate.log"]
EOF

${CONTAINER_ENGINE} build --pull -f $DOCKERFILE_REGISTRY --tag "${registry_image}:${operator_channel}-latest" .
docker build -f $DOCKERFILE_REGISTRY --tag "${registry_image}:${operator_channel}-latest" .

if [ $? -ne 0 ] ; then
echo "docker build failed, exiting..."
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,17 +41,6 @@ BUNDLE_DIR="${SAAS_OPERATOR_DIR}/${operator_name}"
OPERATOR_NEW_VERSION=$(ls "${BUNDLE_DIR}" | sort -t . -k 3 -g | tail -n 1)
OPERATOR_PREV_VERSION=$(ls "${BUNDLE_DIR}" | sort -t . -k 3 -g | tail -n 2 | head -n 1)

# Get container engine
CONTAINER_ENGINE=$(command -v podman || command -v docker || true)
[[ -n "$CONTAINER_ENGINE" ]] || echo "WARNING: Couldn't find a container engine. Assuming you already in a container, running unit tests." >&2

# Set SRC container transport based on container engine
if [[ "${CONTAINER_ENGINE##*/}" == "podman" ]]; then
SRC_CONTAINER_TRANSPORT="containers-storage"
else
SRC_CONTAINER_TRANSPORT="docker-daemon"
fi

# Checking SAAS_OPERATOR_DIR exist
if [ ! -d "${SAAS_OPERATOR_DIR}/.git" ] ; then
echo "${SAAS_OPERATOR_DIR} should exist and be a git repository"
Expand Down Expand Up @@ -96,7 +85,7 @@ popd
if [ "$push_catalog" = true ] ; then
# push image
skopeo copy --dest-creds "${QUAY_USER}:${QUAY_TOKEN}" \
"${SRC_CONTAINER_TRANSPORT}:${registry_image}:${operator_channel}-latest" \
"docker-daemon:${registry_image}:${operator_channel}-latest" \
"docker://${registry_image}:${operator_channel}-latest"

if [ $? -ne 0 ] ; then
Expand All @@ -105,7 +94,7 @@ if [ "$push_catalog" = true ] ; then
fi

skopeo copy --dest-creds "${QUAY_USER}:${QUAY_TOKEN}" \
"${SRC_CONTAINER_TRANSPORT}:${registry_image}:${operator_channel}-latest" \
"docker-daemon:${registry_image}:${operator_channel}-latest" \
"docker://${registry_image}:${operator_channel}-${operator_commit_hash}"

if [ $? -ne 0 ] ; then
Expand Down
Loading