[BOP-177] Implement e2e test framework for BOP #21

sakshisharma84 · 2023-12-05T19:07:23Z

JIRA Ticket: https://mirantis.jira.com/browse/BOP-177

This PR adds support for the following.

Adds a new GitHub action to run e2e tests.
A kind cluster is deployed to run the tests.
A simple e2e test that installs the boundless operator and validates the deployment.
A separate e2e test workflow make e2e in Makefile. Segregated from make test

We can add more feature specific tests on top of this.

Output

go test -v ./test/e2e/bopinstall


=== RUN   TestBOPInstall
=== RUN   TestBOPInstall/Boundless_operator
=== RUN   TestBOPInstall/Boundless_operator/Install_BOP
    bop_install_test.go:43: Installing boundless operator from the manifest located at:/Users/ssharma/boundless-operator/deploy/static
=== RUN   TestBOPInstall/Boundless_operator/Check_BOP_deployment_available
    bop_install_test.go:60: Waiting for deployment : boundless-operator-controller-manager to be ready.
    bop_install_test.go:66: Deployment boundless-operator-controller-manager is available. ReadyReplicas: 1
--- PASS: TestBOPInstall (15.12s)
    --- PASS: TestBOPInstall/Boundless_operator (15.12s)
        --- PASS: TestBOPInstall/Boundless_operator/Install_BOP (0.10s)
        --- PASS: TestBOPInstall/Boundless_operator/Check_BOP_deployment_available (15.02s)
PASS
ok  	github.com/mirantis/boundless-operator/test/e2e/bopinstall	51.221s

make e2e

test -s /Users/ssharma/boundless-operator/bin/controller-gen && /Users/ssharma/boundless-operator/bin/controller-gen --version | grep -q v0.11.1 || \
	GOBIN=/Users/ssharma/boundless-operator/bin go install sigs.k8s.io/controller-tools/cmd/[email protected]
/Users/ssharma/boundless-operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/Users/ssharma/boundless-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
KUBEBUILDER_ASSETS="/Users/ssharma/boundless-operator/bin/k8s/1.26.0-darwin-arm64" go test -v ./test/e2e/... -coverprofile cover.out
=== RUN   TestBOPInstall
=== RUN   TestBOPInstall/Boundless_operator
=== RUN   TestBOPInstall/Boundless_operator/Install_BOP
    bop_install_test.go:43: Installing boundless operator from the manifest located at:/Users/ssharma/boundless-operator/deploy/static
=== RUN   TestBOPInstall/Boundless_operator/Check_BOP_deployment_available
    bop_install_test.go:60: Waiting for deployment : boundless-operator-controller-manager to be ready.
    bop_install_test.go:66: Deployment boundless-operator-controller-manager is available. ReadyReplicas: 1
--- PASS: TestBOPInstall (15.10s)
    --- PASS: TestBOPInstall/Boundless_operator (15.10s)
        --- PASS: TestBOPInstall/Boundless_operator/Install_BOP (0.08s)
        --- PASS: TestBOPInstall/Boundless_operator/Check_BOP_deployment_available (15.02s)
PASS
coverage: [no statements]
ok  	github.com/mirantis/boundless-operator/test/e2e/bopinstall	50.284s	coverage: [no statements]

ranyodh · 2023-12-06T20:40:00Z

Latest Operator-sdk(v0.16.3) is NOT compatible with k8s.io/kubernetes v1.28.4 package. After the package is upgraded to v0.16.3, we get the following error in main.go

vet: ./main.go:59:3: invalid field name Metrics.BindAddress in struct literal
vet: ./main.g
o:63:3: unknown field Port in struct literal of type manager.Options

This can be fixed. See this (temp) PR here: https://github.com/Mirantis/boundless-operator/pull/23/files
This should enable you to import v1.28.4

.github/development.env

.github/workflows/PR.yml

.github/workflows/e2e.yml

test/suite_test.go

test/bop_install.go

sakshisharma84 · 2023-12-06T20:53:40Z

https://github.com/Mirantis/boundless-operator/pull/23/files

@ranyodh : Thanks for sharing the PR. I already tried the solution for Metrics.BindAddress. What I was not sure about was the parameterPort i.e. how it is consumed and what will happen if I delete it.

MetricsBindAddress:     metricsAddr,
Port:                   9443,

I see that the PR deletes it. If that is okay, I can upgrade it w/o issues.

vikramhh · 2023-12-07T12:13:39Z

.github/workflows/Release.yml

@@ -15,5 +15,5 @@ jobs:
    uses: ./.github/workflows/build.yml
  push-to-ghcr:
    if: ${{ always() && contains(join(needs.*.result, ','), 'success') }} # if all `needs` jobs are successful
-    needs: [vet, unit-test, build]
+    needs: [vet, unit-test, e2e, build]


@sakshisharma84 @nwneisen - how does a manual override work with git actions? There may be times when we need to allow certain actions even when certain conditions could not be met(e.g. push to ghcr even though not all e2e tests passed) - without disabling checks totally. Would it work to make needs to be input actions so that these could be edited and the whole workflow re-triggered after removing whatever needs to be (temporarily) overridden.

Asking for long-term, should not hold this PR.

@vikramhh workflow_dispatch can be used to manually trigger an action but that runs the main branch. It doesn't look like there is a way to do it from a PR so we would have to get creative.

nwneisen

We should also add to a readme somewhere explaining the different kinds of tests and how to run them. I'd prefer if the root README.me has a test section but adding a README.me to the tests/ folder would also be fine.

Makefile

.github/workflows/PR.yml

…into e2e-test

sakshisharma84 · 2023-12-07T17:56:53Z

We should also add to a readme somewhere explaining the different kinds of tests and how to run them. I'd prefer if the root README.me has a test section but adding a README.me to the tests/ folder would also be fine.

Done!!

nwneisen

LGTM

ranyodh

Just a small suggestion. LGTM otherwise.

test/e2e/bopinstall/main_test.go

sakshisharma84 added 6 commits December 5, 2023 14:06

e2e tests

6b51838

add e2e job in need

bf97bc7

add e2e in PR.yml

2d41d72

fix yaml file path

2e5e9d5

separate e2e tests from unit tests

5a8d363

remove e2e from docker-build

21716ef

sakshisharma84 changed the title ~~<WIP> e2e tests~~ [BOP-177] Implement e2e test framework for BOP Dec 6, 2023

sakshisharma84 marked this pull request as ready for review December 6, 2023 19:22

sakshisharma84 requested review from ranyodh, nwneisen and tppolkow December 6, 2023 19:23

nwneisen reviewed Dec 6, 2023

View reviewed changes

.github/development.env Outdated Show resolved Hide resolved

.github/workflows/PR.yml Show resolved Hide resolved

.github/workflows/e2e.yml Outdated Show resolved Hide resolved

test/suite_test.go Outdated Show resolved Hide resolved

ranyodh reviewed Dec 6, 2023

View reviewed changes

test/bop_install.go Outdated Show resolved Hide resolved

ranyodh reviewed Dec 6, 2023

View reviewed changes

test/bop_install.go Outdated Show resolved Hide resolved

ranyodh reviewed Dec 6, 2023

View reviewed changes

test/bop_install.go Outdated Show resolved Hide resolved

sakshisharma84 added 2 commits December 6, 2023 16:04

bump kubernetes and controller-runtime

11044da

address review comments

596bc76

vikramhh reviewed Dec 7, 2023

View reviewed changes

nwneisen reviewed Dec 7, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved

.github/workflows/PR.yml Show resolved Hide resolved

sakshisharma84 added 2 commits December 7, 2023 10:23

Merge branch 'main' of https://github.com/Mirantis/boundless-operator …

cd04262

…into e2e-test

use kubernetes-sigs/e2e-framework

8b94aaa

nwneisen approved these changes Dec 7, 2023

View reviewed changes

ranyodh approved these changes Dec 7, 2023

View reviewed changes

test/e2e/bopinstall/main_test.go Show resolved Hide resolved

sakshisharma84 merged commit 9799a35 into main Dec 7, 2023

ranyodh deleted the e2e-test branch February 9, 2024 19:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[BOP-177] Implement e2e test framework for BOP #21

[BOP-177] Implement e2e test framework for BOP #21

Uh oh!

sakshisharma84 commented Dec 5, 2023 •

edited

Loading

Uh oh!

ranyodh commented Dec 6, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sakshisharma84 commented Dec 6, 2023

Uh oh!

vikramhh Dec 7, 2023 •

edited

Loading

Uh oh!

nwneisen Dec 7, 2023

Uh oh!

nwneisen left a comment

Uh oh!

Uh oh!

Uh oh!

sakshisharma84 commented Dec 7, 2023

Uh oh!

nwneisen left a comment

Uh oh!

ranyodh left a comment

Uh oh!

Uh oh!

Uh oh!

[BOP-177] Implement e2e test framework for BOP #21

[BOP-177] Implement e2e test framework for BOP #21

Uh oh!

Conversation

sakshisharma84 commented Dec 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ranyodh commented Dec 6, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sakshisharma84 commented Dec 6, 2023

Uh oh!

vikramhh Dec 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nwneisen Dec 7, 2023

Choose a reason for hiding this comment

Uh oh!

nwneisen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sakshisharma84 commented Dec 7, 2023

Uh oh!

nwneisen left a comment

Choose a reason for hiding this comment

Uh oh!

ranyodh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sakshisharma84 commented Dec 5, 2023 •

edited

Loading

vikramhh Dec 7, 2023 •

edited

Loading