Skip to content

Commit 09c36f1

Browse files
NickLarsenNZNickLarsenNZ
authored
test(kuttl): Install cert-manager (#505)
* test(kuttl): try to install cert-manager and remove it if installed. Supports concurrent test runs. * chore(kuttl): set script width to 80 characters for easier diffing in future * chore(kuttl): improve output * chore: fix lint * chore: update changelog * chore(kuttl): fix typo
1 parent bd94fdb commit 09c36f1

File tree

3 files changed

+176
-4
lines changed

3 files changed

+176
-4
lines changed

‎CHANGELOG.md

+5-4
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ All notable changes to this project will be documented in this file.
1313

1414
- Fixed Kerberos keytab provisioning reusing its credential cache ([#490]).
1515
- Fixed listener volumes missing a required permission to inspect manually provisioned listeners ([#497]).
16+
- test: Fixed cert-manager tests by installing cert-manager if it doesn't exist ([#505]).
1617

1718
### Changed
1819

@@ -23,6 +24,7 @@ All notable changes to this project will be documented in this file.
2324
[#490]: https://github.com/stackabletech/secret-operator/pull/490
2425
[#495]: https://github.com/stackabletech/secret-operator/pull/495
2526
[#497]: https://github.com/stackabletech/secret-operator/pull/497
27+
[#505]: https://github.com/stackabletech/secret-operator/pull/505
2628

2729
## [24.7.0] - 2024-07-24
2830

@@ -35,9 +37,9 @@ All notable changes to this project will be documented in this file.
3537

3638
- [BREAKING] The TLS CA Secret is now installed into the Namespace of the operator (typically `stackable-operators`), rather than `default` ([#397]).
3739
- Existing users can either migrate by either:
38-
- (Recommended) Copying the CA into the new location
39-
(`kubectl -n default get secret/secret-provisioner-tls-ca -o json | jq '.metadata.namespace = "stackable-operators"' | kubectl create -f-`)
40-
- Setting the `secretClasses.tls.caSecretNamespace` Helm flag (`--set secretClasses.tls.caSecretNamespace=default`)
40+
- (Recommended) Copying the CA into the new location
41+
(`kubectl -n default get secret/secret-provisioner-tls-ca -o json | jq '.metadata.namespace = "stackable-operators"' | kubectl create -f-`)
42+
- Setting the `secretClasses.tls.caSecretNamespace` Helm flag (`--set secretClasses.tls.caSecretNamespace=default`)
4143
- Reduce CA default lifetime to one year ([#403])
4244
- Update the image docker.stackable.tech/k8s/sig-storage/csi-provisioner
4345
in the Helm values to v4.0.1 ([#440]).
@@ -80,7 +82,6 @@ All notable changes to this project will be documented in this file.
8082
[#357]: https://github.com/stackabletech/secret-operator/pull/357
8183
[#361]: https://github.com/stackabletech/secret-operator/pull/361
8284

83-
8485
## [23.11.0] - 2023-11-24
8586

8687
### Added

‎tests/templates/kuttl/cert-manager-tls/00-maybe-install-cert-manager.yaml

+90
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
---
2+
apiVersion: kuttl.dev/v1beta1
3+
kind: TestStep
4+
commands:
5+
- script: |
6+
#!/usr/bin/env sh
7+
set -eu
8+
9+
CERT_MANAGER_NAMESPACE="cert-manager-beku"
10+
CERT_MANAGER_CHART_VERSION="v1.15.3"
11+
12+
MARKER_CONFIG_MAP_NAME="beku-install-marker"
13+
MARKER_FINALIZER_NAME="tech.stackable.beku/$NAMESPACE"
14+
15+
MAX_SLEEP_SECONDS=10
16+
RANDOM_SLEEP_SECONDS="$((RANDOM % MAX_SLEEP_SECONDS))"
17+
echo "Sleeping for $RANDOM_SLEEP_SECONDS seconds to reduce the chance " \
18+
"of concurrent cert-manager installations"
19+
sleep "$RANDOM_SLEEP_SECONDS"
20+
21+
# If cert-manager already appears to be installed, or is still installing
22+
# in another concurrent test, then add ourselves as a finalizer so it
23+
# doesn't get deleted while we are using it.
24+
if kubectl --namespace "$CERT_MANAGER_NAMESPACE" get configmap \
25+
"$MARKER_CONFIG_MAP_NAME" 2>/dev/null >/dev/null;
26+
then
27+
echo "Skipping cert-manager install, it appears to have been done or " \
28+
"is in progress."
29+
echo "Adding finalizer ${MARKER_FINALIZER_NAME} to marker ConfigMap" \
30+
"${CERT_MANAGER_NAMESPACE}/${MARKER_CONFIG_MAP_NAME}."
31+
32+
kubectl --namespace "$CERT_MANAGER_NAMESPACE" patch configmap \
33+
"$MARKER_CONFIG_MAP_NAME" --type=json --patch-file=/dev/stdin <<EOF
34+
[{
35+
"op": "add",
36+
"path": "/metadata/finalizers/-",
37+
"value": "$MARKER_FINALIZER_NAME"
38+
}]
39+
EOF
40+
41+
# Now wait until the deployment has finished
42+
while ! helm list --namespace "$CERT_MANAGER_NAMESPACE" \
43+
| grep cert-manager >/dev/null
44+
do
45+
echo "Waiting for another instance to finish installing cert-manager"
46+
sleep 5
47+
done
48+
echo "Finished waiting for another installation of cert-manager"
49+
exit 0
50+
else
51+
# If cert-manager appears to be installed, but we didn't do it, skip
52+
# install
53+
if kubectl get crds -o name | grep 'cert-manager.io'; then
54+
echo "Cert Manager appears to already be installed outside of " \
55+
"testing. Skipping install."
56+
exit 0
57+
fi
58+
fi
59+
60+
# Otherwise, we need to install cert-manager
61+
62+
# Create the namespace, and add finalizer for this test (keyed with
63+
# $NAMESPACE)
64+
kubectl create namespace "$CERT_MANAGER_NAMESPACE"
65+
66+
# Create a marker CM and add ourselves as the first and only finalizer
67+
echo "Creating marker ConfigMap ${CERT_MANAGER_NAMESPACE}/" \
68+
"${MARKER_CONFIG_MAP_NAME}."
69+
kubectl --namespace "$CERT_MANAGER_NAMESPACE" create configmap \
70+
"$MARKER_CONFIG_MAP_NAME"
71+
echo "Adding finalizer ${MARKER_FINALIZER_NAME} to marker ConfigMap" \
72+
"${CERT_MANAGER_NAMESPACE}/${MARKER_CONFIG_MAP_NAME}."
73+
kubectl --namespace "$CERT_MANAGER_NAMESPACE" patch configmap \
74+
"$MARKER_CONFIG_MAP_NAME" --type=json --patch-file=/dev/stdin <<EOF
75+
[{
76+
"op": "add",
77+
"path": "/metadata/finalizers",
78+
"value": ["$MARKER_FINALIZER_NAME"]
79+
}]
80+
EOF
81+
82+
helm repo add jetstack https://charts.jetstack.io --force-update
83+
84+
helm install cert-manager jetstack/cert-manager \
85+
--wait \
86+
--namespace "$CERT_MANAGER_NAMESPACE" \
87+
--version "$CERT_MANAGER_CHART_VERSION" \
88+
--set crds.enabled=true \
89+
--set prometheus.enabled=false
90+
timeout: 120

‎tests/templates/kuttl/cert-manager-tls/99-maybe-uninstall-cert-manager.yaml

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
---
2+
apiVersion: kuttl.dev/v1beta1
3+
kind: TestStep
4+
commands:
5+
- script: |
6+
#!/usr/bin/env sh
7+
set -eu
8+
9+
CERT_MANAGER_NAMESPACE="cert-manager-beku"
10+
11+
MARKER_CONFIG_MAP_NAME="beku-install-marker"
12+
MARKER_FINALIZER_NAME="tech.stackable.beku/$NAMESPACE"
13+
14+
# If a marker CM doesn't exist, skip cleanup
15+
echo "Checking if marker ConfigMap exists"
16+
if ! kubectl --namespace "$CERT_MANAGER_NAMESPACE" get configmap \
17+
"$MARKER_CONFIG_MAP_NAME" 2>/dev/null >/dev/null
18+
then
19+
echo "Cert-manager appears to have been installed outside of testing" \
20+
"Skipping clean up for it"
21+
exit 0
22+
fi
23+
24+
# Otherwise, clean up
25+
26+
# Get our finalizer index, so we can delete it.
27+
echo -n "Getting the index of the finalizer... "
28+
IDX=$(
29+
kubectl --namespace "$CERT_MANAGER_NAMESPACE" get configmap \
30+
"$MARKER_CONFIG_MAP_NAME" --output 'jsonpath={.metadata.finalizers}' \
31+
| jq -re --arg finalizer "$MARKER_FINALIZER_NAME" '
32+
map(. == $finalizer) | index(true)
33+
'
34+
)
35+
echo "$IDX"
36+
37+
# TODO: move delete to here
38+
# Try to delete the CM. If there are other finalizers, then it won't
39+
# delete immediately
40+
echo "Trying to delete the marker ConfigMap"
41+
kubectl --namespace "$CERT_MANAGER_NAMESPACE" delete configmap \
42+
"$MARKER_CONFIG_MAP_NAME" --timeout 1s 2>/dev/null || true
43+
44+
# Check if we are the last finalizer. If we are, then we need to cleanup \
45+
# helm, crds, namespace
46+
FINALIZERS_REMAINING_COUNT=$(
47+
kubectl --namespace cert-manager-beku get configmap \
48+
beku-install-marker --output 'jsonpath={$.metadata.finalizers}' \
49+
| jq length
50+
)
51+
52+
# Remove ourselves as a finalizer, hopefully...
53+
# Unfortunately this is non-atomic because of JSON Patch (RFC 6902)
54+
# limitations where we can only delete by index.
55+
echo "Removing self as a finalizer"
56+
kubectl --namespace "$CERT_MANAGER_NAMESPACE" patch configmap \
57+
"$MARKER_CONFIG_MAP_NAME" --type=json --patch-file=/dev/stdin <<EOF
58+
[{
59+
"op": "remove",
60+
"path": "/metadata/finalizers/$IDX",
61+
}]
62+
EOF
63+
64+
if [ "$FINALIZERS_REMAINING_COUNT" -gt 1 ]; then
65+
# Todo, print other finalizers. Or, maybe we can check if their
66+
# namespaces still exist.
67+
echo "Other instances are using cert-manager. Skipping cleanup"
68+
exit 0
69+
fi
70+
71+
echo "uninstalling helm chart"
72+
helm uninstall cert-manager \
73+
--wait \
74+
--namespace "$CERT_MANAGER_NAMESPACE"
75+
76+
echo "Removing CRDs"
77+
kubectl get crds -o name | grep 'cert-manager.io' | xargs kubectl delete
78+
79+
echo "Deleting cert-manager namespace"
80+
kubectl delete namespace "$CERT_MANAGER_NAMESPACE"
81+
timeout: 120

0 commit comments

Comments
 (0)