Skip to content

Commit ef614ce

Browse files
mandreMiguelCarpio
andauthored
OpenStack: Add openstack-provision-etcd-disk-speed step (#58941)
* OpenStack: Add openstack-provision-etcd-disk-speed step This step patches the etcd cluster to configure disk speed. It defaults to being a no-op. Co-Authored-By: Miguel Carpio <[email protected]> * OpenStack: configure slow etcd disk for all OpenStack clusters We know our OpenStack envs are slow. Let's tweak etcd to be more tolerant to disk latency. --------- Co-authored-by: Miguel Carpio <[email protected]>
1 parent ba362b5 commit ef614ce

10 files changed

+99
-0
lines changed

ci-operator/step-registry/ipi/openstack/pre/cgroupsv1/ipi-openstack-pre-cgroupsv1-chain.yaml

+3
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,10 @@ chain:
33
steps:
44
- chain: ipi-conf-openstack-cgroupsv1
55
- chain: ipi-install
6+
- ref: openstack-provision-etcd-disk-speed
67
env:
8+
- name: ETCD_DISK_SPEED
9+
default: "slow"
710
- name: USE_RAMFS
811
default: "true"
912
documentation: |-

ci-operator/step-registry/ipi/openstack/pre/disconnected/ipi-openstack-pre-disconnected-chain.yaml

+3
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,12 @@ chain:
2727
timeout: 5m0s
2828
- ref: nodes-readiness
2929
- ref: multiarch-validate-nodes
30+
- ref: openstack-provision-etcd-disk-speed
3031
env:
3132
- name: USE_RAMFS
3233
default: "true"
34+
- name: ETCD_DISK_SPEED
35+
default: "slow"
3336
documentation: |-
3437
The ipi-openstack-pre-disconnected chain contains the necessary refs
3538
and chains to run ipi install on OpenStack in disconnected environment.

ci-operator/step-registry/ipi/openstack/pre/ipi-openstack-pre-chain.yaml

+3
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,12 @@ chain:
33
steps:
44
- chain: ipi-conf-openstack
55
- chain: ipi-install
6+
- ref: openstack-provision-etcd-disk-speed
67
- ref: openstack-rotate-cloud-credentials
78
- ref: openstack-provision-etcd-on-local-disk
89
env:
10+
- name: ETCD_DISK_SPEED
11+
default: "slow"
912
- name: USE_RAMFS
1013
default: "true"
1114
documentation: |-

ci-operator/step-registry/ipi/openstack/pre/noramfs/ipi-openstack-pre-noramfs-chain.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@ chain:
33
steps:
44
- chain: ipi-conf-openstack-noramfs
55
- chain: ipi-install
6+
- ref: openstack-provision-etcd-disk-speed
7+
env:
8+
- name: ETCD_DISK_SPEED
9+
default: "slow"
610
documentation: |-
711
The ipi-openstack-pre-noramfs chain contains the necessary refs and chains
812
to prepare the system to run ipi install on OpenStack without using a ramfs

ci-operator/step-registry/ipi/openstack/pre/stableinitial/ipi-openstack-pre-stableinitial-chain.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@ chain:
44
- chain: ipi-conf-openstack-noramfs
55
- ref: ipi-conf-openstack-enable-octavia
66
- chain: ipi-install-stableinitial
7+
- ref: openstack-provision-etcd-disk-speed
8+
env:
9+
- name: ETCD_DISK_SPEED
10+
default: "slow"
711
documentation: |-
812
The ipi-openstack-pre-stableinitial chain contains the necessary
913
refs and chains to prepare the system to run
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../OWNERS
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
#!/usr/bin/env bash
2+
3+
set -o nounset
4+
set -o errexit
5+
set -o pipefail
6+
7+
function info() {
8+
printf '%s: INFO: %s\n' "$(date --utc +%Y-%m-%dT%H:%M:%SZ)" "$*"
9+
}
10+
11+
function wait_for_etcd_patch_done() {
12+
info 'Waiting for patch etcd cluster...'
13+
for _ in {1..30}; do
14+
ETCD_CO_AVAILABLE=$(oc get co etcd | grep etcd | awk '{print $3}')
15+
if [[ "${ETCD_CO_AVAILABLE}" == "True" ]]; then
16+
echo "Patched successfully!"
17+
break
18+
fi
19+
sleep 15
20+
done
21+
if [[ "${ETCD_CO_AVAILABLE}" != "True" ]]; then
22+
echo "Etcd patch failed..."
23+
exit 1
24+
fi
25+
info 'Etcd Patched successfully!'
26+
}
27+
28+
# For disconnected or otherwise unreachable environments, we want to
29+
# have steps use an HTTP(S) proxy to reach the API server. This proxy
30+
# configuration file should export HTTP_PROXY, HTTPS_PROXY, and NO_PROXY
31+
# environment variables, as well as their lowercase equivalents (note
32+
# that libcurl doesn't recognize the uppercase variables).
33+
if test -f "${SHARED_DIR}/proxy-conf.sh"
34+
then
35+
# shellcheck disable=SC1090
36+
source "${SHARED_DIR}/proxy-conf.sh"
37+
fi
38+
39+
oc_version=$(oc version -o json | jq -r '.openshiftVersion' | cut -d '.' -f1,2)
40+
41+
# The controlPlaneHardwareSpeed param is only available from 4.16
42+
if [[ $(jq -n "$oc_version < 4.16") == "true" ]]; then
43+
info 'etcd tuning profile are only available from OCP 4.16... Skipping.'
44+
exit 0
45+
fi
46+
47+
# Patch etcd for allowing slower disks
48+
if [[ "${ETCD_DISK_SPEED}" == "slow" ]]; then
49+
info 'Patching etcd cluster operator...'
50+
oc patch etcd cluster --type=merge --patch '{"spec":{"controlPlaneHardwareSpeed":"Slower"}}'
51+
wait_for_etcd_patch_done
52+
fi
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"path": "openstack/provision/etcd-disk-speed/openstack-provision-etcd-disk-speed-ref.yaml",
3+
"owners": {
4+
"approvers": [
5+
"openstack-approvers"
6+
],
7+
"reviewers": [
8+
"openstack-reviewers"
9+
]
10+
}
11+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
ref:
2+
as: openstack-provision-etcd-disk-speed
3+
from: openstack-installer
4+
commands: openstack-provision-etcd-disk-speed-commands.sh
5+
resources:
6+
requests:
7+
cpu: 10m
8+
memory: 100Mi
9+
env:
10+
- name: OS_CLOUD
11+
default: "openstack"
12+
- name: ETCD_DISK_SPEED
13+
default: ""
14+
documentation: |-
15+
This step patches the etcd cluster to configure disk speed.

ci-operator/step-registry/upi/openstack/pre/upi-openstack-pre-chain.yaml

+3
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,12 @@ chain:
33
steps:
44
- chain: ipi-conf-openstack
55
- chain: upi-install-openstack
6+
- ref: openstack-provision-etcd-disk-speed
67
env:
78
- name: USE_RAMFS
89
default: "true"
10+
- name: ETCD_DISK_SPEED
11+
default: "slow"
912
documentation: |-
1013
The upi-openstack-pre chain contains the necessary
1114
refs and chains to prepare the system to run

0 commit comments

Comments
 (0)