Skip to content

Commit 111ba26

Browse files
VietND96VietND96
authored
K8s: Default Node register period is longer with a short cycle (#2662)
Signed-off-by: Viet Nguyen Duc <[email protected]>
1 parent 0a5b6ec commit 111ba26

File tree

9 files changed

+82
-18
lines changed

9 files changed

+82
-18
lines changed

Diff for: ‎ENV_VARIABLES.md

+5-4
Original file line numberDiff line numberDiff line change
@@ -72,13 +72,13 @@
7272
| SE_VNC_NO_PASSWORD | | | |
7373
| SE_VNC_VIEW_ONLY | | | |
7474
| SE_VNC_PASSWORD | | | |
75-
| SE_EVENT_BUS_PUBLISH_PORT | | | |
76-
| SE_EVENT_BUS_SUBSCRIBE_PORT | | | |
75+
| SE_EVENT_BUS_PUBLISH_PORT | 4442 | | |
76+
| SE_EVENT_BUS_SUBSCRIBE_PORT | 4443 | | |
7777
| SE_NODE_SESSION_TIMEOUT | 300 | | |
7878
| SE_NODE_ENABLE_MANAGED_DOWNLOADS | | | |
7979
| SE_NODE_ENABLE_CDP | | | |
80-
| SE_NODE_REGISTER_PERIOD | | | |
81-
| SE_NODE_REGISTER_CYCLE | | | |
80+
| SE_NODE_REGISTER_PERIOD | 120 | | |
81+
| SE_NODE_REGISTER_CYCLE | 10 | | |
8282
| SE_NODE_HEARTBEAT_PERIOD | 30 | | |
8383
| SE_REGISTRATION_SECRET | | | |
8484
| SE_BROWSER_LEFTOVERS_PROCESSES_SECS | 7200 | | |
@@ -143,3 +143,4 @@
143143
| SE_SESSIONS_MAP_EXTERNAL_SCHEME | | | |
144144
| SE_NODE_RELAY_STEREOTYPE | | Capabilities in JSON string to overwrite the default Node relay stereotype | |
145145
| SE_NODE_RELAY_STEREOTYPE_EXTRA | | Extra capabilities in JSON string that wants to merge to the default Node relay stereotype | |
146+
| SE_NODE_REGISTER_SHUTDOWN_ON_FAILURE | true | If this flag is enabled, the Node will shut down after the register period is completed. This is useful for container environments to restart and register again. If restarted multiple times, the Node container status will be CrashLoopBackOff | --register-shutdown-on-failure |

Diff for: ‎NodeBase/Dockerfile

+5
Original file line numberDiff line numberDiff line change
@@ -38,12 +38,17 @@ ENV LANG_WHICH=${LANG_WHICH} \
3838
#========================
3939
# Selenium Configuration
4040
#========================
41+
SE_EVENT_BUS_PUBLISH_PORT="4442" \
42+
SE_EVENT_BUS_SUBSCRIBE_PORT="4443" \
4143
# Drain the Node after N sessions (a value higher than zero enables the feature)
4244
SE_DRAIN_AFTER_SESSION_COUNT="0" \
4345
SE_NODE_MAX_SESSIONS="1" \
4446
SE_NODE_SESSION_TIMEOUT="300" \
4547
SE_NODE_OVERRIDE_MAX_SESSIONS="false" \
4648
SE_NODE_HEARTBEAT_PERIOD="30" \
49+
SE_NODE_REGISTER_PERIOD="120" \
50+
SE_NODE_REGISTER_CYCLE="10" \
51+
SE_NODE_REGISTER_SHUTDOWN_ON_FAILURE="true" \
4752
SE_OTEL_SERVICE_NAME="selenium-node" \
4853
# Setting Selenium Manager to work offline
4954
SE_OFFLINE="true" \

Diff for: ‎NodeBase/generate_config

+6-4
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,12 @@ else
1414
FILENAME="$CONFIG_FILE"
1515
fi
1616

17-
echo "[events]
18-
publish = \"tcp://${SE_EVENT_BUS_HOST}:${SE_EVENT_BUS_PUBLISH_PORT}\"
19-
subscribe = \"tcp://${SE_EVENT_BUS_HOST}:${SE_EVENT_BUS_SUBSCRIBE_PORT}\"
20-
" >"$FILENAME"
17+
if [[ -n "${SE_EVENT_BUS_HOST}" ]]; then
18+
echo "[events]
19+
publish = \"tcp://${SE_EVENT_BUS_HOST}:${SE_EVENT_BUS_PUBLISH_PORT}\"
20+
subscribe = \"tcp://${SE_EVENT_BUS_HOST}:${SE_EVENT_BUS_SUBSCRIBE_PORT}\"
21+
" >"$FILENAME"
22+
fi
2123

2224
if [[ -z "${SE_NODE_HOST}" ]] && [[ -z "${SE_NODE_PORT}" ]]; then
2325
echo "Configuring server..."

Diff for: ‎NodeBase/start-selenium-node.sh

+4
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,10 @@ if [ ! -z "$SE_NODE_REGISTER_CYCLE" ]; then
6969
append_se_opts "--register-cycle" "${SE_NODE_REGISTER_CYCLE}"
7070
fi
7171

72+
if [ "$SE_NODE_REGISTER_SHUTDOWN_ON_FAILURE" = "true" ]; then
73+
append_se_opts "--register-shutdown-on-failure"
74+
fi
75+
7276
if [ ! -z "$SE_NODE_HEARTBEAT_PERIOD" ]; then
7377
append_se_opts "--heartbeat-period" "${SE_NODE_HEARTBEAT_PERIOD}"
7478
fi

Diff for: ‎charts/selenium-grid/CONFIGURATION.md

+14
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,8 @@ A Helm chart for creating a Selenium Grid Server in Kubernetes
4949
| global.seleniumGrid.affinity | object | `{}` | Specify affinity for all components, can be overridden individually |
5050
| global.seleniumGrid.topologySpreadConstraints | list | `[]` | Specify topologySpreadConstraints for all components, can be overridden individually |
5151
| global.seleniumGrid.nodeMaxSessions | int | `1` | Specify number of max sessions per node. Can be overridden by individual component (this is also set to scaler trigger parameter `nodeMaxSessions` if `autoscaling` is enabled) |
52+
| global.seleniumGrid.nodeRegisterPeriod | int | `120` | How long, in seconds, will the Node try to register to the Distributor for the first time. After this period is completed, the Node will not attempt to register again. |
53+
| global.seleniumGrid.nodeRegisterCycle | int | `5` | How often, in seconds, the Node will try to register itself for the first time to the Distributor. |
5254
| tls.create | bool | `true` | Create a Secret resource for TLS certificate and key. If using an external secret set to false and provide its name in `nameOverride` below |
5355
| tls.nameOverride | string | `nil` | Name of external secret containing the TLS certificate and key |
5456
| tls.enabled | bool | `false` | Enable or disable TLS for the server components (and ingress proxy) |
@@ -454,13 +456,16 @@ A Helm chart for creating a Selenium Grid Server in Kubernetes
454456
| chromeNode.extraVolumeMounts | list | `[]` | Extra volume mounts for chrome-node container |
455457
| chromeNode.extraVolumes | list | `[]` | Extra volumes for chrome-node pod |
456458
| chromeNode.nodeMaxSessions | string | `nil` | Override the number of max sessions per node |
459+
| chromeNode.nodeRegisterPeriod | string | `nil` | Override the same config at the global level |
460+
| chromeNode.nodeRegisterCycle | string | `nil` | Override the same config at the global level |
457461
| chromeNode.scaledOptions | string | `nil` | Override the scaled options for chrome nodes |
458462
| chromeNode.scaledJobOptions | string | `nil` | Override the scaledJobOptions for chrome nodes |
459463
| chromeNode.scaledObjectOptions | string | `nil` | Override the scaledObjectOptions for chrome nodes |
460464
| chromeNode.hpa.browserName | string | `"chrome"` | browserName should match with Node stereotype and request capability is scaled by this scaler |
461465
| chromeNode.hpa.sessionBrowserName | string | `"chrome"` | sessionBrowserName if the browserName is different from the sessionBrowserName |
462466
| chromeNode.hpa.browserVersion | string | `""` | browserVersion should match with Node stereotype and request capability is scaled by this scaler |
463467
| chromeNode.hpa.platformName | string | `""` | platformName should match with Node stereotype and request capability is scaled by this scaler |
468+
| chromeNode.hpa.capabilities | string | `""` | Setting more custom capabilities for matching specific Nodes |
464469
| chromeNode.hpa.unsafeSsl | string | `"{{ template \"seleniumGrid.graphqlURL.unsafeSsl\" . }}"` | Skip check SSL when connecting to the Graphql endpoint |
465470
| chromeNode.initContainers | list | `[]` | It is used to add initContainers in the same pod of the browser node. It should be set using the --set-json option |
466471
| chromeNode.sidecars | list | `[]` | It is used to add sidecars proxy in the same pod of the browser node. It means it will add a new container to the deployment itself. It should be set using the --set-json option |
@@ -508,13 +513,16 @@ A Helm chart for creating a Selenium Grid Server in Kubernetes
508513
| firefoxNode.extraVolumeMounts | list | `[]` | Extra volume mounts for firefox-node container |
509514
| firefoxNode.extraVolumes | list | `[]` | Extra volumes for firefox-node pod |
510515
| firefoxNode.nodeMaxSessions | string | `nil` | Override the number of max sessions per node |
516+
| firefoxNode.nodeRegisterPeriod | string | `nil` | Override the same config at the global level |
517+
| firefoxNode.nodeRegisterCycle | string | `nil` | Override the same config at the global level |
511518
| firefoxNode.scaledOptions | string | `nil` | Override the scaled options for firefox nodes |
512519
| firefoxNode.scaledJobOptions | string | `nil` | Override the scaledJobOptions for firefox nodes |
513520
| firefoxNode.scaledObjectOptions | string | `nil` | Override the scaledObjectOptions for firefox nodes |
514521
| firefoxNode.hpa.browserName | string | `"firefox"` | browserName should match with Node stereotype and request capability is scaled by this scaler |
515522
| firefoxNode.hpa.sessionBrowserName | string | `"firefox"` | sessionBrowserName if the browserName is different from the sessionBrowserName |
516523
| firefoxNode.hpa.browserVersion | string | `""` | browserVersion should match with Node stereotype and request capability is scaled by this scaler |
517524
| firefoxNode.hpa.platformName | string | `""` | platformName should match with Node stereotype and request capability is scaled by this scaler |
525+
| firefoxNode.hpa.capabilities | string | `""` | Setting more custom capabilities for matching specific Nodes |
518526
| firefoxNode.hpa.unsafeSsl | string | `"{{ template \"seleniumGrid.graphqlURL.unsafeSsl\" . }}"` | Skip check SSL when connecting to the Graphql endpoint |
519527
| firefoxNode.initContainers | list | `[]` | It is used to add initContainers in the same pod of the browser node. It should be set using the --set-json option |
520528
| firefoxNode.sidecars | list | `[]` | It is used to add sidecars proxy in the same pod of the browser node. It means it will add a new container to the deployment itself. It should be set using the --set-json option |
@@ -562,13 +570,16 @@ A Helm chart for creating a Selenium Grid Server in Kubernetes
562570
| edgeNode.extraVolumeMounts | list | `[]` | Extra volume mounts for edge-node container |
563571
| edgeNode.extraVolumes | list | `[]` | Extra volumes for edge-node pod |
564572
| edgeNode.nodeMaxSessions | string | `nil` | Override the number of max sessions per node |
573+
| edgeNode.nodeRegisterPeriod | string | `nil` | Override the same config at the global level |
574+
| edgeNode.nodeRegisterCycle | string | `nil` | Override the same config at the global level |
565575
| edgeNode.scaledOptions | string | `nil` | Override the scaled options for edge nodes |
566576
| edgeNode.scaledJobOptions | string | `nil` | Override the scaledJobOptions for edge nodes |
567577
| edgeNode.scaledObjectOptions | string | `nil` | Override the scaledObjectOptions for edge nodes |
568578
| edgeNode.hpa.browserName | string | `"MicrosoftEdge"` | browserName should match with Node stereotype and request capability is scaled by this scaler |
569579
| edgeNode.hpa.sessionBrowserName | string | `"msedge"` | sessionBrowserName if the browserName is different from the sessionBrowserName |
570580
| edgeNode.hpa.browserVersion | string | `""` | browserVersion should match with Node stereotype and request capability is scaled by this scaler |
571581
| edgeNode.hpa.platformName | string | `""` | platformName should match with Node stereotype and request capability is scaled by this scaler |
582+
| edgeNode.hpa.capabilities | string | `""` | Setting more custom capabilities for matching specific Nodes |
572583
| edgeNode.hpa.unsafeSsl | string | `"{{ template \"seleniumGrid.graphqlURL.unsafeSsl\" . }}"` | Skip check SSL when connecting to the Graphql endpoint |
573584
| edgeNode.initContainers | list | `[]` | It is used to add initContainers in the same pod of the browser node. It should be set using the --set-json option |
574585
| edgeNode.sidecars | list | `[]` | It is used to add sidecars proxy in the same pod of the browser node. It means it will add a new container to the deployment itself. It should be set using the --set-json option |
@@ -616,13 +627,16 @@ A Helm chart for creating a Selenium Grid Server in Kubernetes
616627
| relayNode.extraVolumeMounts | list | `[]` | Extra volume mounts for relay-node container |
617628
| relayNode.extraVolumes | list | `[]` | Extra volumes for relay-node pod |
618629
| relayNode.nodeMaxSessions | string | `nil` | Override the number of max sessions per node |
630+
| relayNode.nodeRegisterPeriod | string | `nil` | Override the same config at the global level |
631+
| relayNode.nodeRegisterCycle | string | `nil` | Override the same config at the global level |
619632
| relayNode.scaledOptions | string | `nil` | Override the scaled options for relay nodes |
620633
| relayNode.scaledJobOptions | string | `nil` | Override the scaledJobOptions for relay nodes |
621634
| relayNode.scaledObjectOptions | string | `nil` | Override the scaledObjectOptions for relay nodes |
622635
| relayNode.hpa.browserName | string | `"chrome"` | browserName should match with Node stereotype and request capability is scaled by this scaler |
623636
| relayNode.hpa.sessionBrowserName | string | `""` | sessionBrowserName if the browserName is different from the sessionBrowserName |
624637
| relayNode.hpa.browserVersion | string | `""` | browserVersion should match with Node stereotype and request capability is scaled by this scaler |
625638
| relayNode.hpa.platformName | string | `"Android"` | platformName should match with Node stereotype and request capability is scaled by this scaler |
639+
| relayNode.hpa.capabilities | string | `""` | Setting more custom capabilities for matching specific Nodes |
626640
| relayNode.hpa.unsafeSsl | string | `"{{ template \"seleniumGrid.graphqlURL.unsafeSsl\" . }}"` | Skip check SSL when connecting to the Graphql endpoint |
627641
| relayNode.initContainers | list | `[]` | It is used to add initContainers in the same pod of the browser node. It should be set using the --set-json option |
628642
| relayNode.sidecars | list | `[]` | It is used to add sidecars proxy in the same pod of the browser node. It means it will add a new container to the deployment itself. It should be set using the --set-json option |

Diff for: ‎charts/selenium-grid/templates/_helpers.tpl

+8-6
Original file line numberDiff line numberDiff line change
@@ -284,6 +284,8 @@ Common pod template
284284
{{- $videoImageRegistry := default $.Values.global.seleniumGrid.imageRegistry .recorder.imageRegistry -}}
285285
{{- $videoImageTag := default $.Values.global.seleniumGrid.videoImageTag .recorder.imageTag -}}
286286
{{- $nodeMaxSessions := default $.Values.global.seleniumGrid.nodeMaxSessions .node.nodeMaxSessions | int64 -}}
287+
{{- $nodeRegisterPeriod := default $.Values.global.seleniumGrid.nodeRegisterPeriod .node.nodeRegisterPeriod | int64 -}}
288+
{{- $nodeRegisterCycle := default $.Values.global.seleniumGrid.nodeRegisterCycle .node.nodeRegisterCycle | int64 -}}
287289
template:
288290
metadata:
289291
labels:
@@ -356,6 +358,10 @@ template:
356358
{{- if and (eq (include "seleniumGrid.useKEDA" $) "true") }}
357359
- name: SE_NODE_PLATFORM_NAME
358360
value: {{ if hasKey .node.hpa "platformName" }}{{ .node.hpa.platformName | quote }}{{ else }}""{{ end }}
361+
{{- end }}
362+
{{- if and (eq (include "seleniumGrid.useKEDA" $) "true") }}
363+
- name: SE_NODE_STEREOTYPE_EXTRA
364+
value: {{ if hasKey .node.hpa "capabilities" }}{{ .node.hpa.capabilities | quote }}{{ else }}""{{ end }}
359365
{{- end }}
360366
- name: SE_NODE_CONTAINER_NAME
361367
valueFrom:
@@ -373,14 +379,10 @@ template:
373379
fieldPath: status.podIP
374380
- name: SE_NODE_PORT
375381
value: {{ .node.port | quote }}
376-
{{- with .node.startupProbe.timeoutSeconds }}
377382
- name: SE_NODE_REGISTER_PERIOD
378-
value: {{ . | quote }}
379-
{{- end }}
380-
{{- with .node.startupProbe.periodSeconds }}
383+
value: {{ $nodeRegisterPeriod | quote }}
381384
- name: SE_NODE_REGISTER_CYCLE
382-
value: {{ . | quote }}
383-
{{- end }}
385+
value: {{ $nodeRegisterCycle | quote }}
384386
{{- with .node.extraEnvironmentVariables }}
385387
{{- tpl (toYaml .) $ | nindent 10 }}
386388
{{- end }}

Diff for: ‎charts/selenium-grid/values.yaml

+28
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,10 @@ global:
5050
# Note: If not define labelSelector, it will be added automatically based on "app" label in each component
5151
# -- Specify number of max sessions per node. Can be overridden by individual component (this is also set to scaler trigger parameter `nodeMaxSessions` if `autoscaling` is enabled)
5252
nodeMaxSessions: 1
53+
# -- How long, in seconds, will the Node try to register to the Distributor for the first time. After this period is completed, the Node will not attempt to register again.
54+
nodeRegisterPeriod: 120
55+
# -- How often, in seconds, the Node will try to register itself for the first time to the Distributor.
56+
nodeRegisterCycle: 5
5357

5458
tls:
5559
# -- Create a Secret resource for TLS certificate and key. If using an external secret set to false and provide its name in `nameOverride` below
@@ -1259,6 +1263,10 @@ chromeNode:
12591263

12601264
# -- Override the number of max sessions per node
12611265
nodeMaxSessions:
1266+
# -- Override the same config at the global level
1267+
nodeRegisterPeriod:
1268+
# -- Override the same config at the global level
1269+
nodeRegisterCycle:
12621270
# -- Override the scaled options for chrome nodes
12631271
scaledOptions:
12641272
# -- Override the scaledJobOptions for chrome nodes
@@ -1274,6 +1282,8 @@ chromeNode:
12741282
browserVersion: ""
12751283
# -- platformName should match with Node stereotype and request capability is scaled by this scaler
12761284
platformName: ""
1285+
# -- Setting more custom capabilities for matching specific Nodes
1286+
capabilities: ""
12771287
# browserVersion: '91.0' # Optional. Only required when supporting multiple versions of browser in your Selenium Grid.
12781288
# -- Skip check SSL when connecting to the Graphql endpoint
12791289
unsafeSsl: '{{ template "seleniumGrid.graphqlURL.unsafeSsl" . }}' # Optional
@@ -1448,6 +1458,10 @@ firefoxNode:
14481458

14491459
# -- Override the number of max sessions per node
14501460
nodeMaxSessions:
1461+
# -- Override the same config at the global level
1462+
nodeRegisterPeriod:
1463+
# -- Override the same config at the global level
1464+
nodeRegisterCycle:
14511465
# -- Override the scaled options for firefox nodes
14521466
scaledOptions:
14531467
# -- Override the scaledJobOptions for firefox nodes
@@ -1463,6 +1477,8 @@ firefoxNode:
14631477
browserVersion: ""
14641478
# -- platformName should match with Node stereotype and request capability is scaled by this scaler
14651479
platformName: ""
1480+
# -- Setting more custom capabilities for matching specific Nodes
1481+
capabilities: ""
14661482
# -- Skip check SSL when connecting to the Graphql endpoint
14671483
unsafeSsl: '{{ template "seleniumGrid.graphqlURL.unsafeSsl" . }}' # Optional
14681484

@@ -1636,6 +1652,10 @@ edgeNode:
16361652

16371653
# -- Override the number of max sessions per node
16381654
nodeMaxSessions:
1655+
# -- Override the same config at the global level
1656+
nodeRegisterPeriod:
1657+
# -- Override the same config at the global level
1658+
nodeRegisterCycle:
16391659
# -- Override the scaled options for edge nodes
16401660
scaledOptions:
16411661
# -- Override the scaledJobOptions for edge nodes
@@ -1651,6 +1671,8 @@ edgeNode:
16511671
browserVersion: ""
16521672
# -- platformName should match with Node stereotype and request capability is scaled by this scaler
16531673
platformName: ""
1674+
# -- Setting more custom capabilities for matching specific Nodes
1675+
capabilities: ""
16541676
# -- Skip check SSL when connecting to the Graphql endpoint
16551677
unsafeSsl: '{{ template "seleniumGrid.graphqlURL.unsafeSsl" . }}' # Optional
16561678

@@ -1824,6 +1846,10 @@ relayNode:
18241846

18251847
# -- Override the number of max sessions per node
18261848
nodeMaxSessions:
1849+
# -- Override the same config at the global level
1850+
nodeRegisterPeriod:
1851+
# -- Override the same config at the global level
1852+
nodeRegisterCycle:
18271853
# -- Override the scaled options for relay nodes
18281854
scaledOptions:
18291855
# -- Override the scaledJobOptions for relay nodes
@@ -1839,6 +1865,8 @@ relayNode:
18391865
browserVersion: ""
18401866
# -- platformName should match with Node stereotype and request capability is scaled by this scaler
18411867
platformName: "Android"
1868+
# -- Setting more custom capabilities for matching specific Nodes
1869+
capabilities: ""
18421870
# -- Skip check SSL when connecting to the Graphql endpoint
18431871
unsafeSsl: '{{ template "seleniumGrid.graphqlURL.unsafeSsl" . }}' # Optional
18441872

Diff for: ‎scripts/generate_list_env_vars/description.yaml

+6
Original file line numberDiff line numberDiff line change
@@ -434,3 +434,9 @@
434434
description: Extra capabilities in JSON string that wants to merge to the default
435435
Node relay stereotype
436436
cli: ''
437+
- name: SE_NODE_REGISTER_SHUTDOWN_ON_FAILURE
438+
description: If this flag is enabled, the Node will shut down after the register
439+
period is completed. This is useful for container environments to restart and
440+
register again. If restarted multiple times, the Node container status will be
441+
CrashLoopBackOff
442+
cli: --register-shutdown-on-failure

Diff for: ‎scripts/generate_list_env_vars/value.yaml

+6-4
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,9 @@
3131
- name: SE_EVENT_BUS_PORT
3232
default: ''
3333
- name: SE_EVENT_BUS_PUBLISH_PORT
34-
default: ''
34+
default: '4442'
3535
- name: SE_EVENT_BUS_SUBSCRIBE_PORT
36-
default: ''
36+
default: '4443'
3737
- name: SE_EXTERNAL_URL
3838
default: ''
3939
- name: SE_FRAME_RATE
@@ -107,9 +107,11 @@
107107
- name: SE_NODE_PRESTOP_WAIT_STRATEGY
108108
default: ''
109109
- name: SE_NODE_REGISTER_CYCLE
110-
default: ''
110+
default: '10'
111111
- name: SE_NODE_REGISTER_PERIOD
112-
default: ''
112+
default: '120'
113+
- name: SE_NODE_REGISTER_SHUTDOWN_ON_FAILURE
114+
default: 'true'
113115
- name: SE_NODE_RELAY_BROWSER_NAME
114116
default: ''
115117
- name: SE_NODE_RELAY_MAX_SESSIONS

0 commit comments

Comments
 (0)