Skip to content

Commit ddbf9cf

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #8678 from musaprg/add-docs-configuring-kubelet
📖 Add a section for configuring Kubelet in Cluster API
2 parents 9767b3c + 1018694 commit ddbf9cf

File tree

3 files changed

+356
-1
lines changed

3 files changed

+356
-1
lines changed

‎docs/book/src/SUMMARY.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,8 @@
1010
- [Generating a Kubeconfig](./tasks/certs/generate-kubeconfig.md)
1111
- [Auto Rotate Certificates in KCP](./tasks/certs/auto-rotate-certificates-in-kcp.md)
1212
- [Bootstrap](./tasks/bootstrap/index.md)
13-
- [Kubeadm based bootstrap](./tasks/bootstrap/kubeadm-bootstrap.md)
13+
- [Kubeadm based bootstrap](./tasks/bootstrap/kubeadm-bootstrap/index.md)
14+
- [Kubelet configuration](./tasks/bootstrap/kubeadm-bootstrap/kubelet-config.md)
1415
- [MicroK8s based bootstrap](./tasks/bootstrap/microk8s-bootstrap.md)
1516
- [Upgrading management and workload clusters](./tasks/upgrading-clusters.md)
1617
- [External etcd](./tasks/external-etcd.md)

‎docs/book/src/tasks/bootstrap/kubeadm-bootstrap.md renamed to ‎docs/book/src/tasks/bootstrap/kubeadm-bootstrap/index.md

File renamed without changes.

‎docs/book/src/tasks/bootstrap/kubeadm-bootstrap/kubelet-config.md

Lines changed: 354 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,354 @@
1+
# Kubelet Configuration
2+
3+
CAPBK has several ways to configure kubelet.
4+
5+
- [Pass `KubeletConfiguration` file via `KubeadmConfigSpec.files`](#pass-kubeletconfiguration-file-via-kubeadmconfigspecfiles)
6+
- [Set kubelet flags via `KubeadmConfigSpec.kubeletExtraArgs`](#set-kubelet-flags-via-kubeadmconfigspeckubeletextraargs)
7+
- [`kubeletconfiguration` patch target](#use-the-kubeletconfiguration-patch-target)
8+
9+
## Pass `KubeletConfiguration` file via `KubeadmConfigSpec.files`
10+
11+
You can use `KubeadmConfigSpec.files` to put any files on nodes. This example puts a `KubeletConfiguration` file on nodes via `KubeadmConfigSpec.files`, and makes kubelet use it via `KubeadmConfigSpec.kubeletExtraArgs`. You can check available configurations of `KubeletConfiguration` on [Kubelet Configuration (v1beta1) | Kubernetes](https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/#kubelet-config-k8s-io-v1beta1-KubeletConfiguration).
12+
13+
This method is easy to replace the whole kubelet configuration generated by kubeadm, but it is not easy to replace only a part of the kubelet configuration.
14+
15+
### KubeadmControlPlaneTemplate
16+
17+
```yaml
18+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
19+
kind: KubeadmControlPlaneTemplate
20+
metadata:
21+
name: cloudinit-control-plane
22+
namespace: default
23+
spec:
24+
template:
25+
spec:
26+
kubeadmConfigSpec:
27+
files:
28+
# We put a KubeletConfiguration file on nodes via KubeadmConfigSpec.files
29+
# In this example, we directly put the file content in the KubeadmConfigSpec.files.content field.
30+
- path: /etc/kubernetes/kubelet/config.yaml
31+
owner: "root:root"
32+
permissions: "0644"
33+
content: |
34+
apiVersion: kubelet.config.k8s.io/v1beta1
35+
kind: KubeletConfiguration
36+
kubeReserved:
37+
cpu: "1"
38+
memory: "2Gi"
39+
ephemeral-storage: "1Gi"
40+
systemReserved:
41+
cpu: "500m"
42+
memory: "1Gi"
43+
ephemeral-storage: "1Gi"
44+
evictionHard:
45+
memory.available: "500Mi"
46+
nodefs.available: "10%"
47+
authentication:
48+
anonymous:
49+
enabled: false
50+
webhook:
51+
cacheTTL: 0s
52+
enabled: true
53+
x509:
54+
clientCAFile: /etc/kubernetes/pki/ca.crt
55+
authorization:
56+
mode: Webhook
57+
webhook:
58+
cacheAuthorizedTTL: 0s
59+
cacheUnauthorizedTTL: 0s
60+
cgroupDriver: systemd
61+
clusterDNS:
62+
- 10.128.0.10
63+
clusterDomain: cluster.local
64+
containerRuntimeEndpoint: ""
65+
cpuManagerReconcilePeriod: 0s
66+
evictionPressureTransitionPeriod: 0s
67+
fileCheckFrequency: 0s
68+
healthzBindAddress: 127.0.0.1
69+
healthzPort: 10248
70+
httpCheckFrequency: 0s
71+
imageMinimumGCAge: 0s
72+
logging:
73+
flushFrequency: 0
74+
options:
75+
json:
76+
infoBufferSize: "0"
77+
verbosity: 0
78+
memorySwap: {}
79+
nodeStatusReportFrequency: 0s
80+
nodeStatusUpdateFrequency: 0s
81+
rotateCertificates: true
82+
runtimeRequestTimeout: 0s
83+
shutdownGracePeriod: 0s
84+
shutdownGracePeriodCriticalPods: 0s
85+
staticPodPath: /etc/kubernetes/manifests
86+
streamingConnectionIdleTimeout: 0s
87+
syncFrequency: 0s
88+
volumeStatsAggPeriod: 0s
89+
clusterConfiguration:
90+
controllerManager:
91+
extraArgs:
92+
enable-hostpath-provisioner: "true"
93+
initConfiguration:
94+
nodeRegistration:
95+
criSocket: unix:///var/run/containerd/containerd.sock
96+
# Here we configure kubelet to use the KubeletConfiguration file we put on nodes via KubeadmConfigSpec.files
97+
kubeletExtraArgs:
98+
config: "/etc/kubernetes/kubelet/config.yaml"
99+
joinConfiguration:
100+
nodeRegistration:
101+
criSocket: unix:///var/run/containerd/containerd.sock
102+
# Here we configure kubelet to use the KubeletConfiguration file we put on nodes via KubeadmConfigSpec.files
103+
kubeletExtraArgs:
104+
config: "/etc/kubernetes/kubelet/config.yaml"
105+
```
106+
107+
### KubeadmConfigTemplate
108+
109+
```yaml
110+
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
111+
kind: KubeadmConfigTemplate
112+
metadata:
113+
name: cloudinit-default-worker-bootstraptemplate
114+
namespace: default
115+
spec:
116+
template:
117+
spec:
118+
files:
119+
# We puts a KubeletConfiguration file on nodes via KubeadmConfigSpec.files
120+
# In this example, we directly put the file content in the KubeadmConfigSpec.files.content field.
121+
- path: /etc/kubernetes/kubelet/config.yaml
122+
owner: "root:root"
123+
permissions: "0644"
124+
content: |
125+
apiVersion: kubelet.config.k8s.io/v1beta1
126+
kind: KubeletConfiguration
127+
kubeReserved:
128+
cpu: "1"
129+
memory: "2Gi"
130+
ephemeral-storage: "1Gi"
131+
systemReserved:
132+
cpu: "500m"
133+
memory: "1Gi"
134+
ephemeral-storage: "1Gi"
135+
evictionHard:
136+
memory.available: "500Mi"
137+
nodefs.available: "10%"
138+
authentication:
139+
anonymous:
140+
enabled: false
141+
webhook:
142+
cacheTTL: 0s
143+
enabled: true
144+
x509:
145+
clientCAFile: /etc/kubernetes/pki/ca.crt
146+
authorization:
147+
mode: Webhook
148+
webhook:
149+
cacheAuthorizedTTL: 0s
150+
cacheUnauthorizedTTL: 0s
151+
cgroupDriver: systemd
152+
clusterDNS:
153+
- 10.128.0.10
154+
clusterDomain: cluster.local
155+
containerRuntimeEndpoint: ""
156+
cpuManagerReconcilePeriod: 0s
157+
evictionPressureTransitionPeriod: 0s
158+
fileCheckFrequency: 0s
159+
healthzBindAddress: 127.0.0.1
160+
healthzPort: 10248
161+
httpCheckFrequency: 0s
162+
imageMinimumGCAge: 0s
163+
logging:
164+
flushFrequency: 0
165+
options:
166+
json:
167+
infoBufferSize: "0"
168+
verbosity: 0
169+
memorySwap: {}
170+
nodeStatusReportFrequency: 0s
171+
nodeStatusUpdateFrequency: 0s
172+
rotateCertificates: true
173+
runtimeRequestTimeout: 0s
174+
shutdownGracePeriod: 0s
175+
shutdownGracePeriodCriticalPods: 0s
176+
staticPodPath: /etc/kubernetes/manifests
177+
streamingConnectionIdleTimeout: 0s
178+
syncFrequency: 0s
179+
volumeStatsAggPeriod: 0s
180+
joinConfiguration:
181+
nodeRegistration:
182+
criSocket: unix:///var/run/containerd/containerd.sock
183+
# Here we configure kubelet to use the KubeletConfiguration file we put on nodes via KubeadmConfigSpec.files
184+
kubeletExtraArgs:
185+
config: "/etc/kubernetes/kubelet/config.yaml"
186+
```
187+
188+
## Set kubelet flags via `KubeadmConfigSpec.kubeletExtraArgs`
189+
190+
We can pass kubelet command-line flags via `KubeadmConfigSpec.kubeletExtraArgs`. This example is equivalent to setting `--kube-reserved`, `--system-reserved`, and `--eviction-hard` flags for the kubelet command.
191+
192+
This method is useful when you want to set kubelet flags that are not configurable via the `KubeletConfiguration` file, however, it is not recommended to use this method to set flags that are configurable via the `KubeletConfiguration` file.
193+
194+
### KubeadmControlPlaneTemplate
195+
196+
```yaml
197+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
198+
kind: KubeadmControlPlaneTemplate
199+
metadata:
200+
name: kubelet-extra-args-control-plane
201+
namespace: default
202+
spec:
203+
template:
204+
spec:
205+
kubeadmConfigSpec:
206+
clusterConfiguration:
207+
controllerManager:
208+
extraArgs:
209+
enable-hostpath-provisioner: "true"
210+
initConfiguration:
211+
nodeRegistration:
212+
criSocket: unix:///var/run/containerd/containerd.sock
213+
# Set kubelet flags via KubeadmConfigSpec.kubeletExtraArgs
214+
kubeletExtraArgs:
215+
kube-reserved: cpu=1,memory=2Gi,ephemeral-storage=1Gi
216+
system-reserved: cpu=500m,memory=1Gi,ephemeral-storage=1Gi
217+
eviction-hard: memory.available<500Mi,nodefs.available<10%
218+
joinConfiguration:
219+
nodeRegistration:
220+
criSocket: unix:///var/run/containerd/containerd.sock
221+
# Set kubelet flags via KubeadmConfigSpec.kubeletExtraArgs
222+
kubeletExtraArgs:
223+
kube-reserved: cpu=1,memory=2Gi,ephemeral-storage=1Gi
224+
system-reserved: cpu=500m,memory=1Gi,ephemeral-storage=1Gi
225+
eviction-hard: memory.available<500Mi,nodefs.available<10%
226+
```
227+
228+
### KubeadmConfigTemplate
229+
230+
```yaml
231+
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
232+
kind: KubeadmConfigTemplate
233+
metadata:
234+
name: kubelet-extra-args-default-worker-bootstraptemplate
235+
namespace: default
236+
spec:
237+
template:
238+
spec:
239+
joinConfiguration:
240+
nodeRegistration:
241+
criSocket: unix:///var/run/containerd/containerd.sock
242+
# Set kubelet flags via KubeadmConfigSpec.kubeletExtraArgs
243+
kubeletExtraArgs:
244+
kube-reserved: cpu=1,memory=2Gi,ephemeral-storage=1Gi
245+
system-reserved: cpu=500m,memory=1Gi,ephemeral-storage=1Gi
246+
eviction-hard: memory.available<500Mi,nodefs.available<10%
247+
```
248+
249+
## Use kubeadm's `kubeletconfiguration` patch target
250+
251+
We can use kubeadm's `kubeletconfiguration` patch target to patch the kubelet configuration file. In this example, we put a patch file for `kubeletconfiguration` target in `strategic` `patchtype` on nodes via `KubeadmConfigSpec.files`. For more details, see [Customizing components with the kubeadm API | Kubernetes](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/control-plane-flags/#patches)
252+
253+
This method is useful when you want to change the kubelet configuration file partially on specific nodes. For example, you can deploy a partially patched kubelet configuration file on specific nodes based on the default configuration used for `kubeadm init` or `kubeadm join`.
254+
255+
### KubeadmControlPlaneTemplate
256+
257+
```yaml
258+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
259+
kind: KubeadmControlPlaneTemplate
260+
metadata:
261+
name: kubeadm-config-template-control-plane
262+
namespace: default
263+
spec:
264+
template:
265+
spec:
266+
kubeadmConfigSpec:
267+
files:
268+
# Here we put a patch file for kubeletconfiguration target in strategic patchtype on nodes via KubeadmConfigSpec.files
269+
# The naming convention of the patch file is kubeletconfiguration{suffix}+{patchtype}.json where {suffix} is an string and {patchtype} is one of the following: strategic, merge, json.
270+
# {suffix} determines the order of the patch files. The patches are applied in the alpha-numerical order of the {suffix}.
271+
- path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
272+
owner: "root:root"
273+
permissions: "0644"
274+
content: |
275+
{
276+
"apiVersion": "kubelet.config.k8s.io/v1beta1",
277+
"kind": "KubeletConfiguration",
278+
"kubeReserved": {
279+
"cpu": "1",
280+
"memory": "2Gi",
281+
"ephemeral-storage": "1Gi",
282+
},
283+
"systemReserved": {
284+
"cpu": "500m",
285+
"memory": "1Gi",
286+
"ephemeral-storage": "1Gi",
287+
},
288+
"evictionHard": {
289+
"memory.available": "500Mi",
290+
"nodefs.available": "10%",
291+
},
292+
}
293+
clusterConfiguration:
294+
controllerManager:
295+
extraArgs:
296+
enable-hostpath-provisioner: "true"
297+
initConfiguration:
298+
nodeRegistration:
299+
criSocket: unix:///var/run/containerd/containerd.sock
300+
# Here we specify the directory that contains the patch files
301+
patches:
302+
directory: /etc/kubernetes/patches
303+
joinConfiguration:
304+
nodeRegistration:
305+
criSocket: unix:///var/run/containerd/containerd.sock
306+
# Here we specify the directory that contains the patch files
307+
patches:
308+
directory: /etc/kubernetes/patches
309+
```
310+
311+
### KubeadmConfigTemplate
312+
313+
```yaml
314+
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
315+
kind: KubeadmConfigTemplate
316+
metadata:
317+
name: kubeadm-config-template-default-worker-bootstraptemplate
318+
namespace: default
319+
spec:
320+
template:
321+
spec:
322+
files:
323+
# Here we put a patch file for kubeletconfiguration target in strategic patchtype on nodes via KubeadmConfigSpec.files
324+
# The naming convention of the patch file is kubeletconfiguration{suffix}+{patchtype}.json where {suffix} is an string and {patchtype} is one of the following: strategic, merge, json.
325+
# {suffix} determines the order of the patch files. The patches are applied in the alpha-numerical order of the {suffix}.
326+
- path: /etc/kubernetes/patches/kubeletconfiguration0+strategic.json
327+
owner: "root:root"
328+
permissions: "0644"
329+
content: |
330+
{
331+
"apiVersion": "kubelet.config.k8s.io/v1beta1",
332+
"kind": "KubeletConfiguration",
333+
"kubeReserved": {
334+
"cpu": "1",
335+
"memory": "2Gi",
336+
"ephemeral-storage": "1Gi",
337+
},
338+
"systemReserved": {
339+
"cpu": "500m",
340+
"memory": "1Gi",
341+
"ephemeral-storage": "1Gi",
342+
},
343+
"evictionHard": {
344+
"memory.available": "500Mi",
345+
"nodefs.available": "10%",
346+
},
347+
}
348+
joinConfiguration:
349+
nodeRegistration:
350+
criSocket: unix:///var/run/containerd/containerd.sock
351+
# Here we specify the directory that contains the patch files
352+
patches:
353+
directory: /etc/kubernetes/patches
354+
```

0 commit comments

Comments
 (0)