Certificate rotation

[garloff]

We need to ensure we rotate all certificates to avoid expiration.
This is tracked in SovereignCloudStack/issues#155

[garloff]

Tracked here now.
I see three places where to enable kubelet client certificate rotation:
configmap kube-system/kubelet-config-1.xx:

data:
  kubelet: |
    rotateCertificates: true

and in KubeadmControlPlane in cluster-template.yaml

spec:
  kubeadmConfigSpec:
    initConfiguration:
      nodeRegistration:
        name: '{{ local_hostname }}'
        kubeletExtraArgs:
          cloud-provider: external
          rotateCertificates: true
    joinConfiguration:
      nodeRegistration:
        name: '{{ local_hostname }}'
        kubeletExtraArgs:
          cloud-provider: external
          rotateCertificates: true

which I believe will be propagated to /var/lib/kubelet/kubeadm-flags.env on the deployed nodes.
Do we need to tweak both places?
The setting in the configmap already seems to be there in newer versions, will need to research where it was added.

[garloff]

Passing --rotate-certificates as command line parameter is deprecated in favor of using rotateCertificates: true in the kubelet config file (/var/lib/kubelet/config.yaml). This is already enabled by default (thanks to upstream changes I assume, which I still need to track down and identify).
This issue may be non-existing for reasonably recent setups.

[garloff]

According to kubelet documentation the rotateCertificates: flag defaults to false, so we're lucky that it is set to true explicitly.
Looking through kubeadm, I can not find where this would be set. Nor do I see it in cluster-api, where some of the plumbing for kubeadm happens.

[garloff]

TODO:

• Need to test it really works (does really rotate the certs)
• Identify where it was implemented

[batistein]

In KCP & KCT you need to write rotate-certificates: true under the kubeletExtraArgs. The notation of commandline flags is always used there.

[batistein]

Currently, certificates are not automatically rotated in place. When using CAPBK, the CA certificate is copied to the node and kubeadm then creates new certificates.
Currently, the client certificate rotation is simply solved by exchanging the machines within one year. See more information here in the upstream tracking issue (this is also about serving certs): kubernetes-sigs/cluster-api#6529

[garloff]

OK, we will need to document this!

[garloff]

Andrej has a good text proposal in in issues/#155:
SovereignCloudStack/issues#155 (comment)

[jschoone]

As discussed in container meeting on april 3 it could depend on the clusterapi version which needs to be researched

[jschoone]

Hi @batistein, since this is still in status Doing, will you be able to address this next week?