[KEP-2400] Node memory swap support (replacement for inactive issue #2400)

[iholder101]

This is a replacement issue for the outdated, inactive previous issue: #2400.

Enhancement Description

• One-line enhancement description (can be used as a release note): Kubernetes nodes support swap memory.
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/2400-node-swap
• Discussion Link: https://docs.google.com/document/d/1CZtRtC8W8FwW_VWQLKP9DW_2H-hcLBcH9KBbukie67M/edit# Kubelet/Kubernetes should work with Swap Enabled kubernetes#53533
• Primary contact (assignee): @iholder101
• Responsible SIGs: Node
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.22
  • Beta release target (x.y): 1.28
  • Stable release target (x.y):
• Alpha
  • KEP (k/enhancements) update PR(s): Node system swap support #2602
  • Code (k/k) update PR(s):
    • CI/test infra changes: Setup new node image with swap enabled test-infra#22602 Add swap periodics for node test-infra#22809
    • CRI and kubelet changes Alpha node swap support kubernetes#102823
  • Docs (k/website) update PR(s): KEP-2400: Alpha node support for swap memory website#28838
• Beta1
  • KEP (k/enhancements) update PR(s):
    • KEP-2400: Update swap KEP for 1.23 beta #2858
    • KEP-2400: Graduate swap to Beta 1 #3957
  • Issues and Code (k/k) update PR(s):
    • Support cgroups v2 with auto-calculated LimitedSwap for Burstable QoS pods
      • [KEP-2400] Enable Swap Support for Cgroup v2 Only kubernetes#119427
      • [KEP-2400] Enable auto-calculated LimitedSwap for Burstable QoS pods kubernetes#119428
      • [KEP-2400] Add full cgroup v2 swap support with automatically calculated swap limit for LimitedSwap and Burstable QoS Pods kubernetes#118764
    • Add Swap stats to /stats/summary and /metrics/resource
      • [KEP-2400] Add swap memory to the Kubelet stats API kubernetes#119425
      • [KEP-2400] Add swap to stats to Summary API and Prometheus endpoints (/stats/summary and /metrics/resource) kubernetes#118865
    • Determine a set of metrics for measuring node performance with swap enabled
      • [KEP-2400] Determine a set of metrics for measuring node performance with swap enabled kubernetes#119424
      • [KEP-2400] Add swap to stats to Summary API and Prometheus endpoints (/stats/summary and /metrics/resource) kubernetes#118865
      • Add a metric for machine swap capacity google/cadvisor#3339
    • Improve test coverage and make sure tests are green
      • [KEP-2400] Swap: improve test coverage and make sure tests are green kubernetes#119430
      • [KEP-2400] Add full cgroup v2 swap support with automatically calculated swap limit for LimitedSwap and Burstable QoS Pods kubernetes#118764
  • Docs (k/website) update(s):
    • Swap graduates to Beta1: add a blog-post website#42009
    • Swap graduates to Beta1: add documentation and update feature gates website#42167
  • Misc
    • Bugfix: MachineInfo Clone() to also clone "SwapCapacity" google/cadvisor#3293
    • Bump cadvisor version to v0.47.3 kubernetes#119225
  • Archive / Outdated
• Beta2
• Future (related PRs, can be assigned to a specific track later)
  • Add the ability to set a system-reserved quantity of swap on a node
    • [KEP-2400] Add the ability to set a system-reserved quantity of swap on a node kubernetes#105019
    • add --system-reserved support for swap(cgroup v2 only): only for burstable pods kubernetes#105271
  • Fail Kubelet at startup if swap is configured with cgroup v1 kubernetes#119327

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

/sig node

[iholder101]

/cc @harche @haircommander @rphillips @SergeyKanzhelev @mrunalp

[ffromani]

/cc

[sftim]

I've given kubernetes/website#42009 a first pass review. I strongly recommend a security review of the documentation we're providing for this newly beta feature. Lots of clusters run with beta features enabled, and users may need to understand the risks.

[pacoxu]

I've given kubernetes/website#42009 a first pass review. I strongly recommend a security review of the documentation we're providing for this newly beta feature. Lots of clusters run with beta features enabled, and users may need to understand the risks.

This beta feature is a little special in that the default FG is off.

NodeSwap: {Default: false, PreRelease: featuregate.Beta},  // 1.28

[sftim]

As soon as it's beta, we can't assume that security-minded folk would disable it out of habit. Instead we really must review the advice to end users. Someone might take the graduation to beta as a signal that it's now safe to enable as documented.

[iholder101]

As soon as it's beta, we can't assume that security-minded folk would disable it out of habit. Instead we really must review the advice to end users. Someone might take the graduation to beta as a signal that it's now safe to enable as documented.

@sftim Bear in mind that Swap is disabled by default in Beta1, and that we address the security both in the KEP and in the documentation.

However, I agree we need to make sure we warn and guide the users correctly.

Thanks!