binding-infra-node-workloads-using-taints-tolerations.adoc

Binding infrastructure node workloads using taints and tolerations

If you have an infra node that has the infra and worker roles assigned, you must configure the node so that user workloads are not assigned to it.

Important

It is recommended that you preserve the dual infra,worker label that is created for infra nodes and use taints and tolerations to manage nodes that user workloads are scheduled on. If you remove the worker label from the node, you must create a custom pool to manage it. A node with a label other than master or worker is not recognized by the MCO without a custom pool. Maintaining the worker label allows the node to be managed by the default worker machine config pool, if no custom pools that select the custom label exists. The infra label communicates to the cluster that it does not count toward the total number of subscriptions.

Prerequisites

• Configure additional MachineSet objects in your {product-title} cluster.

Procedure

1. Add a taint to the infra node to prevent scheduling user workloads on it:

   1. Determine if the node has the taint:

      $ oc describe nodes <node_name>
      Sample output

      oc describe node ci-ln-iyhx092-f76d1-nvdfm-worker-b-wln2l
      Name:               ci-ln-iyhx092-f76d1-nvdfm-worker-b-wln2l
      Roles:              worker
       ...
      Taints:             node-role.kubernetes.io/infra:NoSchedule
       ...

      This example shows that the node has a taint. You can proceed with adding a toleration to your pod in the next step.

   2. If you have not configured a taint to prevent scheduling user workloads on it:

      $ oc adm taint nodes <node_name> <key>:<effect>

      For example:

      $ oc adm taint nodes node1 node-role.kubernetes.io/infra:NoSchedule

      Tip	You can alternatively apply the following YAML to add the taint: kind: Node apiVersion: v1 metadata: name: <node_name> labels: ... spec: taints: - key: node-role.kubernetes.io/infra effect: NoSchedule ...

      This example places a taint on node1 that has key node-role.kubernetes.io/infra and taint effect NoSchedule. Nodes with the NoSchedule effect schedule only pods that tolerate the taint, but allow existing pods to remain scheduled on the node.

      Note	If a descheduler is used, pods violating node taints could be evicted from the cluster.

2. Add tolerations for the pod configurations you want to schedule on the infra node, like router, registry, and monitoring workloads. Add the following code to the Pod object specification:

   tolerations:
     - effect: NoSchedule (1)
       key: node-role.kubernetes.io/infra (2)
       operator: Exists (3)

   1. Specify the effect that you added to the node.

   2. Specify the key that you added to the node.

   3. Specify the Exists Operator to require a taint with the key node-role.kubernetes.io/infra to be present on the node.

      This toleration matches the taint created by the oc adm taint command. A pod with this toleration can be scheduled onto the infra node.

      Note	Moving pods for an Operator installed via OLM to an infra node is not always possible. The capability to move Operator pods depends on the configuration of each Operator.

3. Schedule the pod to the infra node using a scheduler. See the documentation for Controlling pod placement onto nodes for details.