Skip to content

OCPBUGS-27961 Procedure for configuring Whereabouts IP reconciler schedule #71081

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
// Module included in the following assemblies:
//
// * networking/multiple_networks/configuring-additional-network.adoc

:_mod-docs-content-type: PROCEDURE

[id="nw-multus-configuring-whereabouts-ip-reconciler-schedule_{context}"]
= Configuring the Whereabouts IP reconciler schedule

The Whereabouts IPAM CNI plugin runs the IP reconciler daily. This process cleans up any stranded IP allocations that might result in exhausting IPs and therefore prevent new pods from getting an IP allocated to them.

Use this procedure to change the frequency at which the IP reconciler runs.

.Prerequisites
* You installed the OpenShift CLI (`oc`).
* You have access to the cluster as a user with the `cluster-admin` role.
* You have deployed the `whereabouts-reconciler` daemon set, and the `whereabouts-reconciler` pods are up and running.

.Procedure

. Run the following command to create a `ConfigMap` object named `whereabouts-config` in the `openshift-multus` namespace with a specific cron expression for the IP reconciler:
+
[source,terminal]
----
$ oc create configmap whereabouts-config -n openshift-multus --from-literal=reconciler_cron_expression="*/15 * * * *"
----
+
This cron expression indicates the IP reconciler runs every 15 minutes. Adjust the expression based on your specific requirements.
+
[NOTE]
====
The `whereabouts-reconciler` daemon set can only consume a cron expression pattern that includes five asterisks. The sixth, which is used to denote seconds, is currently not supported.
====

. Retrieve information about resources related to the `whereabouts-reconciler` daemon set and pods within the `openshift-multus` namespace by running the following command:
+
[source,terminal]
----
$ oc get all -n openshift-multus | grep whereabouts-reconciler
----
+
.Example output
+
[source,terminal]
----
pod/whereabouts-reconciler-2p7hw 1/1 Running 0 4m14s
pod/whereabouts-reconciler-76jk7 1/1 Running 0 4m14s
pod/whereabouts-reconciler-94zw6 1/1 Running 0 4m14s
pod/whereabouts-reconciler-mfh68 1/1 Running 0 4m14s
pod/whereabouts-reconciler-pgshz 1/1 Running 0 4m14s
pod/whereabouts-reconciler-xn5xz 1/1 Running 0 4m14s
daemonset.apps/whereabouts-reconciler 6 6 6 6 6 kubernetes.io/os=linux 4m16s
----

. Run the following command to verify that the `whereabouts-reconciler` pod runs the IP reconciler with the configured interval:
+
[source,terminal]
----
$ oc -n openshift-multus logs whereabouts-reconciler-2p7hw
----
+
.Example output

[source,terminal]
----
2024-02-02T16:33:54Z [debug] event not relevant: "/cron-schedule/..2024_02_02_16_33_54.1375928161": CREATE
2024-02-02T16:33:54Z [debug] event not relevant: "/cron-schedule/..2024_02_02_16_33_54.1375928161": CHMOD
2024-02-02T16:33:54Z [debug] event not relevant: "/cron-schedule/..data_tmp": RENAME
2024-02-02T16:33:54Z [verbose] using expression: */15 * * * *
2024-02-02T16:33:54Z [verbose] configuration updated to file "/cron-schedule/..data". New cron expression: */15 * * * *
2024-02-02T16:33:54Z [verbose] successfully updated CRON configuration id "00c2d1c9-631d-403f-bb86-73ad104a6817" - new cron expression: */15 * * * *
2024-02-02T16:33:54Z [debug] event not relevant: "/cron-schedule/config": CREATE
2024-02-02T16:33:54Z [debug] event not relevant: "/cron-schedule/..2024_02_02_16_26_17.3874177937": REMOVE
2024-02-02T16:45:00Z [verbose] starting reconciler run
2024-02-02T16:45:00Z [debug] NewReconcileLooper - inferred connection data
2024-02-02T16:45:00Z [debug] listing IP pools
2024-02-02T16:45:00Z [debug] no IP addresses to cleanup
2024-02-02T16:45:00Z [verbose] reconciler success
----
76 changes: 76 additions & 0 deletions modules/nw-multus-creating-whereabouts-reconciler-daemon-set.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
// Module included in the following assemblies:
//
// * networking/multiple_networks/configuring-additional-network.adoc

:_mod-docs-content-type: PROCEDURE

[id="nw-multus-creating-whereabouts-reconciler-daemon-set_{context}"]
= Creating a whereabouts-reconciler daemon set

The Whereabouts reconciler is responsible for managing dynamic IP address assignments for the pods within a cluster using the Whereabouts IP Address Management (IPAM) solution. It ensures that each pod gets a unique IP address from the specified IP address range. It also handles IP address releases when pods are deleted or scaled down.

[NOTE]
====
You can also use a `NetworkAttachmentDefinition` custom resource (CR) for dynamic IP address assignment.
====

The `whereabouts-reconciler` daemon set is automatically created when you configure an additional network through the Cluster Network Operator. It is not automatically created when you configure an additional network from a YAML manifest.

To trigger the deployment of the `whereabouts-reconciler` daemon set, you must manually create a `whereabouts-shim` network attachment by editing the Cluster Network Operator custom resource (CR) file.

Use the following procedure to deploy the `whereabouts-reconciler` daemon set.

.Procedure

. Edit the `Network.operator.openshift.io` custom resource (CR) by running the following command:
+
[source,terminal]
----
$ oc edit network.operator.openshift.io cluster
----

. Modify the `additionalNetworks` parameter in the CR to add the `whereabouts-shim` network attachment definition. For example:
+
[source,yaml]
----
apiVersion: operator.openshift.io/v1
kind: Network
metadata:
name: cluster
spec:
additionalNetworks:
- name: whereabouts-shim
namespace: default
rawCNIConfig: |-
{
"name": "whereabouts-shim",
"cniVersion": "0.3.1",
"type": "bridge",
"ipam": {
"type": "whereabouts"
}
}
type: Raw
----

. Save the file and exit the text editor.

. Verify that the `whereabouts-reconciler` daemon set deployed successfully by running the following command:
+
[source,terminal]
----
$ oc get all -n openshift-multus | grep whereabouts-reconciler
----
+
.Example output
+
[source,terminal]
----
pod/whereabouts-reconciler-jnp6g 1/1 Running 0 6s
pod/whereabouts-reconciler-k76gg 1/1 Running 0 6s
pod/whereabouts-reconciler-k86t9 1/1 Running 0 6s
pod/whereabouts-reconciler-p4sxw 1/1 Running 0 6s
pod/whereabouts-reconciler-rvfdv 1/1 Running 0 6s
pod/whereabouts-reconciler-svzw9 1/1 Running 0 6s
daemonset.apps/whereabouts-reconciler 6 6 6 6 6 kubernetes.io/os=linux 6s
----
73 changes: 1 addition & 72 deletions modules/nw-multus-ipam-object.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -239,75 +239,4 @@ Whereabouts can be used for both IPv4 and IPv6 addresses.

ifdef::sr-iov[]
:!sr-iov:
endif::[]

[id="nw-multus-creating-whereabouts-reconciler-daemon-set_{context}"]
== Creating a Whereabouts reconciler daemon set

The Whereabouts reconciler is responsible for managing dynamic IP address assignments for the pods within a cluster using the Whereabouts IP Address Management (IPAM) solution. It ensures that each pods gets a unique IP address from the specified IP address range. It also handles IP address releases when pods are deleted or scaled down.

[NOTE]
====
You can also use a `NetworkAttachmentDefinition` custom resource for dynamic IP address assignment.
====

The Whereabouts reconciler daemon set is automatically created when you configure an additional network through the Cluster Network Operator. It is not automatically created when you configure an additional network from a YAML manifest.

To trigger the deployment of the Whereabouts reconciler daemonset, you must manually create a `whereabouts-shim` network attachment by editing the Cluster Network Operator custom resource file.

Use the following procedure to deploy the Whereabouts reconciler daemonset.

.Procedure

. Edit the `Network.operator.openshift.io` custom resource (CR) by running the following command:
+
[source,terminal]
----
$ oc edit network.operator.openshift.io cluster
----

. Modify the `additionalNetworks` parameter in the CR to add the `whereabouts-shim` network attachment definition. For example:
+
[source,yaml]
----
apiVersion: operator.openshift.io/v1
kind: Network
metadata:
name: cluster
spec:
additionalNetworks:
- name: whereabouts-shim
namespace: default
rawCNIConfig: |-
{
"name": "whereabouts-shim",
"cniVersion": "0.3.1",
"type": "bridge",
"ipam": {
"type": "whereabouts"
}
}
type: Raw
----

. Save the file and exit the text editor.

. Verify that the `whereabouts-reconciler` daemon set deployed successfully by running the following command:
+
[source,terminal]
----
$ oc get all -n openshift-multus | grep whereabouts-reconciler
----
+
.Example output
+
[source,terminal]
----
pod/whereabouts-reconciler-jnp6g 1/1 Running 0 6s
pod/whereabouts-reconciler-k76gg 1/1 Running 0 6s
pod/whereabouts-reconciler-k86t9 1/1 Running 0 6s
pod/whereabouts-reconciler-p4sxw 1/1 Running 0 6s
pod/whereabouts-reconciler-rvfdv 1/1 Running 0 6s
pod/whereabouts-reconciler-svzw9 1/1 Running 0 6s
daemonset.apps/whereabouts-reconciler 6 6 6 6 6 kubernetes.io/os=linux 6s
----
endif::[]
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,10 @@ include::modules/configuring-pods-static-ip.adoc[leveloffset=+3]

include::modules/nw-multus-ipam-object.adoc[leveloffset=+1]

include::modules/nw-multus-creating-whereabouts-reconciler-daemon-set.adoc[leveloffset=+2]

include::modules/nw-multus-configuring-whereabouts-ip-reconciler-schedule.adoc[leveloffset=+2]

include::modules/nw-multus-configure-dualstack-ip-address.adoc[leveloffset=+2]

[role="_additional-resources"]
Expand Down