OCPBUGS-38869: MCN are not updated when we use OCL #4579

dkhater-redhat · 2024-09-10T21:42:46Z

- What I did
Fixed a bug in updateOnClusterBuild where mcn's were not correctly updating their .spec.configVersion.desired field when using ocl. The fix involved leveraging the GenerateAndApplyMachineConfigNodeSpec function to ensure that the MCNs are correctly updated with the latest desired configuration after applying a new MachineConfig.

- How to verify it

Enable ocl
Apply a new mc that adds or modifies a file on the worker nodes.
Verify that the .spec.configVersion.desired field of mcn's reflects the updated mc across all nodes in the worker pool.

- Description for the changelog
Fixed an issue where mcn's were not updating their .spec.configVersion.desired field correctly when using ocl in the mcd

openshift-ci-robot · 2024-09-10T21:42:52Z

@dkhater-redhat: This pull request references Jira Issue OCPBUGS-38869, which is invalid:

expected the bug to target the "4.18.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

- What I did
Fixed a bug in updateOnClusterBuild where mcn's were not correctly updating their .spec.configVersion.desired field when using ocl. The fix involved leveraging the GenerateAndApplyMachineConfigNodeSpec function to ensure that the MCNs are correctly updated with the latest desired configuration after applying a new MachineConfig.

- How to verify it

Enable ocl

Apply a new mc that adds or modifies a file on the worker nodes.

Verify that the .spec.configVersion.desired field of mcn's reflects the updated mc across all nodes in the worker pool.

- Description for the changelog
Fixed an issue where mcn's were not updating their .spec.configVersion.desired field correctly when using ocl in the mcd

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

dkhater-redhat · 2024-09-11T15:21:47Z

/test e2e-gcp-op-single-node

dkhater-redhat · 2024-09-11T22:00:42Z

/test e2e-gcp-op-single-node

dkhater-redhat · 2024-09-12T22:22:34Z

/test e2e-gcp-op-single-node

pkg/daemon/update.go

umohnani8 · 2024-09-16T16:27:42Z

LGTM

umohnani8 · 2024-09-17T17:01:13Z

/lgtm

sergiordlr · 2024-09-25T18:21:49Z

@dkhater-redhat Hello! We have problems installing the images created for the PRs (all the PRs not only this one). Nevertheless, nightly builds are installing without problems. Could you please merge this PR? we will test it post-merge in the nightly build. Thanks!!

We don't add the qe-approved label since it will verified post-merge.

dkhater-redhat · 2024-09-30T17:07:23Z

/retest-required

dkhater-redhat · 2024-10-02T16:32:05Z

/jira refresh

openshift-ci-robot · 2024-10-02T16:32:09Z

@dkhater-redhat: This pull request references Jira Issue OCPBUGS-38869, which is invalid:

expected the bug to target the "4.18.0" version, but no target version was set
expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

dkhater-redhat · 2024-10-02T16:33:33Z

/jira refresh

openshift-ci-robot · 2024-10-02T16:33:37Z

@dkhater-redhat: This pull request references Jira Issue OCPBUGS-38869, which is invalid:

expected the bug to target only the "4.18.0" version, but multiple target versions were set
expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

dkhater-redhat · 2024-10-02T16:34:28Z

/jira refresh

openshift-ci-robot · 2024-10-02T16:34:32Z

@dkhater-redhat: This pull request references Jira Issue OCPBUGS-38869, which is invalid:

expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

ptalgulk01 · 2024-10-22T07:17:23Z

Pre-merge verified:
Verified using IPI on GCP

Enable OCL

2. Apply any MOSC

oc create -f - << EOF apiVersion: machineconfiguration.openshift.io/v1alpha1 kind: MachineOSConfig metadata: name: check spec: machineConfigPool: name: worker buildOutputs: currentImagePullSecret: name: $(oc get -n openshift-machine-config-operator sa default -ojsonpath='{.secrets[0].name}') buildInputs: imageBuilder: imageBuilderType: PodImageBuilder baseImagePullSecret: name: $(oc get secret -n openshift-config pull-secret -o json | jq "del(.metadata.namespace, .metadata.creationTimestamp, .metadata.resourceVersion, .metadata.uid, .metadata.name)" | jq '.metadata.name="pull-copy"' | oc -n openshift-machine-config-operator create -f - &> /dev/null; echo -n "pull-copy") renderedImagePushSecret: name: $(oc get -n openshift-machine-config-operator sa builder -ojsonpath='{.secrets[0].name}') renderedImagePushspec: "image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/ocb-image:latest" containerFile: - containerfileArch: noarch content: |- # Pull the centos base image and enable the EPEL repository. FROM quay.io/centos/centos:stream9 AS centos RUN dnf install -y epel-release

    # Pull an image containing the yq utility.
    FROM docker.io/mikefarah/yq:latest AS yq

    # Build the final OS image for this MachineConfigPool.
    FROM configs AS final

    # Copy the EPEL configs into the final image.
    COPY --from=yq /usr/bin/yq /usr/bin/yq
    COPY --from=centos /etc/yum.repos.d /etc/yum.repos.d
    COPY --from=centos /etc/pki/rpm-gpg/RPM-GPG-KEY-* /etc/pki/rpm-gpg/

    # Install cowsay and ripgrep from the EPEL repository into the final image,
    # along with a custom cow file.
    RUN sed -i 's/\$stream/9-stream/g' /etc/yum.repos.d/centos*.repo && \
        rpm-ostree install cowsay ripgrep

EOF

Wait for machineosbuild to complete

Apply any MC
Wait for MOSB to be successful
When nodes are updating run below cmd to verify the .spec.confi.gVersion.desired field.

$ oc get node ppt-2201c-76w4j-worker-a-rcc5b  -o jsonpath='{.metadata.annotations.machineconfiguration\.openshift\.io/desiredConfig}' 
rendered-worker-09284a94cf349a37ade57c8bbc08be98
$ oc get machineconfignode ppt-2201c-76w4j-worker-a-rcc5b  -o jsonpath='{.spec.configVersion.desired}'
rendered-worker-09284a94cf349a37ade57c8bbc08be98

Run the below test case -

PASS OCP-69184 Author:rioliu [sig-mco] MCO MachineConfigNode Enable feature gate MachineConfigNodes [Serial]
PASS OCP-69187 Author:rioliu [sig-mco] MCO MachineConfigNode validate MachineConfigNodes properties [Serial]
FAIL OCP-69197 Author:rioliu [sig-mco] MCO MachineConfigNode validate MachineConfigNode condition status transition [Disruptive] [Serial]
FAIL OCP-69205 Author:rioliu [sig-mco] MCO MachineConfigNode MachineConfigNode corresponding condition status is Unknown when node is degraded [Disruptive] [Serial]
PASS OCP-69755 Author:rioliu [sig-mco] MCO MachineConfigNode MachineConfigNode resources should be synced when node is created/deleted [Disruptive] [Serial]
PASS OCP-74644 Author:sregidor [sig-mco] MCO MachineConfigNode Scope each MCN object to only be accessible from its associated MCD [Disruptive] [Serial]

For the failed test-cases reported the bug:
OCPBUGS-43435
OCPBUGS-42550

/label qe-approved

LorbusChris · 2024-10-28T14:43:51Z

pkg/daemon/update.go

@@ -970,6 +970,23 @@ func (dn *Daemon) updateOnClusterBuild(oldConfig, newConfig *mcfgv1.MachineConfi
 		return err
 	}

+	// Determine the pool (worker or master) based on the node's labels


Question: Couldn't this be custom pool (ie non-{worker/master}), too?

MCN technically doesn't yet support custom pools: https://issues.redhat.com/browse/MCO-1299

umohnani8 · 2025-01-07T17:30:51Z

/lgtm

openshift-ci · 2025-01-07T17:31:12Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dkhater-redhat, umohnani8

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [dkhater-redhat,umohnani8]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

isabella-janssen · 2025-01-16T21:43:50Z

/jira refresh

openshift-ci-robot · 2025-01-16T21:43:54Z

@isabella-janssen: This pull request references Jira Issue OCPBUGS-38869, which is invalid:

expected the bug to target either version "4.19." or "openshift-4.19.", but it targets "4.18" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

isabella-janssen · 2025-01-16T21:44:39Z

/jira refresh

openshift-ci-robot · 2025-01-16T21:44:44Z

@isabella-janssen: This pull request references Jira Issue OCPBUGS-38869, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.19.0) matches configured target version for branch (4.19.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira ([email protected]), skipping review request.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

isabella-janssen · 2025-01-16T21:46:55Z

/hold

openshift-ci · 2025-03-06T21:54:42Z

@dkhater-redhat: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-vsphere-ovn-upi	`ea0201f`	link	false	`/test e2e-vsphere-ovn-upi`
ci/prow/e2e-aws-ovn-upgrade-out-of-change	`ea0201f`	link	false	`/test e2e-aws-ovn-upgrade-out-of-change`
ci/prow/e2e-vsphere-ovn-zones	`ea0201f`	link	false	`/test e2e-vsphere-ovn-zones`
ci/prow/e2e-vsphere-ovn-upi-zones	`ea0201f`	link	false	`/test e2e-vsphere-ovn-upi-zones`
ci/prow/e2e-azure-ovn-upgrade-out-of-change	`ea0201f`	link	false	`/test e2e-azure-ovn-upgrade-out-of-change`
ci/prow/e2e-gcp-op	`ea0201f`	link	true	`/test e2e-gcp-op`
ci/prow/e2e-hypershift	`ea0201f`	link	true	`/test e2e-hypershift`
ci/prow/unit	`ea0201f`	link	true	`/test unit`
ci/prow/4.12-upgrade-from-stable-4.11-images	`ea0201f`	link	true	`/test 4.12-upgrade-from-stable-4.11-images`
ci/prow/e2e-gcp-op-single-node	`ea0201f`	link	true	`/test e2e-gcp-op-single-node`
ci/prow/cluster-bootimages	`ea0201f`	link	true	`/test cluster-bootimages`
ci/prow/okd-images	`ea0201f`	link	true	`/test okd-images`
ci/prow/e2e-aws-ovn-upgrade	`ea0201f`	link	true	`/test e2e-aws-ovn-upgrade`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

isabella-janssen · 2025-03-07T13:12:42Z

/close

Closing this PR as it does not support custom MCPs. The idea behind this PR should still work to address the bug, but it will need to be updated once MCO-1501 (PR #4876) is merged.

openshift-ci · 2025-03-07T13:13:18Z

@isabella-janssen: Closed this PR.

In response to this:

/close

Closing this PR as it does not support custom MCPs. The idea behind this PR should still work to address the bug, but it will need to be updated once MCO-1501 (PR #4876) is merged.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci-robot · 2025-03-07T13:13:25Z

@dkhater-redhat: This pull request references Jira Issue OCPBUGS-38869. The bug has been updated to no longer refer to the pull request using the external bug tracker. All external bug links have been closed. The bug has been moved to the NEW state.

In response to this:

- What I did
Fixed a bug in updateOnClusterBuild where mcn's were not correctly updating their .spec.configVersion.desired field when using ocl. The fix involved leveraging the GenerateAndApplyMachineConfigNodeSpec function to ensure that the MCNs are correctly updated with the latest desired configuration after applying a new MachineConfig.

- How to verify it

Enable ocl

Apply a new mc that adds or modifies a file on the worker nodes.

Verify that the .spec.configVersion.desired field of mcn's reflects the updated mc across all nodes in the worker pool.

- Description for the changelog
Fixed an issue where mcn's were not updating their .spec.configVersion.desired field correctly when using ocl in the mcd

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci bot requested review from cheesesashimi and yuqi-zhang September 10, 2024 21:43

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 10, 2024

umohnani8 reviewed Sep 16, 2024

View reviewed changes

pkg/daemon/update.go Outdated Show resolved Hide resolved

ensures mcn spec is updated with correct desiredconfig

d08769f

dkhater-redhat force-pushed the mcn-not-updated-ocl branch from 8a75719 to d08769f Compare September 16, 2024 16:25

openshift-ci bot assigned umohnani8 Sep 17, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 17, 2024

Merge remote-tracking branch 'origin/master' into mcn-not-updated-ocl

ea0201f

openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Oct 18, 2024

openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Oct 22, 2024

LorbusChris reviewed Oct 28, 2024

View reviewed changes

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 7, 2025

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jan 16, 2025

openshift-ci bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 16, 2025

openshift-ci bot closed this Mar 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OCPBUGS-38869: MCN are not updated when we use OCL #4579

OCPBUGS-38869: MCN are not updated when we use OCL #4579

dkhater-redhat commented Sep 10, 2024

openshift-ci-robot commented Sep 10, 2024

dkhater-redhat commented Sep 11, 2024

dkhater-redhat commented Sep 11, 2024

dkhater-redhat commented Sep 12, 2024

umohnani8 commented Sep 16, 2024

umohnani8 commented Sep 17, 2024

sergiordlr commented Sep 25, 2024

dkhater-redhat commented Sep 30, 2024

dkhater-redhat commented Oct 2, 2024

openshift-ci-robot commented Oct 2, 2024

dkhater-redhat commented Oct 2, 2024

openshift-ci-robot commented Oct 2, 2024

dkhater-redhat commented Oct 2, 2024

openshift-ci-robot commented Oct 2, 2024

ptalgulk01 commented Oct 22, 2024

LorbusChris Oct 28, 2024

yuqi-zhang Oct 29, 2024

umohnani8 commented Jan 7, 2025

openshift-ci bot commented Jan 7, 2025

isabella-janssen commented Jan 16, 2025

openshift-ci-robot commented Jan 16, 2025

isabella-janssen commented Jan 16, 2025

openshift-ci-robot commented Jan 16, 2025

isabella-janssen commented Jan 16, 2025

openshift-ci bot commented Mar 6, 2025

isabella-janssen commented Mar 7, 2025

openshift-ci bot commented Mar 7, 2025

openshift-ci-robot commented Mar 7, 2025

OCPBUGS-38869: MCN are not updated when we use OCL #4579

OCPBUGS-38869: MCN are not updated when we use OCL #4579

Conversation

dkhater-redhat commented Sep 10, 2024

openshift-ci-robot commented Sep 10, 2024

dkhater-redhat commented Sep 11, 2024

dkhater-redhat commented Sep 11, 2024

dkhater-redhat commented Sep 12, 2024

umohnani8 commented Sep 16, 2024

umohnani8 commented Sep 17, 2024

sergiordlr commented Sep 25, 2024

dkhater-redhat commented Sep 30, 2024

dkhater-redhat commented Oct 2, 2024

openshift-ci-robot commented Oct 2, 2024

dkhater-redhat commented Oct 2, 2024

openshift-ci-robot commented Oct 2, 2024

dkhater-redhat commented Oct 2, 2024

openshift-ci-robot commented Oct 2, 2024

ptalgulk01 commented Oct 22, 2024

LorbusChris Oct 28, 2024

Choose a reason for hiding this comment

yuqi-zhang Oct 29, 2024

Choose a reason for hiding this comment

umohnani8 commented Jan 7, 2025

openshift-ci bot commented Jan 7, 2025

isabella-janssen commented Jan 16, 2025

openshift-ci-robot commented Jan 16, 2025

isabella-janssen commented Jan 16, 2025

openshift-ci-robot commented Jan 16, 2025

isabella-janssen commented Jan 16, 2025

openshift-ci bot commented Mar 6, 2025

isabella-janssen commented Mar 7, 2025

openshift-ci bot commented Mar 7, 2025

openshift-ci-robot commented Mar 7, 2025