Skip to content

🐛 fix(addons): avoid infinite reconciliation loop #5458

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

🐛 fix(addons): avoid infinite reconciliation loop #5458

wants to merge 1 commit into from

Conversation

mloiseleur
Copy link

@mloiseleur mloiseleur commented Apr 7, 2025 •
edited
Loading

What type of PR is this?

/kind regression

What this PR does / why we need it:

With latest v2.7.2 and this PR, there is an infinite reconciliation loop for eks addons.

We have seen two problems:

  1. It tries to compare resolveConflicts despite the fact that it's not part of describe-addon output
  2. It fails to compare emptyString ("") (desired when not set) with nil pointer (describe-addon output) on optional Configuration field

Special notes for your reviewer:

Checklist:

  • squashed commits
  • includes documentation
  • includes emoji in title
  • adds unit tests
  • adds or updates e2e tests

Release note:

Fix: Ignore conflict resolution and correctly compare configuration for AddOn reconciliation

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/regression Categorizes issue or PR as related to a regression from a prior release. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-priority labels Apr 7, 2025
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Apr 7, 2025
@mloiseleur
Copy link
Author

mloiseleur commented Apr 7, 2025 •
edited
Loading

I have confirmed this PR fixes the issue on a live cluster.

cc @damdo @richardcase @gutmensch @faiq

I did not test it, but this issue should also affect v2.8.2 release.
I can rebase this PR on master branch if needed.

@mloiseleur mloiseleur force-pushed the fix/eks-addon-reconciliation branch from 8d226cb to 72b584d Compare April 7, 2025 10:30
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Apr 7, 2025
@mloiseleur mloiseleur force-pushed the fix/eks-addon-reconciliation branch 2 times, most recently from 4591066 to 393b45c Compare April 7, 2025 11:54
@faiq
Copy link
Contributor

faiq commented Apr 7, 2025

/test ?

@k8s-ci-robot
Copy link
Contributor

@faiq: The following commands are available to trigger required jobs:

/test pull-cluster-api-provider-aws-build

/test pull-cluster-api-provider-aws-build-docker

/test pull-cluster-api-provider-aws-build-docker-release-2-7

/test pull-cluster-api-provider-aws-build-release-2-7

/test pull-cluster-api-provider-aws-test-release-2-7

/test pull-cluster-api-provider-aws-verify-release-2-7

The following commands are available to trigger optional jobs:

/test pull-cluster-api-provider-aws-apidiff-release-2-7

/test pull-cluster-api-provider-aws-e2e-blocking-release-2-7

/test pull-cluster-api-provider-aws-e2e-conformance-release-2-7

/test pull-cluster-api-provider-aws-e2e-conformance-with-ci-artifacts-release-2-7

/test pull-cluster-api-provider-aws-e2e-eks-gc-release-2-7

/test pull-cluster-api-provider-aws-e2e-eks-release-2-7

/test pull-cluster-api-provider-aws-e2e-eks-testing-release-2-7

/test pull-cluster-api-provider-aws-e2e-release-2-7

Use /test all to run the following jobs that were automatically triggered:

pull-cluster-api-provider-aws-apidiff-release-2-7

pull-cluster-api-provider-aws-build

pull-cluster-api-provider-aws-build-docker

pull-cluster-api-provider-aws-build-docker-release-2-7

pull-cluster-api-provider-aws-build-release-2-7

pull-cluster-api-provider-aws-test-release-2-7

pull-cluster-api-provider-aws-verify-release-2-7

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@faiq
Copy link
Contributor

faiq commented Apr 7, 2025

/test pull-cluster-api-provider-aws-e2e-eks-release-2-7

@faiq
Copy link
Contributor

faiq commented Apr 7, 2025

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Apr 7, 2025
@richardcase
Copy link
Member

I did not test it, but this issue should also affect v2.8.2 release.
I can rebase this PR on master branch if needed.

Thanks for this @mloiseleur :bow.

Would you be able to rebase on `main? We can then cherrypick the change to release-2.8 and release2-7.

@mloiseleur mloiseleur force-pushed the fix/eks-addon-reconciliation branch from 393b45c to aa414e6 Compare April 13, 2025 15:28
@k8s-ci-robot k8s-ci-robot added do-not-merge/contains-merge-commits needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Apr 13, 2025
@mloiseleur mloiseleur changed the base branch from release-2.7 to main April 13, 2025 15:29
@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 13, 2025
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Apr 13, 2025 •
edited
Loading

@mloiseleur: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cluster-api-provider-aws-verify-release-2-7 aa414e6 link true /test pull-cluster-api-provider-aws-verify-release-2-7
pull-cluster-api-provider-aws-build-release-2-7 aa414e6 link true /test pull-cluster-api-provider-aws-build-release-2-7
pull-cluster-api-provider-aws-build-docker-release-2-7 aa414e6 link true /test pull-cluster-api-provider-aws-build-docker-release-2-7
pull-cluster-api-provider-aws-apidiff-release-2-7 aa414e6 link false /test pull-cluster-api-provider-aws-apidiff-release-2-7
pull-cluster-api-provider-aws-test-release-2-7 aa414e6 link true /test pull-cluster-api-provider-aws-test-release-2-7

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Apr 13, 2025
@mloiseleur
Copy link
Author

/retest

@mloiseleur mloiseleur force-pushed the fix/eks-addon-reconciliation branch from aa414e6 to 7633f7f Compare April 13, 2025 15:31
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign nrb for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mloiseleur
Copy link
Author

@richardcase It's rebased. I changed the PR toward main branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dlipovetsky dlipovetsky Awaiting requested review from dlipovetsky

@luthermonson luthermonson Awaiting requested review from luthermonson

Assignees
No one assigned
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/regression Categorizes issue or PR as related to a regression from a prior release. needs-priority ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mloiseleur @faiq @k8s-ci-robot @richardcase