Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding additional wait to avoid eventual consistency issues #6006

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

RanVaknin
Copy link
Contributor

Motivation and Context

The Kinesis stability test is failing often with the following error:

ResourceInUseException: Stream kinesisstabilitytest1743709690580 under account 990724664260 not ACTIVE or UPDATING, instead in state CREATING

This is despite the fact we are using a waiter that is supposed to verify that the stream is active before operating on it.
Adding logs reveals an eventual consistency issue with Kinesis where the stream state has no propagated server side and causes this exception:

Checking stream status at 3764ms
Current stream status: ACTIVE at 3850ms
Stream kinesisstabilitytest1743709690580 is now ACTIVE after 3851ms
ResourceInUseException: Stream kinesisstabilitytest1743709690580 under account 990724664260 not ACTIVE or UPDATING, instead in state CREATING

@joviegas has run in the same problem with the KinesisintegTest, so I borrowed his solution and it seems to have fixed the problem. #5949

Testing

Running this test with a @RepeatedTest(20) and observed a 25% failure rate. After adding the additional waiting I see a 0% failure rate.

@RanVaknin RanVaknin requested a review from a team as a code owner April 3, 2025 22:29
Copy link

sonarqubecloud bot commented Apr 3, 2025

Quality Gate Failed Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

@RanVaknin RanVaknin added this pull request to the merge queue Apr 4, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Apr 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants