Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky integ test where the SubscribeToShardIntegrationTest #5949

Merged

Conversation

joviegas
Copy link
Contributor

@joviegas joviegas commented Mar 11, 2025

Motivation and Context

  • Test cases fail in @setup() stage
< ERROR! java.util.concurrent.CompletionException: software.amazon.awssdk.services.kinesis.model.ResourceInUseException: 
Stream subscribe-to-shard-integ-test-123456789 under account 123456789 
not ACTIVE or UPDATING, instead in state CREATING 
(Service: Kinesis, Status Code: 400, Request ID: ca299ffa-90c6-9006-aa63-1a1ca6eea3b3, Extended Request ID: ABC
  • The above log indicates that stream is still in CREATING

Modifications

  • Added additional verification by adding a list call to ensure that stream is fully operational

License

  • I confirm that this pull request can be released under the Apache 2 license

@joviegas joviegas requested a review from a team as a code owner March 11, 2025 20:39
asyncClient.listShards(r -> r.streamName(streamName)).join();
return true;
} catch (Exception e) {
if (e.getCause() instanceof ResourceInUseException) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have a Waiter waiting for the ACTIVE state, but we are failing because the stream is in the CREATING stage? That seems weird, are we sure this is helping?

Copy link
Contributor Author

@joviegas joviegas Mar 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeap , I have attached the complete log in the internal issue.
The above log is taken from the codebuild logs.
I have started Integ test too as part of the codebuild

Copy link
Contributor Author

@joviegas joviegas Mar 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The additional check will ensure that the actual state is properly reflected in the system and will provide additional time with retry logic rather than immediately failing(same as we retrying the test case when it fails)
I recommend proceeding with this safe approach since the current failure of this test case is unnecessarily blocking the release and creating significant churn.

@joviegas joviegas enabled auto-merge March 11, 2025 22:07
@joviegas joviegas added this pull request to the merge queue Mar 11, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Mar 11, 2025
@joviegas joviegas added this pull request to the merge queue Mar 11, 2025
Merged via the queue into master with commit b2fe89e Mar 12, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants