Skip to content

Fix incorrectly added functions. #4317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

baezzys
Copy link
Contributor

@baezzys baezzys commented Feb 28, 2023

Correct that hashcode and equals were added in the chunk iterator class instead of in the chunk class.

This PR is related to #4314

@baezzys baezzys force-pushed the Fix_adding_an_incorrectly_added_function branch from f3a85b1 to 3e9f48d Compare February 28, 2023 04:49
@fmbenhassine
Copy link
Contributor

Thank you for your PR. While testing it, I noticed there is test failure in FaultTolerantChunkProcessorTests.testWriteRetryOnTwoExceptions. Can you check please?

Also , please run ./mvnw spring-javaformat:apply to fix code formatting as well.

@fmbenhassine fmbenhassine linked an issue Mar 21, 2023 that may be closed by this pull request
@fmbenhassine fmbenhassine added the status: waiting-for-reporter Issues for which we are waiting for feedback from the reporter label Mar 21, 2023
@baezzys baezzys force-pushed the Fix_adding_an_incorrectly_added_function branch from a4cb5dd to a80f0a7 Compare March 24, 2023 15:39
@baezzys
Copy link
Contributor Author

baezzys commented Mar 24, 2023

I'll fix it

@baezzys
Copy link
Contributor Author

baezzys commented Apr 21, 2023

@fmbenhassine

This is a problem related to Spring retry.

    private void registerContext(RetryContext context, RetryState state) {
        if (state != null) {
            Object key = state.getKey();
            if (key != null) {
                if (context.getRetryCount() > 1 && !this.retryContextCache.containsKey(key)) {
                    throw new RetryException("Inconsistent state for failed item key: cache key has changed. Consider whether equals() or hashCode() for the key might be inconsistent, or if you need to supply a better key");
                }

                this.retryContextCache.put(key, context);
            }
        }

    }

In this code, the chunk class is being used as the key value in the retryContextCache.

At the point where the error occurs, the item of class chunk has changed. Because this changes the hash code, it goes inside the if conditional, unlike before.

That's why the error occurs in the 4th assertEquals method of testWriteRetryOnTwoExceptions().

This is because it throws a RetryException, which is not the expected error.

How would you suggest fixing it, exclude the item from the hash code and equals? Or would it be better to modify the test code?

@baezzys baezzys force-pushed the Fix_adding_an_incorrectly_added_function branch from a80f0a7 to 736eb9e Compare April 21, 2023 06:28
@fmbenhassine
Copy link
Contributor

Thank you for tracing the issue down to Spring Retry!

In Spring Batch, failed items are retried without any modification (it does not make sense to change an item before retrying it). So if the test changes the items before retrying them, then the test is incorrect and should be fixed. Do not hesitate to adapt it as needed. Let me know if you need help on this. Thank you!

@baezzys
Copy link
Contributor Author

baezzys commented Apr 26, 2023

Thank you for your review. I'll try to modify the test code according to your idea.

@baezzys
Copy link
Contributor Author

baezzys commented Apr 29, 2023

Done.

@baezzys baezzys force-pushed the Fix_adding_an_incorrectly_added_function branch from 7e81eba to e694e89 Compare May 2, 2023 13:50
@fmbenhassine fmbenhassine added status: feedback-provided Issues for which the feedback requested from the reporter was provided and removed status: waiting-for-reporter Issues for which we are waiting for feedback from the reporter labels May 3, 2023
@fmbenhassine fmbenhassine reopened this May 3, 2023
@fmbenhassine
Copy link
Contributor

The test is actually correct. Your changes of the test makes it inconsistent with the test name.

This is actually deeper than I thought. The issue is not with Spring Retry. The issue is with Spring Batch, which is using the Chunk itself as a key in the HashMap-based retry context cache here. This is different from another code path where items themselves are used as keys in the context here.

So we are in a case where the Chunk is used as a key of the HashMap of the retry context, and which is changing on retry attempts (debugging the case shows that the chunk is shrinking one item at a time when scanning is performed). Therefore, the hash code is changing on each attempt which leads to the error.

The correct way to fix this issue is by using items in the retry context, not the Chunk itself. This makes all the tests of FaultTolerantChunkProcessorTests pass without any modification, but unfortunately makes some other tests to fail.

At first, I thought this is related to the item buffering feature, but it is not (the test in question is still failing with or without buffering after adding equals/hashcode to Chunk). So I believe this is an old hidden bug that is now popping up after adding equals and hashcode to the Chunk class. For that reason, I created #4370 to dig further and fix the issue correctly.

For your PR, I took the commit that adds equals/hashcode, but I omitted the commit that changes the test (because the test is correct as explained previously).

Thank you for your contribution!

@fmbenhassine
Copy link
Contributor

Rebased and merged as 30e8766.

@fmbenhassine fmbenhassine removed the status: feedback-provided Issues for which the feedback requested from the reporter was provided label May 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement equals and hashCode in Chunk class
2 participants