[ML] No error when datafeed stops during updating to started #46495
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Investigating the test failure reported in elastic#45518 it appears that the datafeed task was not found during a tast state update. There are only two places where such an update is performed: when we set the state to `started` and when we set it to `stopping`. We handle `ResourceNotFoundException` in the latter but not in the former. Thus the test reveals a rare race condition where the datafeed gets requested to stop before we managed to update its state to `started`. I could not reproduce this scenario but it would be my best guess. This commit catches `ResourceNotFoundException` while updating the state to `started` and lets the task terminate smoothly. Closes elastic#45518
dimitris-athanasiou
added
>test
|
Pinging @elastic/ml-core |
dimitris-athanasiou
deleted the
no-error-when-datafeed-stops-before-updating-to-started
branch
dimitris-athanasiou
added a commit
to dimitris-athanasiou/elasticsearch
that referenced
this pull request
Sep 10, 2019
…astic#46495) Investigating the test failure reported in elastic#45518 it appears that the datafeed task was not found during a tast state update. There are only two places where such an update is performed: when we set the state to `started` and when we set it to `stopping`. We handle `ResourceNotFoundException` in the latter but not in the former. Thus the test reveals a rare race condition where the datafeed gets requested to stop before we managed to update its state to `started`. I could not reproduce this scenario but it would be my best guess. This commit catches `ResourceNotFoundException` while updating the state to `started` and lets the task terminate smoothly. Closes elastic#45518 Backport of elastic#46495
dimitris-athanasiou
added a commit
that referenced
this pull request
Sep 11, 2019
…6495) (#46542) Investigating the test failure reported in #45518 it appears that the datafeed task was not found during a tast state update. There are only two places where such an update is performed: when we set the state to `started` and when we set it to `stopping`. We handle `ResourceNotFoundException` in the latter but not in the former. Thus the test reveals a rare race condition where the datafeed gets requested to stop before we managed to update its state to `started`. I could not reproduce this scenario but it would be my best guess. This commit catches `ResourceNotFoundException` while updating the state to `started` and lets the task terminate smoothly. Closes #45518 Backport of #46495
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Investigating the test failure reported in #45518 it appears that
the datafeed task was not found during a tast state update. There
are only two places where such an update is performed: when we set
the state to
startedand when we set it tostopping. We handleResourceNotFoundExceptionin the latter but not in the former.Thus the test reveals a rare race condition where the datafeed gets
requested to stop before we managed to update its state to
started.I could not reproduce this scenario but it would be my best guess.
This commit catches
ResourceNotFoundExceptionwhile updating thestate to
startedand lets the task terminate smoothly.Closes #45518