[7.x][ML] Ensure program counters cache always cleared (#1774) #1778
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The collection of static program counters is cached prior to persistence. This provides the background persistence thread access to a consistent set of counters as they are being written. As it is desired to only persist the program counters the once for each model state snapshot, their persistence, and the clearing of the cache, is coupled to the persistence of the simple count detector, which is assumed to always exist. However there is a scenario where persistence operates on an empty collection of detectors. This occurs when no data has been seen but time has advanced (see elastic#393 for more details). In this situation the program counter cache is populated but not cleared. A subsequent persistence operation will lead to a warning that the counter cache is being overwritten. To avoid the warning message, we take the approach of ensuring that the program counter cache is always cleared at the end of the persistence operation, regardless of its success or not.
|
The Windows CI build has failed due to a known issue unrelated to the changes included in this PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The collection of static program counters is cached prior to
persistence. This provides the background persistence thread access to a
consistent set of counters as they are being written.
As it is desired to only persist the program counters the once for each
model state snapshot, their persistence, and the clearing of the cache,
is coupled to the persistence of the simple count detector, which is
assumed to always exist.
However there is a scenario where persistence operates on an empty
collection of detectors. This occurs when no data has been seen but time
has advanced (see #393 for more details). In this situation the program
counter cache is populated but not cleared. A subsequent persistence
operation will lead to a warning that the counter cache is being
overwritten.
To avoid the warning message, we take the approach of ensuring
that the program counter cache is always cleared at the end of the
persistence operation, regardless of its success or not.
Backports #1774