Skip to content

Commit 4efd680

Browse files
authored
[ML] Do not write model size stats after autodetect no-op (#516)
Similar to the fix for state and quantiles in #437, if no input is received and time is not advanced then there is no need to write model size stats when the autodetect process exits. Doing this can actually cause a problem for a job that has never ever seen any input, as the unnecessary model size stats were written with a negative timestamp. This change also adds an extra defensive check to prevent that ever happening, although the only situation when it is thought to be possible should be prevented by the first change. Backport of #512
1 parent f0ae0b2 commit 4efd680

File tree

2 files changed

+17
-5
lines changed

2 files changed

+17
-5
lines changed

docs/CHANGELOG.asciidoc

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,12 @@ to the model. (See {ml-pull}214[#214].)
6161
6262
== {es} version 7.0.0-alpha1
6363
64+
== {es} version 6.8.2
65+
66+
=== Bug Fixes
67+
68+
* Don't write model size stats when job is closed without any input {ml-pull}512[#512] (issue: {ml-issue}394[#394])
69+
6470
== {es} version 6.7.2
6571
6672
=== Enhancements

lib/api/CAnomalyJob.cc

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -221,14 +221,14 @@ bool CAnomalyJob::handleRecord(const TStrStrUMap& dataRowFields) {
221221

222222
void CAnomalyJob::finalise() {
223223
// Persist final state of normalizer iff an input record has been handled or time has been advanced.
224-
if (this->isPersistenceNeeded("quantiles state")) {
224+
if (this->isPersistenceNeeded("quantiles state and model size stats")) {
225225
m_JsonOutputWriter.persistNormalizer(m_Normalizer, m_LastNormalizerPersistTime);
226-
}
227226

228-
// Prune the models so that the final persisted state is as neat as possible
229-
this->pruneAllModels();
227+
// Prune the models so that the final persisted state is as neat as possible
228+
this->pruneAllModels();
230229

231-
this->refreshMemoryAndReport();
230+
this->refreshMemoryAndReport();
231+
}
232232

233233
// Wait for any ongoing periodic persist to complete, so that the data adder
234234
// is not used by both a periodic periodic persist and final persist at the
@@ -1267,6 +1267,12 @@ void CAnomalyJob::writeOutModelPlot(const TModelPlotDataVec& modelPlotData) {
12671267
}
12681268

12691269
void CAnomalyJob::refreshMemoryAndReport() {
1270+
if (m_LastFinalisedBucketEndTime < m_ModelConfig.bucketLength()) {
1271+
LOG_ERROR(<< "Cannot report memory usage because last finalized bucket end time ("
1272+
<< m_LastFinalisedBucketEndTime << ") is smaller than bucket span ("
1273+
<< m_ModelConfig.bucketLength() << ')');
1274+
return;
1275+
}
12701276
// Make sure model size stats are up to date and then send a final memory
12711277
// usage report
12721278
for (const auto& detector_ : m_Detectors) {

0 commit comments

Comments
 (0)