[ML] Report the "actual" memory usage of the autodetect process #2846
Conversation
Determine the actual memory usgae of the autodetect process as reported by the OS, e.g. on Linux this mould be the value of the maximum resident set size returned by a call to `getrusage`. Add this value to the model size stats record returned to the ES Java process so it can be included in the `job counts` tab for anomaly detection jobs.
There was a problem hiding this comment.
Hi Ed. I did the first pass.
We should discuss the naming of the new field. While "actual" conveys the intention of the value, it is confusing to the user.
Also, does maximum resident set size actually correspond to the actual current memory usage or is it the historical peak process memory usage?
include/model/CResourceMonitor.h
Outdated
| @@ -180,6 +181,8 @@ class MODEL_EXPORT CResourceMonitor { | |||
| //! Returns the sum of used memory plus any extra memory | |||
| std::size_t totalMemory() const; | |||
|
|
|||
| std::size_t actualMemoryUsage() const; | |||
There was a problem hiding this comment.
I think we can come up with something better than actualMemoryUsage. Maybe: systemMemoryUsage?
The resident set size (RSS) represents the process's current RAM usage (so not counting pages that have been swapped out etc.), and the max RSS is the high water mark of that value. I think that reporting both would be useful for our purposes. |
* ActualMemory -> SystemMemory * Report current resident set size as well as max
include/model/ModelTypes.h
Outdated
| E_AssignmentBasisSystemMemoryBytes = 4, //!< Use the current system memory size | ||
| E_AssignmentBasisMaxSystemMemoryBytes = 5 //!< Use the highest ever system memory size |
There was a problem hiding this comment.
I don't think we need this assignment basis reasons (at least so far)
bin/autodetect/Main.cc
Outdated
| ml::counter_t::E_TSADResidentSetSize, | ||
| ml::counter_t::E_TSADMaxResidentSetSize}; |
There was a problem hiding this comment.
Should we call these counters SystemMemoryUsage for consistency?
lib/model/ModelTypes.cc
Outdated
| case E_AssignmentBasisSystemMemoryBytes: | ||
| return "system_memory_bytes"; | ||
| case E_AssignmentBasisMaxSystemMemoryBytes: | ||
| return "max_system_memory_bytes"; |
Co-authored-by: Valeriy Khakhutskyy <[email protected]>
Co-authored-by: Valeriy Khakhutskyy <[email protected]>
Co-authored-by: Valeriy Khakhutskyy <[email protected]>
…mem_usage # Conflicts: # bin/autodetect/Main.cc # include/model/CResourceMonitor.h
|
Determine the actual memory usage of the autodetect process as reported by the OS, e.g. on Linux this would be the value of the maximum resident set size returned by a call to
getrusage.Add this value to the model size stats record returned to the ES Java process so it can be included in the
job countstab for anomaly detection jobs.Relates elastic/elasticsearch#126256