Skip to content

Commit 5c77ebe

Browse files
[ML] Implement new rules design (#31110)
Rules allow users to supply a detector with domain knowledge that can improve the quality of the results. The model detects statistically anomalous results but it has no knowledge of the meaning of the values being modelled. For example, a detector that performs a population analysis over IP addresses could benefit from a list of IP addresses that the user knows to be safe. Then anomalous results for those IP addresses will not be created and will not affect the quantiles either. Another example would be a detector looking for anomalies in the median value of CPU utilization. A user might want to inform the detector that any results where the actual value is less than 5 is not interesting. This commit introduces a `custom_rules` field to the `Detector`. A detector may have multiple rules which are combined with `or`. A rule has 3 fields: `actions`, `scope` and `conditions`. Actions is a list of what should happen when the rule applies. The current options include `skip_result` and `skip_model_update`. The default value for `actions` is the `skip_result` action. Scope is optional and allows for applying filters on any of the partition/over/by field. When not defined the rule applies to all series. The `filter_id` needs to be specified to match the id of the filter to be used. Optionally, the `filter_type` can be specified as either `include` (default) or `exclude`. When set to `include` the rule applies to entities that are in the filter. When set to `exclude` the rule only applies to entities not in the filter. There may be zero or more conditions. A condition requires `applies_to`, `operator` and `value` to be specified. The `applies_to` value can be either `actual`, `typical` or `diff_from_typical` and it specifies the numerical value to which the condition applies. The `operator` (`lt`, `lte`, `gt`, `gte`) and `value` complete the definition. Conditions are combined with `and` and allow to specify numerical conditions for when a rule applies. A rule must either have a scope or one or more conditions. Finally, a rule with scope and conditions applies when all of them apply.
1 parent eb85852 commit 5c77ebe

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+1062
-1801
lines changed

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ml/calendars/ScheduledEvent.java

+1-3
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,6 @@
1616
import org.elasticsearch.common.xcontent.XContentBuilder;
1717
import org.elasticsearch.common.xcontent.XContentParser;
1818
import org.elasticsearch.xpack.core.ml.MlMetaIndex;
19-
import org.elasticsearch.xpack.core.ml.job.config.Connective;
2019
import org.elasticsearch.xpack.core.ml.job.config.DetectionRule;
2120
import org.elasticsearch.xpack.core.ml.job.config.Operator;
2221
import org.elasticsearch.xpack.core.ml.job.config.RuleAction;
@@ -148,8 +147,7 @@ public DetectionRule toDetectionRule(TimeValue bucketSpan) {
148147
conditions.add(RuleCondition.createTime(Operator.LT, bucketEndTime));
149148

150149
DetectionRule.Builder builder = new DetectionRule.Builder(conditions);
151-
builder.setActions(RuleAction.FILTER_RESULTS, RuleAction.SKIP_SAMPLING);
152-
builder.setConditionsConnective(Connective.AND);
150+
builder.setActions(RuleAction.SKIP_RESULT, RuleAction.SKIP_MODEL_UPDATE);
153151
return builder.build();
154152
}
155153

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ml/job/config/Condition.java

-132
This file was deleted.

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ml/job/config/Connective.java

-42
This file was deleted.

0 commit comments

Comments
 (0)