@@ -42,30 +42,31 @@ result field to be present.
42
42
==== {api-request-body-title}
43
43
44
44
`evaluation`::
45
- (Required, object) Defines the type of evaluation you want to perform. The
46
- value of this object can be different depending on the type of evaluation you
47
- want to perform. See <<ml-evaluate-dfanalytics-resources>>.
45
+ (Required, object) Defines the type of evaluation you want to perform.
46
+ See <<ml-evaluate-dfanalytics-resources>>.
48
47
+
49
48
--
50
49
Available evaluation types:
50
+
51
51
* `binary_soft_classification`
52
52
* `regression`
53
53
* `classification`
54
+
54
55
--
55
56
56
57
`index`::
57
58
(Required, object) Defines the `index` in which the evaluation will be
58
59
performed.
59
60
60
61
`query`::
61
- (Optional, object) A query clause that retrieves a subset of data from the
62
+ (Optional, object) A query clause that retrieves a subset of data from the
62
63
source index. See <<query-dsl>>.
63
64
64
65
[[ml-evaluate-dfanalytics-resources]]
65
66
==== {dfanalytics-cap} evaluation resources
66
67
67
68
[[binary-sc-resources]]
68
- ===== Binary soft classification configuration objects
69
+ ===== Binary soft classification evaluation objects
69
70
70
71
Binary soft classification evaluates the results of an analysis which outputs
71
72
the probability that each document belongs to a certain class. For example, in
@@ -86,24 +87,24 @@ document is an outlier.
86
87
(Optional, object) Specifies the metrics that are used for the evaluation.
87
88
Available metrics:
88
89
89
- `auc_roc`::
90
+ `auc_roc`:::
90
91
(Optional, object) The AUC ROC (area under the curve of the receiver
91
92
operating characteristic) score and optionally the curve. Default value is
92
93
{"includes_curve": false}.
93
94
94
- `precision`::
95
+ `confusion_matrix`:::
96
+ (Optional, object) Set the different thresholds of the {olscore} at where
97
+ the metrics (`tp` - true positive, `fp` - false positive, `tn` - true
98
+ negative, `fn` - false negative) are calculated. Default value is
99
+ {"at": [0.25, 0.50, 0.75]}.
100
+
101
+ `precision`:::
95
102
(Optional, object) Set the different thresholds of the {olscore} at where
96
103
the metric is calculated. Default value is {"at": [0.25, 0.50, 0.75]}.
97
104
98
- `recall`::
105
+ `recall`:::
99
106
(Optional, object) Set the different thresholds of the {olscore} at where
100
107
the metric is calculated. Default value is {"at": [0.25, 0.50, 0.75]}.
101
-
102
- `confusion_matrix`::
103
- (Optional, object) Set the different thresholds of the {olscore} at where
104
- the metrics (`tp` - true positive, `fp` - false positive, `tn` - true
105
- negative, `fn` - false negative) are calculated. Default value is
106
- {"at": [0.25, 0.50, 0.75]}.
107
108
108
109
109
110
[[regression-evaluation-resources]]
@@ -121,9 +122,18 @@ which outputs a prediction of values.
121
122
in other words the results of the {regression} analysis.
122
123
123
124
`metrics`::
124
- (Required, object) Specifies the metrics that are used for the evaluation.
125
- Available metrics are `r_squared` and `mean_squared_error`.
126
-
125
+ (Optional, object) Specifies the metrics that are used for the evaluation.
126
+ Available metrics:
127
+
128
+ `mean_squared_error`:::
129
+ (Optional, object) Average squared difference between the predicted values and the actual (`ground truth`) value.
130
+ For more information, read https://en.wikipedia.org/wiki/Mean_squared_error[this wiki article].
131
+
132
+ `r_squared`:::
133
+ (Optional, object) Proportion of the variance in the dependent variable that is predictable from the independent variables.
134
+ For more information, read https://en.wikipedia.org/wiki/Coefficient_of_determination[this wiki article].
135
+
136
+
127
137
128
138
[[classification-evaluation-resources]]
129
139
==== {classification-cap} evaluation objects
@@ -133,20 +143,28 @@ outputs a prediction that identifies to which of the classes each document
133
143
belongs.
134
144
135
145
`actual_field`::
136
- (Required, string) The field of the `index` which contains the ground truth.
137
- The data type of this field must be keyword.
138
-
139
- `metrics`::
140
- (Required, object) Specifies the metrics that are used for the evaluation.
141
- Available metric is `multiclass_confusion_matrix`.
146
+ (Required, string) The field of the `index` which contains the `ground truth`.
147
+ The data type of this field must be categorical.
142
148
143
149
`predicted_field`::
144
150
(Required, string) The field in the `index` that contains the predicted value,
145
- in other words the results of the {classanalysis}. The data type of this field
146
- is string. You need to add `.keyword` to the predicted field name (the name
147
- you put in the {classanalysis} object as `prediction_field_name` or the
148
- default value of the same field if you didn't specified explicitly). For
149
- example, `predicted_field` : `ml.animal_class_prediction.keyword`.
151
+ in other words the results of the {classanalysis}.
152
+
153
+ `metrics`::
154
+ (Optional, object) Specifies the metrics that are used for the evaluation.
155
+ Available metrics:
156
+
157
+ `accuracy`:::
158
+ (Optional, object) Accuracy of predictions (per-class and overall).
159
+
160
+ `multiclass_confusion_matrix`:::
161
+ (Optional, object) Multiclass confusion matrix.
162
+
163
+ `precision`:::
164
+ (Optional, object) Precision of predictions (per-class and average).
165
+
166
+ `recall`:::
167
+ (Optional, object) Recall of predictions (per-class and average).
150
168
151
169
152
170
////
@@ -359,7 +377,7 @@ POST _ml/data_frame/_evaluate
359
377
"evaluation": {
360
378
"classification": { <1>
361
379
"actual_field": "animal_class", <2>
362
- "predicted_field": "ml.animal_class_prediction.keyword ", <3>
380
+ "predicted_field": "ml.animal_class_prediction", <3>
363
381
"metrics": {
364
382
"multiclass_confusion_matrix" : {} <4>
365
383
}
@@ -373,8 +391,7 @@ POST _ml/data_frame/_evaluate
373
391
<2> The field that contains the ground truth value for the actual animal
374
392
classification. This is required in order to evaluate results.
375
393
<3> The field that contains the predicted value for animal classification by
376
- the {classanalysis}. Since the field storing predicted class is dynamically
377
- mapped as text and keyword, you need to add the `.keyword` suffix to the name.
394
+ the {classanalysis}.
378
395
<4> Specifies the metric for the evaluation.
379
396
380
397
0 commit comments