@@ -43,30 +43,31 @@ result field to be present.
43
43
==== {api-request-body-title}
44
44
45
45
`evaluation`::
46
- (Required, object) Defines the type of evaluation you want to perform. The
47
- value of this object can be different depending on the type of evaluation you
48
- want to perform. See <<ml-evaluate-dfanalytics-resources>>.
46
+ (Required, object) Defines the type of evaluation you want to perform.
47
+ See <<ml-evaluate-dfanalytics-resources>>.
49
48
+
50
49
--
51
50
Available evaluation types:
51
+
52
52
* `binary_soft_classification`
53
53
* `regression`
54
54
* `classification`
55
+
55
56
--
56
57
57
58
`index`::
58
59
(Required, object) Defines the `index` in which the evaluation will be
59
60
performed.
60
61
61
62
`query`::
62
- (Optional, object) A query clause that retrieves a subset of data from the
63
+ (Optional, object) A query clause that retrieves a subset of data from the
63
64
source index. See <<query-dsl>>.
64
65
65
66
[[ml-evaluate-dfanalytics-resources]]
66
67
==== {dfanalytics-cap} evaluation resources
67
68
68
69
[[binary-sc-resources]]
69
- ===== Binary soft classification configuration objects
70
+ ===== Binary soft classification evaluation objects
70
71
71
72
Binary soft classification evaluates the results of an analysis which outputs
72
73
the probability that each document belongs to a certain class. For example, in
@@ -87,24 +88,24 @@ document is an outlier.
87
88
(Optional, object) Specifies the metrics that are used for the evaluation.
88
89
Available metrics:
89
90
90
- `auc_roc`::
91
+ `auc_roc`:::
91
92
(Optional, object) The AUC ROC (area under the curve of the receiver
92
93
operating characteristic) score and optionally the curve. Default value is
93
94
{"includes_curve": false}.
94
95
95
- `precision`::
96
+ `confusion_matrix`:::
97
+ (Optional, object) Set the different thresholds of the {olscore} at where
98
+ the metrics (`tp` - true positive, `fp` - false positive, `tn` - true
99
+ negative, `fn` - false negative) are calculated. Default value is
100
+ {"at": [0.25, 0.50, 0.75]}.
101
+
102
+ `precision`:::
96
103
(Optional, object) Set the different thresholds of the {olscore} at where
97
104
the metric is calculated. Default value is {"at": [0.25, 0.50, 0.75]}.
98
105
99
- `recall`::
106
+ `recall`:::
100
107
(Optional, object) Set the different thresholds of the {olscore} at where
101
108
the metric is calculated. Default value is {"at": [0.25, 0.50, 0.75]}.
102
-
103
- `confusion_matrix`::
104
- (Optional, object) Set the different thresholds of the {olscore} at where
105
- the metrics (`tp` - true positive, `fp` - false positive, `tn` - true
106
- negative, `fn` - false negative) are calculated. Default value is
107
- {"at": [0.25, 0.50, 0.75]}.
108
109
109
110
110
111
[[regression-evaluation-resources]]
@@ -122,9 +123,18 @@ which outputs a prediction of values.
122
123
in other words the results of the {regression} analysis.
123
124
124
125
`metrics`::
125
- (Required, object) Specifies the metrics that are used for the evaluation.
126
- Available metrics are `r_squared` and `mean_squared_error`.
127
-
126
+ (Optional, object) Specifies the metrics that are used for the evaluation.
127
+ Available metrics:
128
+
129
+ `mean_squared_error`:::
130
+ (Optional, object) Average squared difference between the predicted values and the actual (`ground truth`) value.
131
+ For more information, read https://en.wikipedia.org/wiki/Mean_squared_error[this wiki article].
132
+
133
+ `r_squared`:::
134
+ (Optional, object) Proportion of the variance in the dependent variable that is predictable from the independent variables.
135
+ For more information, read https://en.wikipedia.org/wiki/Coefficient_of_determination[this wiki article].
136
+
137
+
128
138
129
139
[[classification-evaluation-resources]]
130
140
==== {classification-cap} evaluation objects
@@ -134,20 +144,28 @@ outputs a prediction that identifies to which of the classes each document
134
144
belongs.
135
145
136
146
`actual_field`::
137
- (Required, string) The field of the `index` which contains the ground truth.
138
- The data type of this field must be keyword.
139
-
140
- `metrics`::
141
- (Required, object) Specifies the metrics that are used for the evaluation.
142
- Available metric is `multiclass_confusion_matrix`.
147
+ (Required, string) The field of the `index` which contains the `ground truth`.
148
+ The data type of this field must be categorical.
143
149
144
150
`predicted_field`::
145
151
(Required, string) The field in the `index` that contains the predicted value,
146
- in other words the results of the {classanalysis}. The data type of this field
147
- is string. You need to add `.keyword` to the predicted field name (the name
148
- you put in the {classanalysis} object as `prediction_field_name` or the
149
- default value of the same field if you didn't specified explicitly). For
150
- example, `predicted_field` : `ml.animal_class_prediction.keyword`.
152
+ in other words the results of the {classanalysis}.
153
+
154
+ `metrics`::
155
+ (Optional, object) Specifies the metrics that are used for the evaluation.
156
+ Available metrics:
157
+
158
+ `accuracy`:::
159
+ (Optional, object) Accuracy of predictions (per-class and overall).
160
+
161
+ `multiclass_confusion_matrix`:::
162
+ (Optional, object) Multiclass confusion matrix.
163
+
164
+ `precision`:::
165
+ (Optional, object) Precision of predictions (per-class and average).
166
+
167
+ `recall`:::
168
+ (Optional, object) Recall of predictions (per-class and average).
151
169
152
170
153
171
////
@@ -360,7 +378,7 @@ POST _ml/data_frame/_evaluate
360
378
"evaluation": {
361
379
"classification": { <1>
362
380
"actual_field": "animal_class", <2>
363
- "predicted_field": "ml.animal_class_prediction.keyword ", <3>
381
+ "predicted_field": "ml.animal_class_prediction", <3>
364
382
"metrics": {
365
383
"multiclass_confusion_matrix" : {} <4>
366
384
}
@@ -374,8 +392,7 @@ POST _ml/data_frame/_evaluate
374
392
<2> The field that contains the ground truth value for the actual animal
375
393
classification. This is required in order to evaluate results.
376
394
<3> The field that contains the predicted value for animal classification by
377
- the {classanalysis}. Since the field storing predicted class is dynamically
378
- mapped as text and keyword, you need to add the `.keyword` suffix to the name.
395
+ the {classanalysis}.
379
396
<4> Specifies the metric for the evaluation.
380
397
381
398
0 commit comments