Skip to content

Commit 04e5e41

Browse files
jimczijimczi
committed
Merge branch 'master' into long_sort_optimization
2 parents b5cad4d + 778e47f commit 04e5e41

32 files changed

+913
-106
lines changed

‎docs/build.gradle

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1146,3 +1146,42 @@ buildRestTests.setups['kibana_sample_data_ecommerce'] = '''
11461146
number_of_shards: 1
11471147
number_of_replicas: 0
11481148
'''
1149+
buildRestTests.setups['setup_logdata'] = '''
1150+
- do:
1151+
indices.create:
1152+
index: logdata
1153+
body:
1154+
settings:
1155+
number_of_shards: 1
1156+
number_of_replicas: 1
1157+
mappings:
1158+
properties:
1159+
grade:
1160+
type: byte
1161+
- do:
1162+
bulk:
1163+
index: logdata
1164+
refresh: true
1165+
body: |
1166+
{"index":{}}
1167+
{"grade": 100, "weight": 2}
1168+
{"index":{}}
1169+
{"grade": 50, "weight": 3}
1170+
'''
1171+
buildRestTests.setups['logdata_job'] = buildRestTests.setups['setup_logdata'] + '''
1172+
- do:
1173+
ml.put_data_frame_analytics:
1174+
id: "loganalytics"
1175+
body: >
1176+
{
1177+
"source": {
1178+
"index": "logdata"
1179+
},
1180+
"dest": {
1181+
"index": "logdata_out"
1182+
},
1183+
"analysis": {
1184+
"outlier_detection": {}
1185+
}
1186+
}
1187+
'''

‎docs/reference/data-frames/apis/preview-transform.asciidoc

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,15 @@ Previews a {dataframe-transform}.
3030
{stack-ov}/security-privileges.html[Security privileges] and
3131
{stack-ov}/built-in-roles.html[Built-in roles].
3232

33+
[discrete]
34+
[[preview-data-frame-transform-desc]]
35+
==== {api-description-title}
36+
37+
This API generates a preview of the results that you will get when you run the
38+
<<put-data-frame-transform,create {dataframe-transforms} API>> with the same
39+
configuration. It returns a maximum of 100 results. The calculations are based
40+
on all the current data in the source index.
41+
3342
[discrete]
3443
[[preview-data-frame-transform-request-body]]
3544
==== {api-request-body-title}
@@ -40,6 +49,15 @@ Previews a {dataframe-transform}.
4049
`pivot` (Required)::
4150
(object) Defines the pivot function `group by` fields and the aggregation to
4251
reduce the data. See <<data-frame-transform-pivot>>.
52+
53+
[discrete]
54+
[[preview-data-frame-transform-response]]
55+
==== {api-response-body-title}
56+
57+
`preview`::
58+
(array) An array of documents. In particular, they are the JSON
59+
representation of the documents that would be created in the destination index
60+
by the {dataframe-transform}.
4361

4462
[discrete]
4563
==== {api-examples-title}

‎docs/reference/ml/apis/delete-dfanalytics.asciidoc

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
[role="xpack"]
2+
[testenv="platinum"]
3+
[[delete-dfanalytics]]
4+
=== Delete {dfanalytics-jobs} API
5+
[subs="attributes"]
6+
++++
7+
<titleabbrev>Delete {dfanalytics-jobs}</titleabbrev>
8+
++++
9+
10+
experimental[]
11+
12+
Deletes an existing {dfanalytics-job}.
13+
14+
[[ml-delete-dfanalytics-request]]
15+
==== {api-request-title}
16+
17+
`DELETE _ml/data_frame/analytics/<data_frame_analytics_id>`
18+
19+
[[ml-delete-dfanalytics-prereq]]
20+
==== {api-prereq-title}
21+
22+
* You must have `machine_learning_admin` built-in role to use this API. For more
23+
information, see {stack-ov}/security-privileges.html[Security privileges] and
24+
{stack-ov}/built-in-roles.html[Built-in roles].
25+
26+
[[ml-delete-dfanalytics-path-params]]
27+
==== {api-path-parms-title}
28+
29+
`<data_frame_analytics_id>` (Required)::
30+
(string) Identifier for the {dfanalytics-job} you want to delete.
31+
32+
[[ml-delete-dfanalytics-example]]
33+
==== {api-examples-title}
34+
35+
The following example deletes the `loganalytics` {dfanalytics-job}:
36+
37+
[source,js]
38+
--------------------------------------------------
39+
DELETE _ml/data_frame/analytics/loganalytics
40+
--------------------------------------------------
41+
// CONSOLE
42+
// TEST[skip:TBD]
43+
44+
The API returns the following result:
45+
46+
[source,js]
47+
----
48+
{
49+
"acknowledged" : true
50+
}
51+
----
52+
// TESTRESPONSE

‎docs/reference/ml/apis/evaluate-dfanalytics.asciidoc

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
[role="xpack"]
2+
[testenv="platinum"]
3+
[[evaluate-dfanalytics]]
4+
=== Evaluate {dfanalytics} API
5+
6+
[subs="attributes"]
7+
++++
8+
<titleabbrev>Evaluate {dfanalytics}</titleabbrev>
9+
++++
10+
11+
experimental[]
12+
13+
Evaluates the executed analysis on an index that is already annotated with a
14+
field that contains the results of the analytics (the `ground truth`) for each
15+
{dataframe} row. Evaluation is typically done via calculating a set of metrics
16+
that capture various aspects of the quality of the results over the data for
17+
which we have the `ground truth`. For different types of analyses different
18+
metrics are suitable. This API packages together commonly used metrics for
19+
various analyses.
20+
21+
[[ml-evaluate-dfanalytics-request]]
22+
==== {api-request-title}
23+
24+
`POST _ml/data_frame/_evaluate`
25+
26+
[[ml-evaluate-dfanalytics-prereq]]
27+
==== {api-prereq-title}
28+
29+
* You must have `monitor_ml` privilege to use this API. For more
30+
information, see {stack-ov}/security-privileges.html[Security privileges] and
31+
{stack-ov}/built-in-roles.html[Built-in roles].
32+
33+
[[ml-evaluate-dfanalytics-request-body]]
34+
==== {api-request-body-title}
35+
36+
`index` (Required)::
37+
(object) Defines the `index` in which the evaluation will be performed.
38+
39+
`evaluation` (Required)::
40+
(object) Defines the type of evaluation you want to perform. For example:
41+
`binary_soft_classification`.
42+
See Evaluate API resources.
43+
44+
[[ml-evaluate-dfanalytics-example]]
45+
==== {api-examples-title}
46+
47+
[source,js]
48+
--------------------------------------------------
49+
POST _ml/data_frame/_evaluate
50+
{
51+
"index": "my_analytics_dest_index",
52+
"evaluation": {
53+
"binary_soft_classification": {
54+
"actual_field": "is_outlier",
55+
"predicted_probability_field": "ml.outlier_score"
56+
}
57+
}
58+
}
59+
--------------------------------------------------
60+
// CONSOLE
61+
// TEST[skip:TBD]
62+
63+
The API returns the following results:
64+
65+
[source,js]
66+
----
67+
{
68+
"binary_soft_classification": {
69+
"auc_roc": {
70+
"score": 0.92584757746414444
71+
},
72+
"confusion_matrix": {
73+
"0.25": {
74+
"tp": 5,
75+
"fp": 9,
76+
"tn": 204,
77+
"fn": 5
78+
},
79+
"0.5": {
80+
"tp": 1,
81+
"fp": 5,
82+
"tn": 208,
83+
"fn": 9
84+
},
85+
"0.75": {
86+
"tp": 0,
87+
"fp": 4,
88+
"tn": 209,
89+
"fn": 10
90+
}
91+
},
92+
"precision": {
93+
"0.25": 0.35714285714285715,
94+
"0.5": 0.16666666666666666,
95+
"0.75": 0
96+
},
97+
"recall": {
98+
"0.25": 0.5,
99+
"0.5": 0.1,
100+
"0.75": 0
101+
}
102+
}
103+
}
104+
----
105+
// TESTRESPONSE

‎docs/reference/ml/apis/get-dfanalytics-stats.asciidoc

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
[role="xpack"]
2+
[testenv="platinum"]
3+
[[get-dfanalytics-stats]]
4+
=== Get {dfanalytics-jobs} statistics API
5+
[subs="attributes"]
6+
++++
7+
<titleabbrev>Get {dfanalytics-jobs} stats</titleabbrev>
8+
++++
9+
10+
experimental[]
11+
12+
Retrieves usage information for {dfanalytics-jobs}.
13+
14+
[[ml-get-dfanalytics-stats-request]]
15+
==== {api-request-title}
16+
17+
`GET _ml/data_frame/analytics/<data_frame_analytics_id>/_stats` +
18+
19+
`GET _ml/data_frame/analytics/<data_frame_analytics_id>,<data_frame_analytics_id>/_stats` +
20+
21+
`GET _ml/data_frame/analytics/_stats` +
22+
23+
`GET _ml/data_frame/analytics/_all/_stats` +
24+
25+
`GET _ml/data_frame/analytics/*/_stats`
26+
27+
[[ml-get-dfanalytics-stats-prereq]]
28+
==== {api-prereq-title}
29+
30+
* You must have `monitor_ml` privilege to use this API. For more
31+
information, see {stack-ov}/security-privileges.html[Security privileges] and
32+
{stack-ov}/built-in-roles.html[Built-in roles].
33+
34+
[[ml-get-dfanalytics-stats-path-params]]
35+
==== {api-path-parms-title}
36+
37+
`<data_frame_analytics_id>` (Optional)::
38+
(string) Identifier for the {dfanalytics-job}. If you do not specify one of
39+
these options, the API returns information for the first hundred
40+
{dfanalytics-jobs}.
41+
42+
`allow_no_match` (Optional)
43+
(boolean) If `false` and the `data_frame_analytics_id` does not match any
44+
{dfanalytics-job} an error will be returned. The default value is `true`.
45+
46+
[[ml-get-dfanalytics-stats-query-params]]
47+
==== {api-query-parms-title}
48+
49+
`from` (Optional)::
50+
(integer) Skips the specified number of {dfanalytics-jobs}. The default value
51+
is `0`.
52+
53+
`size` (Optional)::
54+
(integer) Specifies the maximum number of {dfanalytics-jobs} to obtain. The
55+
default value is `100`.
56+
57+
[discrete]
58+
[[ml-get-dfanalytics-stats-response-body]]
59+
==== {api-response-body-title}
60+
61+
The API returns the following information:
62+
63+
`data_frame_analytics`::
64+
(array) An array of statistics objects for {dfanalytics-jobs}, which are
65+
sorted by the `id` value in ascending order.
66+
67+
[[ml-get-dfanalytics-stats-example]]
68+
==== {api-examples-title}
69+
70+
[source,js]
71+
--------------------------------------------------
72+
GET _ml/data_frame/analytics/loganalytics/_stats
73+
--------------------------------------------------
74+
// CONSOLE
75+
// TEST[skip:TBD]
76+
77+
The API returns the following results:
78+
79+
[source,js]
80+
----
81+
{
82+
"count": 1,
83+
"data_frame_analytics": [
84+
{
85+
"id": "loganalytics",
86+
"state": "stopped"
87+
}
88+
]
89+
}
90+
----
91+
// TESTRESPONSE

0 commit comments

Comments
 (0)