You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The rank_eval documentation was missing an explanation of the parameter
`k` that controls the number of top hits that are used in the ranking evaluation.
Closes#29205
Copy file name to clipboardExpand all lines: docs/reference/search/rank-eval.asciidoc
+16-5Lines changed: 16 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -1,14 +1,16 @@
1
1
[[search-rank-eval]]
2
2
== Ranking Evaluation API
3
3
4
+
experimental[The ranking evaluation API is experimental and may be changed or removed completely in a future release,
5
+
as well as change in non-backwards compatible ways on minor versions updates. Elastic will take a best effort
6
+
approach to fix any issues, but experimental features are not subject to the support SLA of official GA features.]
7
+
4
8
The ranking evaluation API allows to evaluate the quality of ranked search
5
9
results over a set of typical search queries. Given this set of queries and a
6
10
list or manually rated documents, the `_rank_eval` endpoint calculates and
7
11
returns typical information retrieval metrics like _mean reciprocal rank_,
8
12
_precision_ or _discounted cumulative gain_.
9
13
10
-
experimental[The ranking evaluation API is new and may change in non-backwards compatible ways in the future, even on minor versions updates.]
11
-
12
14
[float]
13
15
=== Overview
14
16
@@ -41,7 +43,7 @@ GET /my_index/_rank_eval
41
43
{
42
44
"requests": [ ... ], <1>
43
45
"metric": { <2>
44
-
"reciprocal_rank": { ... } <3>
46
+
"mean_reciprocal_rank": { ... } <3>
45
47
}
46
48
}
47
49
------------------------------
@@ -85,7 +87,7 @@ The request section contains several search requests typical to your application
85
87
<3> a list of document ratings, each entry containing the documents `_index` and `_id` together with
86
88
the rating of the documents relevance with regards to this search request
87
89
88
-
A document `rating` can be any integer value that expresses the relevance of the document on a user defined scale. For some of the metrics, just giving a binary rating (e.g. `0` for irrelevant and `1` for relevant) will be sufficient, other metrics can use a more fine grained scale.
90
+
A document `rating` can be any integer value that expresses the relevance of the document on a user defined scale. For some of the metrics, just giving a binary rating (e.g. `0` for irrelevant and `1` for relevant) will be sufficient, other metrics can use a more fine grained scale.
89
91
90
92
[float]
91
93
=== Template based ranking evaluation
@@ -158,6 +160,7 @@ GET /twitter/_rank_eval
158
160
}],
159
161
"metric": {
160
162
"precision": {
163
+
"k" : 20,
161
164
"relevant_rating_threshold": 1,
162
165
"ignore_unlabeled": false
163
166
}
@@ -172,7 +175,9 @@ The `precision` metric takes the following optional parameters
|`relevant_rating_threshold` |Sets the rating threshold above which documents are considered to be
178
+
|`k` |sets the maximum number of documents retrieved per query. This value will act in place of the usual `size` parameter
179
+
in the query. Defaults to 10.
180
+
|`relevant_rating_threshold` |sets the rating threshold above which documents are considered to be
176
181
"relevant". Defaults to `1`.
177
182
|`ignore_unlabeled` |controls how unlabeled documents in the search results are counted.
178
183
If set to 'true', unlabeled documents are ignored and neither count as relevant or irrelevant. Set to 'false' (the default), they are treated as irrelevant.
@@ -198,6 +203,7 @@ GET /twitter/_rank_eval
198
203
}],
199
204
"metric": {
200
205
"mean_reciprocal_rank": {
206
+
"k" : 20,
201
207
"relevant_rating_threshold" : 1
202
208
}
203
209
}
@@ -211,6 +217,8 @@ The `mean_reciprocal_rank` metric takes the following optional parameters
|`k` |sets the maximum number of documents retrieved per query. This value will act in place of the usual `size` parameter
260
+
in the query. Defaults to 10.
250
261
|`normalize` | If set to `true`, this metric will calculate the https://en.wikipedia.org/wiki/Discounted_cumulative_gain#Normalized_DCG[Normalized DCG].
0 commit comments