Skip to content

Commit 472b564

Browse files
author
Christoph Büscher
committed
[Docs] Add rank_eval size parameter k (#29218)
The rank_eval documentation was missing an explanation of the parameter `k` that controls the number of top hits that are used in the ranking evaluation. Closes #29205
1 parent 5e2c4ea commit 472b564

File tree

1 file changed

+16
-5
lines changed

1 file changed

+16
-5
lines changed

docs/reference/search/rank-eval.asciidoc

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,16 @@
11
[[search-rank-eval]]
22
== Ranking Evaluation API
33

4+
experimental[The ranking evaluation API is experimental and may be changed or removed completely in a future release,
5+
as well as change in non-backwards compatible ways on minor versions updates. Elastic will take a best effort
6+
approach to fix any issues, but experimental features are not subject to the support SLA of official GA features.]
7+
48
The ranking evaluation API allows to evaluate the quality of ranked search
59
results over a set of typical search queries. Given this set of queries and a
610
list or manually rated documents, the `_rank_eval` endpoint calculates and
711
returns typical information retrieval metrics like _mean reciprocal rank_,
812
_precision_ or _discounted cumulative gain_.
913

10-
experimental[The ranking evaluation API is new and may change in non-backwards compatible ways in the future, even on minor versions updates.]
11-
1214
[float]
1315
=== Overview
1416

@@ -41,7 +43,7 @@ GET /my_index/_rank_eval
4143
{
4244
"requests": [ ... ], <1>
4345
"metric": { <2>
44-
"reciprocal_rank": { ... } <3>
46+
"mean_reciprocal_rank": { ... } <3>
4547
}
4648
}
4749
------------------------------
@@ -85,7 +87,7 @@ The request section contains several search requests typical to your application
8587
<3> a list of document ratings, each entry containing the documents `_index` and `_id` together with
8688
the rating of the documents relevance with regards to this search request
8789

88-
A document `rating` can be any integer value that expresses the relevance of the document on a user defined scale. For some of the metrics, just giving a binary rating (e.g. `0` for irrelevant and `1` for relevant) will be sufficient, other metrics can use a more fine grained scale.
90+
A document `rating` can be any integer value that expresses the relevance of the document on a user defined scale. For some of the metrics, just giving a binary rating (e.g. `0` for irrelevant and `1` for relevant) will be sufficient, other metrics can use a more fine grained scale.
8991

9092
NOTE: To use the ranking evaluation API with indices that use multiple types, you should add a filter on the `_type` field to
9193
the query in the request. Otherwise, if your index uses multiple types with the same id, the provided
@@ -162,6 +164,7 @@ GET /twitter/_rank_eval
162164
}],
163165
"metric": {
164166
"precision": {
167+
"k" : 20,
165168
"relevant_rating_threshold": 1,
166169
"ignore_unlabeled": false
167170
}
@@ -176,7 +179,9 @@ The `precision` metric takes the following optional parameters
176179
[cols="<,<",options="header",]
177180
|=======================================================================
178181
|Parameter |Description
179-
|`relevant_rating_threshold` |Sets the rating threshold above which documents are considered to be
182+
|`k` |sets the maximum number of documents retrieved per query. This value will act in place of the usual `size` parameter
183+
in the query. Defaults to 10.
184+
|`relevant_rating_threshold` |sets the rating threshold above which documents are considered to be
180185
"relevant". Defaults to `1`.
181186
|`ignore_unlabeled` |controls how unlabeled documents in the search results are counted.
182187
If set to 'true', unlabeled documents are ignored and neither count as relevant or irrelevant. Set to 'false' (the default), they are treated as irrelevant.
@@ -202,6 +207,7 @@ GET /twitter/_rank_eval
202207
}],
203208
"metric": {
204209
"mean_reciprocal_rank": {
210+
"k" : 20,
205211
"relevant_rating_threshold" : 1
206212
}
207213
}
@@ -215,6 +221,8 @@ The `mean_reciprocal_rank` metric takes the following optional parameters
215221
[cols="<,<",options="header",]
216222
|=======================================================================
217223
|Parameter |Description
224+
|`k` |sets the maximum number of documents retrieved per query. This value will act in place of the usual `size` parameter
225+
in the query. Defaults to 10.
218226
|`relevant_rating_threshold` |Sets the rating threshold above which documents are considered to be
219227
"relevant". Defaults to `1`.
220228
|=======================================================================
@@ -238,6 +246,7 @@ GET /twitter/_rank_eval
238246
}],
239247
"metric": {
240248
"dcg": {
249+
"k" : 20,
241250
"normalize": false
242251
}
243252
}
@@ -251,6 +260,8 @@ The `dcg` metric takes the following optional parameters:
251260
[cols="<,<",options="header",]
252261
|=======================================================================
253262
|Parameter |Description
263+
|`k` |sets the maximum number of documents retrieved per query. This value will act in place of the usual `size` parameter
264+
in the query. Defaults to 10.
254265
|`normalize` | If set to `true`, this metric will calculate the https://en.wikipedia.org/wiki/Discounted_cumulative_gain#Normalized_DCG[Normalized DCG].
255266
|=======================================================================
256267

0 commit comments

Comments
 (0)