You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/reference/search/rank-eval.asciidoc
+50Lines changed: 50 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -259,6 +259,56 @@ in the query. Defaults to 10.
259
259
|`normalize` | If set to `true`, this metric will calculate the https://en.wikipedia.org/wiki/Discounted_cumulative_gain#Normalized_DCG[Normalized DCG].
Expected Reciprocal Rank (ERR) is an extension of the classical reciprocal rank for the graded relevance case
266
+
(Olivier Chapelle, Donald Metzler, Ya Zhang, and Pierre Grinspan. 2009. http://olivier.chapelle.cc/pub/err.pdf[Expected reciprocal rank for graded relevance].)
267
+
268
+
It is based on the assumption of a cascade model of search, in which a user scans through ranked search
269
+
results in order and stops at the first document that satisfies the information need. For this reason, it
270
+
is a good metric for question answering and navigation queries, but less so for survey oriented information
271
+
needs where the user is interested in finding many relevant documents in the top k results.
272
+
273
+
The metric models the expectation of the reciprocal of the position at which a user stops reading through
274
+
the result list. This means that relevant document in top ranking positions will contribute much to the
275
+
overall score. However, the same document will contribute much less to the score if it appears in a lower rank,
276
+
even more so if there are some relevant (but maybe less relevant) documents preceding it.
277
+
In this way, the ERR metric discounts documents which are shown after very relevant documents. This introduces
278
+
a notion of dependency in the ordering of relevant documents that e.g. Precision or DCG don't account for.
279
+
280
+
[source,js]
281
+
--------------------------------
282
+
GET /twitter/_rank_eval
283
+
{
284
+
"requests": [
285
+
{
286
+
"id": "JFK query",
287
+
"request": { "query": { "match_all": {}}},
288
+
"ratings": []
289
+
}],
290
+
"metric": {
291
+
"expected_reciprocal_rank": {
292
+
"maximum_relevance" : 3,
293
+
"k" : 20
294
+
}
295
+
}
296
+
}
297
+
--------------------------------
298
+
// CONSOLE
299
+
// TEST[setup:twitter]
300
+
301
+
The `expected_reciprocal_rank` metric takes the following parameters:
0 commit comments