Point in time reader context for multiple (scroll) queries

[jimczi]

Currently a scroll context is a point in time snapshot of a set of indices associated with a query.
Each successive scroll query on this context moves the query forward until the query is exhausted. This is very useful when a single operation is performed (eg. reindexing) but when multiple scrolls are needed it is difficult to synchronize them efficiently.
For instance a multi-pass algorithm that needs to scan the data multiple times would have to use different scrolls to extract the results from ES and each pass could return different results.
Similarly if a batch of queries need to be executed, the only way to make sure that they will see the same documents is to freeze the indices and to use a different scroll for each of them.

So instead of coupling a scroll context with a set of shards and a query we could create shareable reader context with no query associated. One or multiple users could then reference this reader to perform different queries using the _search endpoint.

So for instance:

GET index1,index2,index3/_create_context

... creates a reader context for index1, index2 and index3 and returns the context id for this reader.

Then the context id can be used in a search query to retrieve results associated with this context:

GET _search
{
    "context_id": "DXF1ZXJ5QW5...",
    "query": { ... }
}

... but instead of using the _scroll endpoint to paginate through this query we could leverage search_after and let the user decides how he wants to paginate. search_after is difficult to use when the reader can change since it requires a tie breaker but inside a specific context search_after can simply use the docID/shardID as efficient tie breakers (no need to use _id or anything that needs extra memory) so any sort would be eligible to sort_after.

Also the reader context could have the same restriction than a scroll query regarding expiration. The only change would be that different queries could hit the same context so each of them would concurrently extend the context if needed.

[talevy]

Pinging @elastic/es-search-aggs

[nik9000]

GET index1,index2,index3/_create_context

I think it is weird for GET to make a thing on the server, even if all it is making is a context. Maybe POST?

[nik9000]

Otherwise, super ❤️ for the idea.

[jimczi]

Closing as duplicate of #26472