Query Cache: Support shard level query response caching #7161

kimchy · 2014-08-05T10:38:28Z

The query cache allow to cache the (binary serialized) response of the shard level query phase execution based on the actual request as the key. The cache is fully coherent with the semantics of NRT, with a refresh (that actually ended up refreshing) causing previous cached entries on the relevant shard to be invalidated and eventually evicted.

This change enables query caching as an opt in index level setting, called index.cache.query.enable and defaults to false. The setting can be changed dynamically on an index. The cache is only enabled for search requests with search_type count.

The indices query cache is a node level query cache. The indices.cache.query.size controls what is the size (bytes wise) the cache will take, and defaults to 1% of the heap. Note, this cache is very effective with small values in it already. There is also the advanced option to set indices.cache.query.expire that allow to control after a certain time of inaccessibility the cache will be evicted.

Note, the request takes the search "body" as is (bytes), and uses it as the key. This means same JSON but with different key order will constitute different cache entries.

This change includes basic stats (shard level, index/indices level, and node level) for the query cache, showing how much is used and eviction rates.

While this is a good first step, and the goal is to get it in, there are a few things that would be great additions to this work, but they can be done as additional pull requests:

More stats, specifically cache hit and cache miss, per shard.
Request level flag, defaults to "not set" (inheriting what the setting is).
Allowing to change the cache size using the cluster update settings API
Consider enabling the cache to query phase also when asking hits are involved, note, this will only include the "top docs", not the actual hits.
See if there is a performant manner to solve the "out of order" of keys in the JSON case.
Maybe introduce a filter element, that is outside of the request, that is checked, and if it matches all docs in a shard, will not be used as part of the key. This will help with time based indices and moving windows for shards that fall "inside" the window to be more effective caching wise.
Add a more infra level support in search context that allows for any element to mark the search as non deterministic (on top of the support for "now"), and use it to not cache search responses.

jpountz · 2014-08-05T10:47:01Z

src/main/java/org/elasticsearch/index/cache/query/QueryCacheStats.java

+
+    @Override
+    public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
+        builder.startObject(Fields.FilterCacheStats);


s/FilterCache/QueryCache/ ?

aye, will change

jpountz · 2014-08-05T10:58:50Z

src/main/java/org/elasticsearch/indices/cache/query/IndicesQueryCache.java

+
+    private class CleanupKey implements IndexReader.ReaderClosedListener {
+        IndexShard indexShard;
+        long readerVersion;


Can you add a comment explaining why you use the long version as opposed to the cache&delete key?

jpountz · 2014-08-05T11:23:45Z

I left some minor comments on the PR but overall it looks good.

I would add an item to the TODO list that looks important to me: a more generic ability to disable the query cache from the various query/filter parsers so that the random function score could prevent the query from being cached when there is no explicit seed.

The query cache allow to cache the (binary serialized) response of the shard level query phase execution based on the actual request as the key. The cache is fully coherent with the semantics of NRT, with a refresh (that actually ended up refreshing) causing previous cached entries on the relevant shard to be invalidated and eventually evicted. This change enables query caching as an opt in index level setting, called `index.cache.query.enable` and defaults to `false`. The setting can be changed dynamically on an index. The cache is only enabled for search requests with search_type count. The indices query cache is a node level query cache. The `indices.cache.query.size` controls what is the size (bytes wise) the cache will take, and defaults to `1%` of the heap. Note, this cache is very effective with small values in it already. There is also the advanced option to set `indices.cache.query.expire` that allow to control after a certain time of inaccessibility the cache will be evicted. Note, the request takes the search "body" as is (bytes), and uses it as the key. This means same JSON but with different key order will constitute different cache entries. This change includes basic stats (shard level, index/indices level, and node level) for the query cache, showing how much is used and eviction rates. While this is a good first step, and the goal is to get it in, there are a few things that would be great additions to this work, but they can be done as additional pull requests: - More stats, specifically cache hit and cache miss, per shard. - Request level flag, defaults to "not set" (inheriting what the setting is). - Allowing to change the cache size using the cluster update settings API - Consider enabling the cache to query phase also when asking hits are involved, note, this will only include the "top docs", not the actual hits. - See if there is a performant manner to solve the "out of order" of keys in the JSON case. - Maybe introduce a filter element, that is outside of the request, that is checked, and if it matches all docs in a shard, will not be used as part of the key. This will help with time based indices and moving windows for shards that fall "inside" the window to be more effective caching wise. - Add a more infra level support in search context that allows for any element to mark the search as non deterministic (on top of the support for "now"), and use it to not cache search responses. closes elastic#7161

kimchy · 2014-08-05T11:38:10Z

@jpountz addressed your comments, ready for another round...

clintongormley · 2014-08-05T11:52:35Z

If the query cache already takes refreshes into account, why do we need the expire setting?

See if there is a performant manner to solve the "out of order" of keys in the JSON case.

This could also be handled client side, ie: if a cache flag is passed, then we generate "canonical" JSON (ie keys are emitted in sorted order)

kimchy · 2014-08-05T11:58:20Z

If the query cache already takes refreshes into account, why do we need the expire setting?

I don't see a big use case for it, just for completeness sake to be honest. Imagine an index that doesn't change, but still wanting to expire based on time for some reason, and not just based on size.

dakrone · 2014-08-05T12:52:12Z

src/main/java/org/elasticsearch/indices/cache/query/IndicesQueryCache.java

+        super(settings);
+        this.clusterService = clusterService;
+        this.threadPool = threadPool;
+        this.cleanInterval = componentSettings.getAsTime("clean_interval", TimeValue.timeValueSeconds(60));


Can this be a static string for the full-qualified setting? I think we discussed moving away from component settings, and using the full string makes the source code much more grep-able.

dakrone · 2014-08-05T13:23:12Z

I think we should add to the TODOs returning a key in the response whether the response was served from the cache or not, something like "cache_hit": true. It makes 3rd-party tracking of cache hits/misses easier.

kimchy · 2014-08-05T13:24:18Z

@dakrone agreed, that would be nice as well (and it needs to be on the shard level element, btw, so I would opt for only setting it if its there)

jpountz · 2014-08-05T13:28:59Z

src/main/java/org/elasticsearch/index/cache/query/QueryCacheStats.java

+    }
+
+    static final class Fields {
+        static final XContentBuilderString QueryCacheStats = new XContentBuilderString("query_cache");


this one should be in upper case as well?

will change

jpountz · 2014-08-05T15:25:07Z

LGTM

dakrone · 2014-08-05T15:40:50Z

src/main/java/org/elasticsearch/indices/cache/query/IndicesQueryCache.java

+     * since we are checking on the cluster state IndexMetaData always.
+     */
+    public static final String INDEX_CACHE_QUERY_ENABLED = "index.cache.query.enable";
+    public static final String INDEX_CACHE_QUERY_CLEAN_INTERVAL = "index.cache.query.clean_interval";


This should probably be "indices" instead of "index" since this is for all indices instead of a single index-level setting.

agreed!, will fix

dakrone · 2014-08-05T15:42:29Z

@kimchy left one comment about the clean_interval setting name, other than that LGTM.

The query cache allow to cache the (binary serialized) response of the shard level query phase execution based on the actual request as the key. The cache is fully coherent with the semantics of NRT, with a refresh (that actually ended up refreshing) causing previous cached entries on the relevant shard to be invalidated and eventually evicted. This change enables query caching as an opt in index level setting, called `index.cache.query.enable` and defaults to `false`. The setting can be changed dynamically on an index. The cache is only enabled for search requests with search_type count. The indices query cache is a node level query cache. The `indices.cache.query.size` controls what is the size (bytes wise) the cache will take, and defaults to `1%` of the heap. Note, this cache is very effective with small values in it already. There is also the advanced option to set `indices.cache.query.expire` that allow to control after a certain time of inaccessibility the cache will be evicted. Note, the request takes the search "body" as is (bytes), and uses it as the key. This means same JSON but with different key order will constitute different cache entries. This change includes basic stats (shard level, index/indices level, and node level) for the query cache, showing how much is used and eviction rates. While this is a good first step, and the goal is to get it in, there are a few things that would be great additions to this work, but they can be done as additional pull requests: - More stats, specifically cache hit and cache miss, per shard. - Request level flag, defaults to "not set" (inheriting what the setting is). - Allowing to change the cache size using the cluster update settings API - Consider enabling the cache to query phase also when asking hits are involved, note, this will only include the "top docs", not the actual hits. - See if there is a performant manner to solve the "out of order" of keys in the JSON case. - Maybe introduce a filter element, that is outside of the request, that is checked, and if it matches all docs in a shard, will not be used as part of the key. This will help with time based indices and moving windows for shards that fall "inside" the window to be more effective caching wise. - Add a more infra level support in search context that allows for any element to mark the search as non deterministic (on top of the support for "now"), and use it to not cache search responses. closes #7161

Related to #7161 and #7167

…s and indices.stats Relates to #7167 and #7161

The query cache allow to cache the (binary serialized) response of the shard level query phase execution based on the actual request as the key. The cache is fully coherent with the semantics of NRT, with a refresh (that actually ended up refreshing) causing previous cached entries on the relevant shard to be invalidated and eventually evicted. This change enables query caching as an opt in index level setting, called `index.cache.query.enable` and defaults to `false`. The setting can be changed dynamically on an index. The cache is only enabled for search requests with search_type count. The indices query cache is a node level query cache. The `indices.cache.query.size` controls what is the size (bytes wise) the cache will take, and defaults to `1%` of the heap. Note, this cache is very effective with small values in it already. There is also the advanced option to set `indices.cache.query.expire` that allow to control after a certain time of inaccessibility the cache will be evicted. Note, the request takes the search "body" as is (bytes), and uses it as the key. This means same JSON but with different key order will constitute different cache entries. This change includes basic stats (shard level, index/indices level, and node level) for the query cache, showing how much is used and eviction rates. While this is a good first step, and the goal is to get it in, there are a few things that would be great additions to this work, but they can be done as additional pull requests: - More stats, specifically cache hit and cache miss, per shard. - Request level flag, defaults to "not set" (inheriting what the setting is). - Allowing to change the cache size using the cluster update settings API - Consider enabling the cache to query phase also when asking hits are involved, note, this will only include the "top docs", not the actual hits. - See if there is a performant manner to solve the "out of order" of keys in the JSON case. - Maybe introduce a filter element, that is outside of the request, that is checked, and if it matches all docs in a shard, will not be used as part of the key. This will help with time based indices and moving windows for shards that fall "inside" the window to be more effective caching wise. - Add a more infra level support in search context that allows for any element to mark the search as non deterministic (on top of the support for "now"), and use it to not cache search responses. closes #7161

Related to #7161 and #7167

…s and indices.stats Relates to #7167 and #7161

kimchy added review labels Aug 5, 2014

jpountz reviewed Aug 5, 2014
View reviewed changes

jpountz removed the review label Aug 5, 2014

kimchy added the review label Aug 5, 2014

dakrone reviewed Aug 5, 2014
View reviewed changes

jpountz reviewed Aug 5, 2014
View reviewed changes

kimchy added 2 commits August 5, 2014 15:33

[query_cache] address lee comments

96aa803

[query_cache] uppercase

1ecc0e3

dakrone reviewed Aug 5, 2014
View reviewed changes

change to indices query cache interval

50a3214

kimchy closed this in 418ce50 Aug 5, 2014

kimchy deleted the query_cache branch August 5, 2014 15:49

clintongormley added the release highlight label Aug 5, 2014

clintongormley added a commit that referenced this pull request Aug 6, 2014

Documented the query cache module

e7f1aa4

Related to #7161 and #7167

clintongormley added a commit that referenced this pull request Aug 6, 2014

Documented the query cache module

163eae7

Related to #7161 and #7167

clintongormley added a commit that referenced this pull request Aug 6, 2014

REST spec: Added missing query_cache param to clear_cache, nodes.stat…

2d29823

…s and indices.stats Relates to #7167 and #7161

clintongormley added a commit that referenced this pull request Aug 6, 2014

REST spec: Added missing query_cache param to clear_cache, nodes.stat…

11f8edd

…s and indices.stats Relates to #7167 and #7161

clintongormley mentioned this pull request Aug 8, 2014

Node Stats : last index and last delete timestamps added #3933

Closed

jpountz removed the review label Aug 11, 2014

clintongormley added a commit that referenced this pull request Sep 8, 2014

Documented the query cache module

c38252d

Related to #7161 and #7167

clintongormley added a commit that referenced this pull request Sep 8, 2014

REST spec: Added missing query_cache param to clear_cache, nodes.stat…

2f61591

…s and indices.stats Relates to #7167 and #7161

clintongormley added the :Cache label Jun 6, 2015

inpink mentioned this pull request Nov 7, 2024

Add vertical scaling and SoftReference for snapshot repository data cache opensearch-project/OpenSearch#16489

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Query Cache: Support shard level query response caching #7161

Query Cache: Support shard level query response caching #7161

kimchy commented Aug 5, 2014

jpountz Aug 5, 2014

kimchy Aug 5, 2014

kimchy Aug 5, 2014

jpountz Aug 5, 2014

kimchy Aug 5, 2014

jpountz commented Aug 5, 2014

kimchy commented Aug 5, 2014

clintongormley commented Aug 5, 2014

kimchy commented Aug 5, 2014

dakrone Aug 5, 2014

kimchy Aug 5, 2014

dakrone commented Aug 5, 2014

kimchy commented Aug 5, 2014

jpountz Aug 5, 2014

kimchy Aug 5, 2014

jpountz commented Aug 5, 2014

dakrone Aug 5, 2014

kimchy Aug 5, 2014

dakrone commented Aug 5, 2014

Query Cache: Support shard level query response caching #7161

Query Cache: Support shard level query response caching #7161

Conversation

kimchy commented Aug 5, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpountz commented Aug 5, 2014

kimchy commented Aug 5, 2014

clintongormley commented Aug 5, 2014

kimchy commented Aug 5, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dakrone commented Aug 5, 2014

kimchy commented Aug 5, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpountz commented Aug 5, 2014

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dakrone commented Aug 5, 2014