From 8e9c8cb5b1a5b6be5afcd6495d9ebe89cf0f5e46 Mon Sep 17 00:00:00 2001 From: Nick Canzoneri Date: Tue, 22 Nov 2022 09:20:56 -0500 Subject: [PATCH 1/4] Update search-settings documentation to reflect the fact that the indices.query.bool.max_clause_count setting has been deprecated --- .../modules/indices/search-settings.asciidoc | 31 ++++++++++--------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/docs/reference/modules/indices/search-settings.asciidoc b/docs/reference/modules/indices/search-settings.asciidoc index 17fe9ce08b544..60740bbb7a2cf 100644 --- a/docs/reference/modules/indices/search-settings.asciidoc +++ b/docs/reference/modules/indices/search-settings.asciidoc @@ -6,23 +6,24 @@ limits. [[indices-query-bool-max-clause-count]] `indices.query.bool.max_clause_count`:: +deprecated:[8.0.0] (<>, integer) -Maximum number of clauses a query can contain. Defaults to `4096`. -+ -This setting limits the total number of clauses that a query tree can have. The default of 4096 -is quite high and should normally be sufficient. This limit applies to the rewritten query, so -not only `bool` queries can contribute high numbers of clauses, but also all queries that rewrite -to `bool` queries internally such as `fuzzy` queries. The limit is in place to prevent searches -from becoming too large, and taking up too much CPU and memory. In case you're considering -increasing this setting, make sure you've exhausted all other options to avoid having to do this. -Higher values can lead to performance degradations and memory issues, especially in clusters with -a high load or few resources. +This deprecated setting has no effect. + +Elasticsearch will now dynamically set the maximum number of allowed clauses in a query, using +a heuristic based on the size of the search thread pool and the size of the heap allocated to +the JVM. This limit has a minimum value of 1024 and will in most cases be larger (for example, +a node with 30Gb RAM and 48 CPUs will have a maximum clause count of around 27,000). Larger +heaps lead to higher values, and larger thread pools result in lower values. + +Queries with many clauses should be avoided whenever possible. If you previously bumped this +setting to accommodate heavy queries, you might need to increase the amount of memory available +to Elasticsearch, or to reduce the size of your search thread pool so that more memory is +available to each concurrent search. -Elasticsearch offers some tools to avoid running into issues with regards to the maximum number of -clauses such as the <> query, which allows querying many distinct -values while still counting as a single clause, or the <> option -of <> fields, which allows executing prefix queries that expand to a high -number of terms as a single term query. +In previous versions of Lucene you could get around this limit by nesting boolean queries +within each other, but the limit is now based on the total number of leaf queries within the +query as a whole and this workaround will no longer help. [[search-settings-max-buckets]] `search.max_buckets`:: From f95a7539f6f6b01147cc1c1cec75ce1d2e593028 Mon Sep 17 00:00:00 2001 From: Abdon Pijpelink Date: Mon, 28 Nov 2022 16:40:55 +0100 Subject: [PATCH 2/4] Fix indentation --- docs/reference/modules/indices/search-settings.asciidoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/reference/modules/indices/search-settings.asciidoc b/docs/reference/modules/indices/search-settings.asciidoc index 60740bbb7a2cf..0762774d8526e 100644 --- a/docs/reference/modules/indices/search-settings.asciidoc +++ b/docs/reference/modules/indices/search-settings.asciidoc @@ -9,18 +9,18 @@ limits. deprecated:[8.0.0] (<>, integer) This deprecated setting has no effect. - ++ Elasticsearch will now dynamically set the maximum number of allowed clauses in a query, using a heuristic based on the size of the search thread pool and the size of the heap allocated to the JVM. This limit has a minimum value of 1024 and will in most cases be larger (for example, a node with 30Gb RAM and 48 CPUs will have a maximum clause count of around 27,000). Larger heaps lead to higher values, and larger thread pools result in lower values. - ++ Queries with many clauses should be avoided whenever possible. If you previously bumped this setting to accommodate heavy queries, you might need to increase the amount of memory available to Elasticsearch, or to reduce the size of your search thread pool so that more memory is available to each concurrent search. - ++ In previous versions of Lucene you could get around this limit by nesting boolean queries within each other, but the limit is now based on the total number of leaf queries within the query as a whole and this workaround will no longer help. From 51c1b32691b9c28bd80feddefde5bb3218c18500 Mon Sep 17 00:00:00 2001 From: Abdon Pijpelink Date: Tue, 29 Nov 2022 11:00:04 +0100 Subject: [PATCH 3/4] Replace Elasticsearch with {es} --- docs/reference/modules/indices/search-settings.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/reference/modules/indices/search-settings.asciidoc b/docs/reference/modules/indices/search-settings.asciidoc index 0762774d8526e..e43ec076578d4 100644 --- a/docs/reference/modules/indices/search-settings.asciidoc +++ b/docs/reference/modules/indices/search-settings.asciidoc @@ -10,7 +10,7 @@ deprecated:[8.0.0] (<>, integer) This deprecated setting has no effect. + -Elasticsearch will now dynamically set the maximum number of allowed clauses in a query, using +{es} will now dynamically set the maximum number of allowed clauses in a query, using a heuristic based on the size of the search thread pool and the size of the heap allocated to the JVM. This limit has a minimum value of 1024 and will in most cases be larger (for example, a node with 30Gb RAM and 48 CPUs will have a maximum clause count of around 27,000). Larger @@ -18,7 +18,7 @@ heaps lead to higher values, and larger thread pools result in lower values. + Queries with many clauses should be avoided whenever possible. If you previously bumped this setting to accommodate heavy queries, you might need to increase the amount of memory available -to Elasticsearch, or to reduce the size of your search thread pool so that more memory is +to {es}, or to reduce the size of your search thread pool so that more memory is available to each concurrent search. + In previous versions of Lucene you could get around this limit by nesting boolean queries From 6e9762f92ef7bfdbbf630fd0b9b0c3601dcec04c Mon Sep 17 00:00:00 2001 From: Abdon Pijpelink Date: Tue, 29 Nov 2022 11:15:42 +0100 Subject: [PATCH 4/4] Add deprecation entry to release notes --- docs/reference/release-notes/8.1.0.asciidoc | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/reference/release-notes/8.1.0.asciidoc b/docs/reference/release-notes/8.1.0.asciidoc index 8cce7d90c03eb..c1c44871d6416 100644 --- a/docs/reference/release-notes/8.1.0.asciidoc +++ b/docs/reference/release-notes/8.1.0.asciidoc @@ -143,6 +143,9 @@ CRUD:: Cluster Coordination:: * Remove last few mentions of Zen discovery {es-pull}80410[#80410] +Search:: +* Deprecate the `indices.query.bool.max_clause_count` node setting {es-pull}81525[#81525] (issue: {es-issue}46433[#46433]) + SQL:: * Deprecate `index_include_frozen` request parameter {es-pull}83943[#83943] (issue: {es-issue}81939[#81939])