🤖 ESQL: Merge upstream (#179)

🤖 Generated PR to keep ESQL development branch up to date

Author: elasticsearchmachine

TESTING.asciidoc

@@ -46,14 +46,15 @@ run it using Gradle:
 ==== Launching and debugging from an IDE
 
 If you want to run Elasticsearch from your IDE, the `./gradlew run` task
-supports a remote debugging option:
+supports a remote debugging option. Run the following from your terminal:
 
 ---------------------------------------------------------------------------
 ./gradlew run --debug-jvm
 ---------------------------------------------------------------------------
 
-This will instruct all JVMs (including any that run cli tools such as creating the keyring or adding users)
-to suspend and initiate a debug connection on port incrementing from `5005`.
+Next start the "Debug Elasticsearch" run configuration in IntelliJ. This will enable the IDE to connect to the process and allow debug functionality.
+
+
 As such the IDE needs to be instructed to listen for connections on this port.
 Since we might run multiple JVMs as part of configuring and starting the cluster it's
 recommended to configure the IDE to initiate multiple listening attempts. In case of IntelliJ, this option

distribution/docker/src/docker/bin/docker-entrypoint.sh

@@ -81,4 +81,4 @@ fi
 
 # Signal forwarding and child reaping is handled by `tini`, which is the
 # actual entrypoint of the container
-exec /usr/share/elasticsearch/bin/elasticsearch "$@" $POSITIONAL_PARAMETERS <<<"$KEYSTORE_PASSWORD"
+exec /usr/share/elasticsearch/bin/elasticsearch $POSITIONAL_PARAMETERS <<<"$KEYSTORE_PASSWORD"

docs/changelog/88211.yaml

@@ -0,0 +1,5 @@
+pr: 88211
+summary: Add 'mode' option to `_source` field mapper
+area: Search
+type: feature
+issues: []

docs/changelog/88445.yaml

@@ -0,0 +1,5 @@
+pr: 88445
+summary: Add issuer to GET _ssl/certificates
+area: TLS
+type: enhancement
+issues: []

docs/changelog/88450.yaml

@@ -0,0 +1,5 @@
+pr: 88450
+summary: Add new `cache_size` parameter to `trained_model` deployments API
+area: Machine Learning
+type: enhancement
+issues: []

docs/changelog/88479.yaml

@@ -0,0 +1,5 @@
+pr: 88479
+summary: Deduplicate mappings in persisted cluster state
+area: Cluster Coordination
+type: enhancement
+issues: []

docs/changelog/88502.yaml

@@ -0,0 +1,6 @@
+pr: 88586
+summary: Disable URL connection caching in SPIClassIterator
+area: Infra/Plugins
+type: bug
+issues:
+ - 88275

docs/changelog/88586.yaml

@@ -4,15 +4,15 @@
 Though very handy to have around, the source field takes up a significant amount
 of space on disk. Instead of storing source documents on disk exactly as you
 send them, Elasticsearch can reconstruct source content on the fly upon retrieval.
-Enable this by setting `synthetic: true` in `_source`:
+Enable this by setting `mode: synthetic` in `_source`:
 
 [source,console,id=enable-synthetic-source-example]
 ----
 PUT idx
 {
   "mappings": {
     "_source": {
-      "synthetic": true
+      "mode": "synthetic"
     }
   }
 }

docs/reference/mapping/fields/synthetic-source.asciidoc

@@ -228,7 +228,7 @@ Synthetic source always sorts `boolean` fields. For example:
 PUT idx
 {
   "mappings": {
-    "_source": { "synthetic": true },
+    "_source": { "mode": "synthetic" },
     "properties": {
       "bool": { "type": "boolean" }
     }

docs/reference/mapping/types/boolean.asciidoc

@@ -208,7 +208,7 @@ ifeval::["{release-state}"=="unreleased"]
 [[geo-point-synthetic-source]]
 ==== Synthetic source
 `geo_point` fields support <<synthetic-source,synthetic `_source`>> in their
-default configuration. Synthetic `_source` cannot be used together with 
+default configuration. Synthetic `_source` cannot be used together with
 <<ignore-malformed,`ignore_malformed`>>, <<copy-to,`copy_to`>>, or with
 <<doc-values,`doc_values`>> disabled.
 
@@ -219,7 +219,7 @@ longitude) and reduces them to their stored precision. For example:
 PUT idx
 {
   "mappings": {
-    "_source": { "synthetic": true },
+    "_source": { "mode": "synthetic" },
     "properties": {
       "point": { "type": "geo_point" }
     }

docs/reference/mapping/types/geo-point.asciidoc

@@ -165,7 +165,7 @@ Synthetic source always sorts `ip` fields and removes duplicates. For example:
 PUT idx
 {
   "mappings": {
-    "_source": { "synthetic": true },
+    "_source": { "mode": "synthetic" },
     "properties": {
       "ip": { "type": "ip" }
     }

docs/reference/mapping/types/ip.asciidoc

@@ -189,7 +189,7 @@ example:
 PUT idx
 {
   "mappings": {
-    "_source": { "synthetic": true },
+    "_source": { "mode": "synthetic" },
     "properties": {
       "kwd": { "type": "keyword" }
     }

docs/reference/mapping/types/keyword.asciidoc

@@ -243,7 +243,7 @@ Synthetic source always sorts numeric fields and removes duplicates. For example
 PUT idx
 {
   "mappings": {
-    "_source": { "synthetic": true },
+    "_source": { "mode": "synthetic" },
     "properties": {
       "long": { "type": "long" }
     }
@@ -271,7 +271,7 @@ Scaled floats will always apply their scaling factor so:
 PUT idx
 {
   "mappings": {
-    "_source": { "synthetic": true },
+    "_source": { "mode": "synthetic" },
     "properties": {
       "f": { "type": "scaled_float", "scaling_factor": 0.01 }
     }

docs/reference/mapping/types/numeric.asciidoc

@@ -173,7 +173,7 @@ Synthetic source always sorts `keyword` fields and removes duplicates, so
 PUT idx
 {
   "mappings": {
-    "_source": { "synthetic": true },
+    "_source": { "mode": "synthetic" },
     "properties": {
       "text": {
         "type": "text",

docs/reference/mapping/types/text.asciidoc

@@ -97,6 +97,10 @@ The detailed allocation status given the deployment configuration.
 (integer)
 The current number of nodes where the model is allocated.
 
+`cache_size`:::
+(<<byte-units,byte value>>)
+The inference cache size (in memory outside the JVM heap) per node for the model.
+
 `state`:::
 (string)
 The detailed allocation state related to the nodes.

docs/reference/ml/trained-models/apis/get-trained-models-stats.asciidoc

@@ -34,7 +34,7 @@ Increasing `threads_per_allocation` means more threads are used when
 an inference request is processed on a node. This can improve inference speed
 for certain models. It may also result in improvement to throughput.
 
-Increasing `number_of_allocations` means more threads are used to 
+Increasing `number_of_allocations` means more threads are used to
 process multiple inference requests in parallel resulting in throughput
 improvement. Each model allocation uses a number of threads defined by
 `threads_per_allocation`.
@@ -55,6 +55,11 @@ include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=model-id]
 [[start-trained-model-deployment-query-params]]
 == {api-query-parms-title}
 
+`cache_size`::
+(Optional, <<byte-units,byte value>>)
+The inference cache size (in memory outside the JVM heap) per node for the model.
+The default value is the same size as the `model_size_bytes`. To disable the cache, `0b` can be provided.
+
 `number_of_allocations`::
 (Optional, integer)
 The total number of allocations this model is assigned across {ml} nodes.

docs/reference/ml/trained-models/apis/start-trained-model-deployment.asciidoc

@@ -696,7 +696,7 @@ The API returns the following result:
                     ]
                   },
                   {
-                    "name": "MultiBucketCollector: [[my_scoped_agg, my_global_agg]]",
+                    "name": "BucketCollectorWrapper: [BucketCollectorWrapper[bucketCollector=[my_scoped_agg, my_global_agg]]]",
                     "reason": "aggregation",
                     "time_in_nanos": 867617
                   }

docs/reference/search/profile.asciidoc

@@ -0,0 +1,95 @@
+[[circuit-breaker-errors]]
+=== Circuit breaker errors
+
+{es} uses <<circuit-breaker,circuit breakers>> to prevent nodes from running out
+of JVM heap memory. If Elasticsearch estimates an operation would exceed a
+circuit breaker, it stops the operation and returns an error.
+
+By default, the <<parent-circuit-breaker,parent circuit breaker>> triggers at
+95% JVM memory usage. To prevent errors, we recommend taking steps to reduce
+memory pressure if usage consistently exceeds 85%.
+
+[discrete]
+[[diagnose-circuit-breaker-errors]]
+==== Diagnose circuit breaker errors
+
+**Error messages**
+
+If a request triggers a circuit breaker, {es} returns an error with a `429` HTTP
+status code.
+
+[source,js]
+----
+{
+  'error': {
+    'type': 'circuit_breaking_exception',
+    'reason': '[parent] Data too large, data for [<http_request>] would be [123848638/118.1mb], which is larger than the limit of [123273216/117.5mb], real usage: [120182112/114.6mb], new bytes reserved: [3666526/3.4mb]',
+    'bytes_wanted': 123848638,
+    'bytes_limit': 123273216,
+    'durability': 'TRANSIENT'
+  },
+  'status': 429
+}
+----
+// NOTCONSOLE
+
+{es} also writes circuit breaker errors to <<logging,`elasticsearch.log`>>. This
+is helpful when automated processes, such as allocation, trigger a circuit
+breaker.
+
+[source,txt]
+----
+Caused by: org.elasticsearch.common.breaker.CircuitBreakingException: [parent] Data too large, data for [<transport_request>] would be [num/numGB], which is larger than the limit of [num/numGB], usages [request=0/0b, fielddata=num/numKB, in_flight_requests=num/numGB, accounting=num/numGB]
+----
+
+**Check JVM memory usage**
+
+If you've enabled Stack Monitoring, you can view JVM memory usage in {kib}. In
+the main menu, click **Stack Monitoring**. On the Stack Monitoring **Overview**
+page, click **Nodes**. The **JVM Heap** column lists the current memory usage
+for each node.
+
+You can also use the <<cat-nodes,cat nodes API>> to get the current
+`heap.percent` for each node.
+
+[source,console]
+----
+GET _cat/nodes?v=true&h=name,node*,heap*
+----
+
+To get the JVM memory usage for each circuit breaker, use the
+<<cluster-nodes-stats,node stats API>>.
+
+[source,console]
+----
+GET _nodes/stats/breaker
+----
+
+[discrete]
+[[prevent-circuit-breaker-errors]]
+==== Prevent circuit breaker errors
+
+**Reduce JVM memory pressure**
+
+High JVM memory pressure often causes circuit breaker errors. See
+<<high-jvm-memory-pressure>>.
+
+**Avoid using fielddata on `text` fields**
+
+For high-cardinality `text` fields, fielddata can use a large amount of JVM
+memory. To avoid this, {es} disables fielddata on `text` fields by default. If
+you've enabled fielddata and triggered the <<fielddata-circuit-breaker,fielddata
+circuit breaker>>, consider disabling it and using a `keyword` field instead.
+See <<fielddata>>.
+
+**Clear the fieldata cache**
+
+If you've triggered the fielddata circuit breaker and can't disable fielddata,
+use the <<indices-clearcache,clear cache API>> to clear the fielddata cache.
+This may disrupt any in-flight searches that use fielddata.
+
+[source,console]
+----
+POST _cache/clear?fielddata=true
+----
+// TEST[s/^/PUT my-index\n/]

docs/reference/troubleshooting/common-issues/circuit-breaker-errors.asciidoc

@@ -0,0 +1,84 @@
+[[disk-usage-exceeded]]
+=== Error: disk usage exceeded flood-stage watermark, index has read-only-allow-delete block
+
+This error indicates a data node is critically low on disk space and has reached
+the <<cluster-routing-flood-stage,flood-stage disk usage watermark>>. To prevent
+a full disk, when a node reaches this watermark, {es} blocks writes to any index
+with a shard on the node. If the block affects related system indices, {kib} and
+other {stack} features may become unavailable.
+
+{es} will automatically remove the write block when the affected node's disk
+usage goes below the <<cluster-routing-watermark-high,high disk watermark>>. To
+achieve this, {es} automatically moves some of the affected node's shards to
+other nodes in the same data tier.
+
+To verify that shards are moving off the affected node, use the <<cat-shards,cat
+shards API>>.
+
+[source,console]
+----
+GET _cat/shards?v=true
+----
+
+If shards remain on the node, use the <<cluster-allocation-explain,cluster
+allocation explanation API>> to get an explanation for their allocation status.
+
+[source,console]
+----
+GET _cluster/allocation/explain
+{
+  "index": "my-index",
+  "shard": 0,
+  "primary": false,
+  "current_node": "my-node"
+}
+----
+// TEST[s/^/PUT my-index\n/]
+// TEST[s/"primary": false,/"primary": false/]
+// TEST[s/"current_node": "my-node"//]
+
+To immediately restore write operations, you can temporarily increase the disk
+watermarks and remove the write block.
+
+[source,console]
+----
+PUT _cluster/settings
+{
+  "persistent": {
+    "cluster.routing.allocation.disk.watermark.low": "90%",
+    "cluster.routing.allocation.disk.watermark.high": "95%",
+    "cluster.routing.allocation.disk.watermark.flood_stage": "97%"
+  }
+}
+
+PUT */_settings?expand_wildcards=all
+{
+  "index.blocks.read_only_allow_delete": null
+}
+----
+// TEST[s/^/PUT my-index\n/]
+
+As a long-term solution, we recommend you add nodes to the affected data tiers
+or upgrade existing nodes to increase disk space. To free up additional disk
+space, you can delete unneeded indices using the <<indices-delete-index,delete
+index API>>.
+
+[source,console]
+----
+DELETE my-index
+----
+// TEST[s/^/PUT my-index\n/]
+
+When a long-term solution is in place, reset or reconfigure the disk watermarks.
+
+[source,console]
+----
+PUT _cluster/settings
+{
+  "persistent": {
+    "cluster.routing.allocation.disk.watermark.low": null,
+    "cluster.routing.allocation.disk.watermark.high": null,
+    "cluster.routing.allocation.disk.watermark.flood_stage": null
+  }
+}
+----