Skip to content

Latest commit

 

History

History
295 lines (227 loc) · 13.5 KB

8.6.0.asciidoc

File metadata and controls

295 lines (227 loc) · 13.5 KB

{es} version 8.6.0

Known issues

  • Shard rebalancing may temporarily unbalance cluster

    From 8.6.0 onwards the default shard rebalancing algorithm will compute the final desired balance and then make shard movements to reconcile the current state of the cluster with the desired state. However the order in which the shard movements take place may be skewed towards certain nodes, causing the cluster to become temporarily unbalanced while the reconciliation is ongoing. As always, once a node reaches a disk watermark it will not accept any additional shards, but this skew may result in nodes reaching their disk watermarks more often than expected in normal operation. Once the reconciliation process completes, the cluster will be balanced again.

    To avoid this problem, upgrade to 8.8.0 or later.

Bug fixes

Aggregations

  • GeoBoundsAggregations reject sub aggregations {es-pull}91073[#91073] (issue: {es-issue}91072[#91072])

Allocation

  • Avoid NPE when disassociateDeadNodes is executed for a node present in the desired balance {es-pull}91659[#91659]

  • Check NodesShutdownMetadata type before assuming restart {es-pull}90807[#90807]

Authentication

  • Fix time unit for connection request timeout of JWKs reload {es-pull}92080[#92080]

  • Improve performance for role mapping with DNs {es-pull}92074[#92074]

  • Improve robustness of JwkSet reloading {es-pull}92081[#92081]

  • Support stored authentication headers prior to version 6.7 {es-pull}92221[#92221]

Authorization

  • Make adding auth info to REST responses more robust {es-pull}92168[#92168]

  • Security remove datemath special handling {es-pull}91047[#91047]

Cluster Coordination

  • Fix TransportMasterNodeAction holding a CS reference needlessly {es-pull}90724[#90724] (issue: {es-issue}89220[#89220])

  • Include last-committed data in publication {es-pull}92259[#92259] (issue: {es-issue}90158[#90158])

  • Unsafe bootstrap memory optimization {es-pull}92493[#92493]

EQL

  • EQL sequences: support join on multi-values {es-pull}89965[#89965]

Graph

  • Fix potential issue with graph api’s timed out field in response {es-pull}91006[#91006]

Health

  • Don’t account for the unassigned reason when diagnosing NO_VALID_SHARD_COPY {es-pull}92416[#92416]

  • Fix NPE when evaluating the disk health for non-data nodes {es-pull}92643[#92643]

  • Use https in the short URLs for the shards_availability indicator {es-pull}92310[#92310]

Indices APIs

  • Trigger index settings providers when updating component templates {es-pull}91615[#91615] (issue: {es-issue}91592[#91592])

Infra/Core

  • Check reserved state in Metadata.isGlobalStateEquals {es-pull}92124[#92124]

  • Datastream unavailable exception metadata {es-pull}91461[#91461]

  • Fix BytesRefArray on append empty BytesRef {es-pull}91364[#91364]

  • Fix index expression options for requests with a single name or pattern {es-pull}91231[#91231]

  • Force init of Unbox in log4j {es-pull}92377[#92377] (issue: {es-issue}91964[#91964])

  • In file based settings, wait until security index is ready for role mappings {es-pull}92173[#92173] (issue: {es-issue}91939[#91939])

  • Index expression exclusions never trigger "not found" {es-pull}90902[#90902]

  • Update error states from inside the main state executor {es-pull}90346[#90346] (issue: {es-issue}90337[#90337])

Infra/Scripting

  • Fix compile with hex literals ending with d/f {es-pull}91501[#91501] (issue: {es-issue}88614[#88614])

Ingest Node

  • Fixing a race condition in EnrichCoordinatorProxyAction that can leave an item stuck in its queue {es-pull}90688[#90688] (issue: {es-issue}90598[#90598])

Machine Learning

  • Copy more settings when creating DF analytics destination index {es-pull}91546[#91546] (issue: {es-issue}89795[#89795])

  • Fix for 'No statistics' error message {ml-pull}2410[#2410]

  • Fix for 'No counts available' error message {ml-pull}2414[#2414]

  • Guard against input sequences that are too long for Question Answering models {es-pull}91924[#91924]

  • Improve performance of closing files before spawning {ml-pull}2424[#2424]

  • Skip remote clusters when performing up front privileges validation for datafeeds {es-pull}91895[#91895] (issue: {es-issue}87832[#87832])

  • Support fields with commas in data frame analytics analyzed_fields {es-pull}91710[#91710] (issue: {es-issue}72541[#72541])

  • Validate rule filters are present on open anomaly detection api {es-pull}92207[#92207]

Mapping

  • Consolidate field name validation when parsing mappings and documents {es-pull}91328[#91328]

  • Fix handling empty key case in the terms aggregation {es-pull}90822[#90822]

Monitoring

  • Fix logstash loadavg (xpack cases) {es-pull}90494[#90494]

  • [Stack Monitoring] Update ES module mappings {es-pull}90649[#90649]

Network

  • Clean up on exception while chunking XContent {es-pull}92024[#92024]

  • Fix Chunked APIs sending incorrect responses to HEAD requests {es-pull}92042[#92042] (issue: {es-issue}92032[#92032])

  • Reject connection attempts while closing {es-pull}92465[#92465]

SQL

  • Fix NPE on logging when not tracking total hits {es-pull}92425[#92425]

Search

  • Allow different decay values depending on the score function {es-pull}91195[#91195] (issue: {es-issue}78887[#78887])

  • Fix timing bug with DFS profiling {es-pull}92421[#92421]

Snapshot/Restore

  • Simplify and optimize deduplication of RepositoryData for a non-caching repository instance {es-pull}91851[#91851] (issue: {es-issue}89952[#89952])

Store

  • Fix numOpenOutputs and modCount in ByteSizeCachingDirectory {es-pull}92440[#92440] (issue: {es-issue}92434[#92434])

Transform

  • Skip remote clusters when performing up front privileges validation {es-pull}91788[#91788]

Vector Search

  • Make knn search requests fully cancellable {es-pull}90612[#90612]

Deprecations

Allocation

  • Deprecate state field in /_cluster/reroute response {es-pull}90399[#90399]

  • Ensure balance threshold is at least 1 {es-pull}92100[#92100]

Ingest Node

  • Deprecate 'remove_binary' default of false for ingest attachment processor {es-pull}90460[#90460]

Mapping

  • Deprecate silently ignoring type, fields, copy_to and boost in metadata field definition {es-pull}90989[#90989] (issue: {es-issue}35389[#35389])

Enhancements

Allocation

  • Clear up forecasted write load and shard size from previous write index during rollovers {es-pull}91590[#91590]

  • Forecast average shard size during rollovers {es-pull}91561[#91561]

  • Forecast write load during rollovers {es-pull}91425[#91425]

  • Improve shard balancing {es-pull}91603[#91603]

  • Introduce desired-balance allocator {es-pull}91343[#91343]

  • Limit shard realocation retries {es-pull}90296[#90296]

  • Prevalidate node removal API {es-pull}88952[#88952]

  • Set default cluster.routing.allocation.balance.disk_usage {es-pull}91951[#91951]

  • Store write load in the IndexMetadata during data streams rollovers {es-pull}91019[#91019]

  • Update the default cluster.routing.allocation.balance.disk_usage {es-pull}92065[#92065]

  • DesiredBalance: expose it via _internal/desired_balance {es-pull}91038[#91038] (issue: {es-issue}90583[#90583])

Authorization

  • [Fleet] Added logs-elastic_agent* read privileges to kibana_system {es-pull}91701[#91701]

CRUD

  • Keep track of average shard write load {es-pull}90768[#90768] (issue: {es-issue}90102[#90102])

Geo

  • Centroid aggregation for cartesian points and shapes {es-pull}89216[#89216] (issue: {es-issue}90156[#90156])

  • Improve H3#hexRing logic and add H3#areNeighborCells method {es-pull}91140[#91140]

  • Move SpatialUtils to geo library {es-pull}88088[#88088] (issue: {es-issue}86607[#86607])

  • Reduce number of object allocations in H3#geoToH3 and speed up computations {es-pull}91492[#91492]

  • Support cartesian_bounds aggregation on point and shape {es-pull}91298[#91298] (issue: {es-issue}90157[#90157])

ILM+SLM

  • ILM: Get policy support wildcard name {es-pull}89238[#89238]

Infra/Core

  • Handle APM global labels as affix setting {es-pull}91438[#91438] (issue: {es-issue}91278[#91278])

  • Improve date math exclusions in expressions {es-pull}90298[#90298]

  • Introduce a phase to use String.equals on constant strings, rather than def equality {es-pull}91362[#91362] (issue: {es-issue}91235[#91235])

  • More actionable error for ancient indices {es-pull}91243[#91243]

  • Operator/index templates {es-pull}90143[#90143]

  • Operator/ingest {es-pull}89735[#89735]

  • Transport threads and _hot_threads {es-pull}90482[#90482] (issue: {es-issue}90334[#90334])

  • Upgrade XContent to Jackson 2.14.0 and enable Fast Double Parser {es-pull}90553[#90553]

Infra/Plugins

  • Create placeholder plugin when loading stable plugins {es-pull}90870[#90870]

  • Example stable plugin {es-pull}90805[#90805]

  • Make extendedPlugins, HasNativeController and moduleName optional in plugin descriptor {es-pull}90835[#90835]

  • Rename NamedComponent name parameter to value {es-pull}91306[#91306]

Infra/Scripting

  • Use an explicit null check for null receivers in painless, rather than an NPE {es-pull}91347[#91347] (issue: {es-issue}91236[#91236])

Machine Learning

  • Add a filter parameter to frequent items {es-pull}91137[#91137]

  • Add a regex to the output of the categorize_text aggregation {es-pull}90723[#90723]

  • Add ability to filter and sort buckets by change_point numeric values {es-pull}91299[#91299]

  • Add api to update trained model deployment number_of_allocations {es-pull}90728[#90728]

  • Alias timestamp to @timestamp in anomaly detection results index {es-pull}90812[#90812]

  • Allow model_aliases to be used with Pytorch trained models {es-pull}91296[#91296]

  • Allow overriding timestamp field to null in file structure finder {es-pull}90764[#90764]

  • Audit a message every day the datafeed has seen no data {es-pull}91774[#91774]

  • Low priority trained model deployments {es-pull}91234[#91234] (issue: {es-issue}91024[#91024])

  • Provide additional information about anomaly score factors {es-pull}90675[#90675]

Mapping

  • Don’t create IndexCaps objects when recording unmapped fields {es-pull}90806[#90806] (issue: {es-issue}90796[#90796])

  • aggregate metric double add a max min validation {es-pull}90381[#90381]

Recovery

  • Remove resize index settings once shards are started {es-pull}90391[#90391] (issue: {es-issue}90127[#90127])

Rollup

  • Test downsample runtime fields and security {es-pull}90593[#90593]

Search

  • Add LimitedOffsetsEnum to Limited offset token {es-pull}86110[#86110] (issue: {es-issue}86109[#86109])

  • Add profiling and documentation for dfs phase {es-pull}90536[#90536] (issue: {es-issue}89713[#89713])

  • Bulk merge field-caps responses using mapping hash {es-pull}86323[#86323]

  • Enhance nested depth tracking when parsing queries {es-pull}90425[#90425]

  • Expose telemetry about search usage {es-pull}91528[#91528]

  • Return docs when using nested mappings in archive indices {es-pull}90585[#90585] (issue: {es-issue}90523[#90523])

  • Use IndexOrDocValues query for IP range queries {es-pull}90303[#90303] (issue: {es-issue}83658[#83658])

Snapshot/Restore

  • Increase snaphot pool max size to 10 {es-pull}90282[#90282] (issue: {es-issue}89608[#89608])

  • Tie snapshot speed to node bandwidth settings {es-pull}91021[#91021] (issue: {es-issue}57023[#57023])

Store

  • Allow plugins to wrap Lucene directories created by the IndexModule {es-pull}91556[#91556]

TLS

  • Add certificate start/expiry dates to SSL Diagnostic message {es-pull}89461[#89461]

TSDB

  • Generate 'index.routing_path' from dynamic mapping templates {es-pull}90552[#90552] (issue: {es-issue}90528[#90528])

  • Support malformed numbers in synthetic _source {es-pull}90428[#90428]

  • Support synthetic _source for _doc_count field {es-pull}91465[#91465]

  • Synthetic _source: support field in many cases {es-pull}89950[#89950]

  • Synthetic _source: ignore_malformed for ip {es-pull}90038[#90038]

  • Synthetic _source: support wildcard field {es-pull}90196[#90196]

Transform

  • Add a health section to transform stats {es-pull}90760[#90760]

  • Support aggregate_metric_double field type in transform aggregations {es-pull}91045[#91045]

Vector Search

  • Add profiling information for knn vector queries {es-pull}90200[#90200]

New features

Distributed

  • Add "index" and "search" node roles with feature flag and setting {es-pull}90993[#90993]

EQL

  • EQL samples {es-pull}91312[#91312]

Health

  • Use chunked encoding for RestGetHealthAction {es-pull}91515[#91515] (issue: {es-issue}90223[#90223])

  • [HealthAPI] Use the RestCancellableNodeClient infrastructure {es-pull}91587[#91587]

Machine Learning

  • Make categorize_text aggregation GA {es-pull}88600[#88600]

Vector Search

  • Add fielddata and scripting support for byte-sized vectors {es-pull}91184[#91184]

  • Add support for indexing byte-sized knn vectors {es-pull}90774[#90774]

Regressions

Infra/Core

  • Revert "Remove ImmutableOpenMap from snapshot services" {es-pull}90287[#90287]

Upgrades

Infra/Logging

  • Upgrade to log4j 2.19.0 {es-pull}90589[#90589] (issue: {es-issue}90584[#90584])

Network

  • Upgrade to Netty 4.1.82.Final {es-pull}90604[#90604]

  • Upgrade to Netty 4.1.84 {es-pull}91271[#91271]

Snapshot/Restore

  • Upgrade GCS SDK to 2.13.1 {es-pull}92327[#92327]