Deprecate frozen indices endpoint in REST API specification #74983
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change adjusts the ML YAML tests in 7.x so that they do not use undocumented features that are being removed from master. The tests that have been modified had no reason to use undocumented fields in order to test what they were supposed to be testing. Making these tests use only documented functionality means that they can remain unmuted in the REST compatibility layer testing on the master branch. One test remains in 7.x that uses the undocumented delimited input feature (in jobs_crud.yml). This test will be excluded from the REST compatibility layer testing on the master branch.
…ameters (#74177) When a dynamic template is applied to a 7.x index, we do some validation checks against the generated mappings and issue a deprecation warning if the validation fails. We had some tests that were checking this at a low level, but nothing that exercised the full logic. When dynamic runtimes were added, we expected this behaviour to carry over; however, a bug in building the ParserContext to pass to the runtime Builder meant that we would instead always throw an error. In fact, erroring here is a good result, as the only reason we issue a deprecation warning for non-runtime fields is to preserve backward compatibility; given that runtime fields are new, it has never been possible to add a bad dynamic runtime template. This commit adds specific tests for both situations. It ensures that the parser context passed to a runtime builder will know that it is in a dynamic context, but it also removes the version check and deprecation warning, as they were never actually triggered and we can be stricter here.
Co-authored-by: bellengao <[email protected]>
Co-authored-by: James Rodewig <[email protected]> Co-authored-by: James Rodewig <[email protected]>
…74270) If there are outside ways by which a request can be decremented (e.g. due to cancelling a recovery concurrently) we may run into a situation where we try to send a `refcount == 0` request. We have to avoid this by incrementing a request before serializing and decrementing after it returns to make sure we don't corrupt the request bytes while they're being serialized or sent over the wire. Closes #74253
This commit modifies the Node Shutdown Status API to include information about the number of shards left to migrate off of the node in question, as well as checking if shard migration is stalled. "Stalled" is defined as no shards currently relocating off the node, and at least one shard on the node which cannot move per current allocation deciders. Relates #70338
Adds HLRC support for the newly added reset anomaly detection job API.
Today we don't really describe why using `index.shard.check_on_startup` is such a bad idea, or what to do instead. This commit expands the docs to clarify what it does, why it's not really necessary and what to do instead. It also now logs a warning every time the startup checks run to encourage users to stop using this setting.
Follow-up to #73434 Ensures that High Level Rest Client is running against a verified Elasticsearch. When the first request is send on HLRC, a request to the info endpoint is made first to verify the product identification and version.
…apshots (#74279) Today a searchable snapshot index inherits the index.shard.check_on_startup index setting from the snapshot it is mounted from. But this setting can require some expensive processing as documented in #74233, so we decided in #73147 to default the value of this setting to false but allow the user to override this in the mount request. Backport of #74235 Closes #73147
…74287) This adds a service that migrates the indices and ILM policies away from custom node attribute allocation routing to data tiers. Optionally, it also deletes one legacy index template. (cherry picked from commit 6285fac) Signed-off-by: Andrei Dan <[email protected]>
) (#74295) Test failures in #74101 revealed that the last documents persisted from the job running on a node before it goes down may not be deleted when the reset action is executed. The reason is that the results index has not been refreshed thus those docs are not visible to the search the delete by query action is doing. This commit adds a call to the refresh API before running delete by query to the results indices. Closes #74101 Backport of #74292
If sequence numbers are equal across snapshots (and thus no files get written to the repository) the segment count in the index commit is not reliable as it may have changed due to background merges. => fixed by always using the segment count determined from the file names in the snapshot instead closes #74249
Dynamic templates don't support match_mapping_type object and binary when runtime mappings are set, as such types are not supported runtime fields. This was not tested before, and this commit adds unit tests for these scenarios.
Additional fixes for `ActionConfigStatsTests.testEqualsAndHashcode` Relates to #74849 Co-authored-by: Przemko Robakowski <[email protected]>
The plain highlighter should order the fragments by order of appearance by default, but sorts them by score internally. This fix changes the sorting comparator in case ordering by score is not selected (the default) and adds testing around this. Closes #58236
…74918) Dynamic mappings updates containing nested mappings were not fixing their redundant include settings before being posted back to the put mappings service; this meant that a dynamic mapping that had both 'include_in_parent' and 'include_in_root' set to 'true' could cause an error if it was returned from multiple shards, because the first mapping would be updated to have 'include_in_root' set to 'false' when merged, and merging this with the second mapping with 'include_in_root' set to 'true' would cause a conflict. This commit ensures that fixRedundantIncludes is called on dynamic mappings before they are posted back to the mapping service. Fixes #74899
Backport of #74924 to 7.x branch. I forgot to adjust the versioning when data stream aliases was backported to 7.x branch. Co-authored-by: Martijn van Groningen <[email protected]>
No longer create as many RoutingNodes, since they are expensive. Closes #74325
Forks the Lucene data structure into a class called PackedArray that can be serialised using Elasticsearch streams.
* Fix GeoIpProcessor when there's no updated db (#74944) This change fixes problem with GeoIpProcessor when there's GeoIpTaskState present in the cluster state but there's no database matching the one used by the processor. It can happen when there are some but not all databases already updated. * fix compilation Co-authored-by: Przemko Robakowski <[email protected]>
…74768) AbstractScriptFieldType extends MappedFieldType and also implements RuntimeField. This means that the name() of a runtime field has to be the same as the name() of its corresponding mapped field type. However, for object fields we will want the nested child fields to have a specific short name, while their MappedFieldType will need to have a name that includes the parent. To make this distinction possible, this commit changes AbstractScriptFieldType subclasses to return their own implementations of RuntimeField from their type parsers instead of implementing the interface directly.
stevejgordon
added
Team:Data Management
|
Pinging @elastic/es-core-features (Team:Core/Features) |
elasticsearchmachine
added
v7.15.0
external-contributor
14 tasks
|
Pinging @elastic/clients-team (Team:Clients) |
danhermann
suggested changes
Jul 6, 2021
There was a problem hiding this comment.
We do want to keep the original verbiage on the warning (Frozen indices are deprecated because they provide no benefit given improvements in heap memory utilization. They will be removed in a future release.) because we are not committing to removing them in the 8.0 release.
I can adjust that for sure. |
|
Closing and will resubmit based on master. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Relates to #72618 which added deprecation warnings for this endpoint. This PR adds the deprecation to the REST API spec so that clients can generate the appropriate code comments.
@danhermann perhaps you can ensure this is as you'd expect based on your previous PR?
I believe this should be merged into 7.x and 7.14 branches. I've targeted 7.x as the base for the PR and added the
v7.14.0label +auto-backport. I may need some help confirming that this is the correct labelling before this gets merged.