Add SearchableSnapshotRepository #50239
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds a repository type that allows the creation of an index backed by searchable snapshots. These indices are allocated and recovered just like normal indices, but the underlying `SearchableSnapshotDirectory` makes sure that no recovery need take place since the correct files all seem to already exist on the target node. There are a number of limitations in this implementation: - like normal indices, after the intial allocation the primary is always allocated to a node that previously held an in-sync copy. If the cluster loses all copies of a snapshot-backed index then it does not attempt to recover. - peer recoveries of indices containing deletes do not currently work. - when performing disk-based shard allocation we make no attempt to quantify the disk usage of these shards any differently.
DaveCTurner
added
>enhancement
:Distributed Coordination/Snapshot/Restore
|
Pinging @elastic/es-distributed (:Distributed/Snapshot/Restore) |
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this pull request
Mar 16, 2020
In elastic#50239 we introduced a temporary workaround in Store#tryOpenIndex to allow searchable snapshots shards to be allocated anywhere. elastic#52527 reverts this workaround but did not revert the associated test changes. This commit reverts the associated test changes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds a repository type that allows the creation of an index backed
by searchable snapshots. These indices are allocated and recovered just like
normal indices, but the underlying
SearchableSnapshotDirectorymakes surethat no recovery need take place since the correct files all seem to already
exist on the target node.
There are a number of limitations in this implementation:
like normal indices, after the intial allocation the primary is always
allocated to a node that previously held an in-sync copy. If the cluster
loses all copies of a snapshot-backed index then it does not attempt to
recover.
peer recoveries of indices containing deletes do not currently work.
when performing disk-based shard allocation we make no attempt to quantify
the disk usage of these shards any differently.