-
Notifications
You must be signed in to change notification settings - Fork 25.2k
Add API to mount a snapshot as a searchable index #53084
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add API to mount a snapshot as a searchable index #53084
Conversation
This commit adds an API for mounting a snapshot as a searchable index. Internally, a request is made to the repository to fetch metadata information necessary from the snapshot in question, which is then passed to the regular snapshot restore mechanism with some specialized settings. Assume you have an existing snapshot repository and snapshot, the API looks like: ```json POST /my-new-index/_snapshot/mount?wait_for_completion=true { "repository": "my-s3-repo", "snapshot": "production-2019-12-11-aoeufao", "snapshot_index": "important-data", "index_settings": { "index.number_of_replicas": 2 }, "ignore_index_settings": ["index.refresh_interval"] } ``` While the request may appear similar to a snapshot restore request, it has slight differences. The `repository`, `snapshot`, and `snapshot_index` fields are required (all other fields are optional). The response is identical to the response when restoring a snapshot. Relates to elastic#50999
Pinging @elastic/es-distributed (:Distributed/Snapshot/Restore) |
Note for reviewers: this adds the API, but I tried to make as minimal changes as possible to the rest of the searchable snapshot infrastructure. This does not remove the "searchable" repository type, and only changes a few assertions made about the repository type where necessary to have mounting work with no issues. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks@dakrone, and thanks for having minimized the changes. I made a first bunch of comments (I didn't look at the tests yet).
.../src/main/java/org/elasticsearch/xpack/searchablesnapshots/SearchableSnapshotRepository.java
Outdated
Show resolved
Hide resolved
...n/java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotAction.java
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @dakrone, I left a few small comments too.
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
.../src/main/java/org/elasticsearch/xpack/searchablesnapshots/SearchableSnapshotRepository.java
Outdated
Show resolved
Hide resolved
...java/org/elasticsearch/xpack/searchablesnapshots/rest/RestMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
@elasticmachine please run elasticsearch-ci/bwc |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good, thanks for picking this work David. I've left some comments, but nothing to worry about.
...n/java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotAction.java
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
.../org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequestTests.java
Outdated
Show resolved
Hide resolved
Everything addressed @tlrx, I think, this is ready for another look. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM - if CI is happy, I'll be happy
This commit adds an API for mounting a snapshot as a searchable index.
Internally, a request is made to the repository to fetch metadata information necessary from the
snapshot in question, which is then passed to the regular snapshot restore mechanism with some
specialized settings.
Assume you have an existing snapshot repository and snapshot, the API looks like:
While the request may appear similar to a snapshot restore request, it has slight differences. The
repository
,snapshot
, andsnapshot_index
fields are required (all other fields are optional).The response is identical to the response when restoring a snapshot. In the above example the newly
restored/mounted index will be called
my-new-index
.Relates to #50999