Skip to content

Fix Edge Case Datastream Snapshot Create Bug #72747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

original-brownbear
Copy link
Member

We always clean up the list of datastreams depending on the indices actually
in the snapshot. Even for non-partial snapshots datastream indices could be
excluded from the snapshot by index exclusions that remove all the datastream's
indices from the snapshot.

We always clean up the list of datastreams depending on the indices acutally
in the snapshot. Even for non-partial snapshots datastream indices could be
excluded from the snapshot by index exlusions that remove all the datastream's
indices from the snapshot.
@original-brownbear original-brownbear added >bug :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs v8.0.0 :Data Management/Data streams Data streams and their lifecycles v7.14.0 v7.13.1 labels May 5, 2021
@elasticmachine elasticmachine added Team:Data Management Meta label for data/management team Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. labels May 5, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (Team:Core/Features)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

Copy link
Contributor

@danhermann danhermann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM. Thanks for addressing it, @original-brownbear!

Tangentially related, but I wonder if we should document that snapshots of data streams can include a subset of their current backing indices and upon restore can have a different write index. If so, I can open a PR for that.

@original-brownbear
Copy link
Member Author

Thanks Dan! and ++ on documenting this better.

@original-brownbear original-brownbear merged commit e683d71 into elastic:master May 5, 2021
@original-brownbear original-brownbear deleted the ds-and-non-partial-snapshots-bug branch May 5, 2021 14:07
original-brownbear added a commit to original-brownbear/elasticsearch that referenced this pull request May 5, 2021
We always clean up the list of datastreams depending on the indices acutally
in the snapshot. Even for non-partial snapshots datastream indices could be
excluded from the snapshot by index exlusions that remove all the datastream's
indices from the snapshot.
original-brownbear added a commit to original-brownbear/elasticsearch that referenced this pull request May 5, 2021
We always clean up the list of datastreams depending on the indices acutally
in the snapshot. Even for non-partial snapshots datastream indices could be
excluded from the snapshot by index exlusions that remove all the datastream's
indices from the snapshot.
original-brownbear added a commit that referenced this pull request May 5, 2021
We always clean up the list of datastreams depending on the indices acutally
in the snapshot. Even for non-partial snapshots datastream indices could be
excluded from the snapshot by index exlusions that remove all the datastream's
indices from the snapshot.
original-brownbear added a commit that referenced this pull request May 5, 2021
We always clean up the list of datastreams depending on the indices acutally
in the snapshot. Even for non-partial snapshots datastream indices could be
excluded from the snapshot by index exlusions that remove all the datastream's
indices from the snapshot.
@danhermann
Copy link
Contributor

See #72761

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Data Management/Data streams Data streams and their lifecycles :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Data Management Meta label for data/management team Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. v7.13.0 v7.14.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants