Skip to content

Stateless primary relocations #97162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

DaveCTurner
Copy link
Contributor

Adds support for proper online relocations of stateless primaries.

Adds support for proper online relocations of stateless primaries.
@DaveCTurner DaveCTurner added >non-issue :Distributed Indexing/Recovery Anything around constructing a new shard, either from a local or a remote source. v8.10.0 labels Jun 28, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Jun 28, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

Copy link
Contributor

@kingherc kingherc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've seen most of this before, so LGTM. One comment on the test only.


assertThat(routingNodes.node("node-1").getByShardId(shardId), nullValue());
assertThat(routingNodes.node("node-2").getByShardId(shardId), nullValue());
assertThat(routingNodes.node("node-1").getByShardId(shardId).state(), equalTo(RELOCATING));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only part that I had not seen before in your changes. I see we're asserting an intermediate state here where the relocation has not fully completed yet (it's in transit from node-1 to node-3). Can we also wait for the completion of the relocation here and assert that it's finally only in node-3?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we hadn't run these tests before, only the serverless ones, so I expect there might be a few changes needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no actual relocation in progress here, this is only to verify that the routing table changes as we expect. This test (and the one above) were added to support the temporary change that triggered stateless relocations by just closing all the shards, so now really this test can say that the routing table behaves the same for stateless and regular relocations. I've removed the duplication in d4ca54f.

@DaveCTurner
Copy link
Contributor Author

@elasticmachine please run elasticsearch-ci/part-1 (#97183)
@elasticmachine please run elasticsearch-ci/part-3 (#80703)

@DaveCTurner DaveCTurner added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Jun 28, 2023
@elasticsearchmachine elasticsearchmachine merged commit 9453d69 into elastic:main Jun 28, 2023
@DaveCTurner DaveCTurner deleted the 2023-06-28-stateless-primary-recovery branch June 28, 2023 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) :Distributed Indexing/Recovery Anything around constructing a new shard, either from a local or a remote source. >non-issue Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. v8.10.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants