Simulate shard moves using cluster_concurrent_rebalance=2 #93977
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BalancedShardAllocator is prone to perform unnecessary moves when cluster_concurrent_rebalance is set to high values (>2). (See elastic#87279) Above allocator is used in DesiredBalanceComputer. Since we do not move actual shard data during calculation it is possible to artificially set above setting to 1 to avoid unnecessary moves in desired balance.
idegtiarenko
added
>enhancement
:Distributed Coordination/Allocation
|
Pinging @elastic/es-distributed (Team:Distributed) |
|
Hi @idegtiarenko, I've created a changelog YAML for you. |
idegtiarenko
changed the title
DaveCTurner
reviewed
Feb 21, 2023
Comment on lines
463
to
466
| assumeTrue( | ||
| "Only fixed in DesiredBalanceShardsAllocator", | ||
| allocationService.shardsAllocator instanceof DesiredBalanceShardsAllocator | ||
| ); |
There was a problem hiding this comment.
I think it would be nicer to change it so that org.elasticsearch.cluster.ESAllocationTestCase#createShardsAllocator respects the cluster.routing.allocation.type setting if set.
DaveCTurner
approved these changes
Feb 21, 2023
| @@ -62,10 +62,17 @@ public Decision canRebalance(ShardRouting shardRouting, RoutingAllocation alloca | |||
|
|
|||
| @Override | |||
| public Decision canRebalance(RoutingAllocation allocation) { | |||
| int relocatingShards = allocation.routingNodes().getRelocatingShardCount(); | |||
| if (allocation.isSimulating() && relocatingShards >= 1) { | |||
There was a problem hiding this comment.
Maybe we should use 2 here just for consistency with the past default behaviour? I think I've seen odd things happening at 1 sometimes.
There was a problem hiding this comment.
Is there a chance you could describe or link some of them?
May be we could add some tests for them and assess the better value
idegtiarenko
changed the title
# Conflicts: # test/framework/src/main/java/org/elasticsearch/cluster/ESAllocationTestCase.java
|
@elasticsearchmachine please run elasticsearch-ci/bwc |
💚 Backport successful
|
idegtiarenko
added a commit
to idegtiarenko/elasticsearch
that referenced
this pull request
Feb 23, 2023
) BalancedShardAllocator is prone to perform unnecessary moves when cluster_concurrent_rebalance is set to high values (>2). (See elastic#87279) Above allocator is used in DesiredBalanceComputer. Since we do not move actual shard data during calculation it is possible to artificially set above setting to 2 to avoid unnecessary moves in desired balance.
elasticsearchmachine
pushed a commit
that referenced
this pull request
Mar 1, 2023
…) (#94082) * Simulate shard moves using cluster_concurrent_rebalance=2 (#93977) BalancedShardAllocator is prone to perform unnecessary moves when cluster_concurrent_rebalance is set to high values (>2). (See #87279) Above allocator is used in DesiredBalanceComputer. Since we do not move actual shard data during calculation it is possible to artificially set above setting to 2 to avoid unnecessary moves in desired balance. * fix merge --------- Co-authored-by: Elastic Machine <[email protected]>
33 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
BalancedShardAllocator is prone to perform unnecessary moves when cluster_concurrent_rebalance is set to high values (>2). (See #87279)
Above allocator is used in DesiredBalanceComputer. Since we do not move actual shard data during calculation
it is possible to artificially set above setting to 1 to avoid unnecessary moves in desired balance.
Related to: #87279