Skip to content

Provide a way to disable allocation awareness for specific indices #13780

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ppf2 opened this issue Sep 24, 2015 · 3 comments
Closed

Provide a way to disable allocation awareness for specific indices #13780

ppf2 opened this issue Sep 24, 2015 · 3 comments
Labels
discuss :Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) >enhancement

Comments

@ppf2
Copy link
Member

ppf2 commented Sep 24, 2015

Details on the use case is documented in #12431. Providing an option to disable allocation awareness at the index level will help address such use case.

@lcawl lcawl added :Distributed Indexing/Distributed A catch all label for anything in the Distributed Indexing Area. Please avoid if you can. and removed :Allocation labels Feb 13, 2018
@clintongormley clintongormley added :Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) and removed :Distributed Indexing/Distributed A catch all label for anything in the Distributed Indexing Area. Please avoid if you can. labels Feb 14, 2018
@DaveCTurner
Copy link
Contributor

This is an interesting idea, but since its opening we have not seen enough feedback that it is something we should work on. We prefer to close this issue as a clear indication that we are not going to work on this at this time. We are always open to reconsidering this in the future based on compelling feedback; despite this issue being closed please feel free to leave feedback (including +1s).

@jdcrown
Copy link

jdcrown commented Jan 9, 2020

This is typically a problem with multiple availability zones and auto balancing evenness when an index has "auto_expand_replicas: 0-n" set such as the case with .security.

So if i am using a single autoscaling group in AWS that has multiple availability zones, AWS does not guarantee evenness, it's mainly based on availability. This can wreak havoc on the .security index in this case.

@DaveCTurner
Copy link
Contributor

From the AWS docs:

Amazon EC2 Auto Scaling attempts to distribute instances evenly between the Availability Zones that are enabled for your Auto Scaling group.

Although there's no guarantee of evenness, an uneven distribution across AZs represents a failure state that should be resolved in due course.

Additionally, since 6.5.0 the .security-* indices default to auto_expand_replicas: 0-1 (cf. #33131). If you have two or more AZs and have allocation awareness enabled then this ensures that there's no more than one copy in each AZ, which should be sufficient (cf. discussion in #29712). If you are asking for more replicas (i.e. auto_expand_replicas: 0-n where n≥2) and struggling to get them all allocated then I suspect you're setting n too high.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss :Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) >enhancement
Projects
None yet
Development

No branches or pull requests

5 participants