Move shared cache pre-allocation and support macOS #70285
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit moves the shared cache pre-allocation code out of bootstrap, and instead to the searchable snapshots code. We got out of our way to only grant permissions to a specific library used for the pre-allocation. Additionally, to ensure our interfaces are sound, we add a macOS implementation based on fcntl and ftruncate.
|
The main purpose of this relocation is to keep this plugin-specific code out of bootstrap. Additionally, for some complicated reasons, we can't touch the roles until after plugins are done being loaded. This means that we can't make the validation fo these settings dependent on the node roles unless we relocate these settings out of bootstrap so that they are not touched until after plugins are done being loaded. This gives us a second reason to move this code, so we take those steps now. |
rjernst
approved these changes
Mar 11, 2021
ywelsch
reviewed
Mar 11, 2021
...src/main/java/org/elasticsearch/xpack/searchablesnapshots/preallocate/MacOsPreallocator.java
Outdated
Show resolved
Hide resolved
|
@ywelsch Thanks for making a pass, and especially for addressing precommit. 🙏 I took your suggestion to first attempt to allocate contiguously on macOS. I agree on adding testing, I'll do that in a follow-up. |
ywelsch
approved these changes
Mar 11, 2021
| fst.fst_length = new NativeLong(fileSize); | ||
| // first, try allocating contiguously | ||
| if (Natives.fcntl(fd, Natives.Fcntl.F_PREALLOCATE, fst) != 0) { | ||
| // that failed, so let us try allocating non-contigulsouly |
jasontedor
added
:Core/Infra/Core
elasticmachine
added
Team:Distributed (Obsolete)
|
Pinging @elastic/es-distributed (Team:Distributed) |
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Mar 11, 2021
This commit moves the shared cache pre-allocation code out of bootstrap, and instead to the searchable snapshots code. We go out of our way to only grant permissions to a specific library used for the pre-allocation. Additionally, to ensure our interfaces are sound, we add a macOS implementation based on fcntl and ftruncate. Co-authored-by: Yannick Welsch <[email protected]>
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Mar 11, 2021
This commit moves the shared cache pre-allocation code out of bootstrap, and instead to the searchable snapshots code. We go out of our way to only grant permissions to a specific library used for the pre-allocation. Additionally, to ensure our interfaces are sound, we add a macOS implementation based on fcntl and ftruncate. Co-authored-by: Yannick Welsch <[email protected]>
This was referenced Mar 11, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit moves the shared cache pre-allocation code out of bootstrap, and instead to the searchable snapshots code. We got out of our way to only grant permissions to a specific library used for the pre-allocation.
Additionally, to ensure our interfaces are sound, we add a macOS implementation based on fcntl and ftruncate.
Relates #68858