-
Notifications
You must be signed in to change notification settings - Fork 25.2k
Use alias name from rollover request to query indices stats #40774
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
In `TransportRolloverAction` before doing rollover we resolve source index name (write index) from the alias in the rollover request. Before evaluating the conditions and executing rollover action, we retrieve stats, but to do so we used the source index name resolved from the alias instead of what is in the request. This fails when user is assigned role with index privilege on the alias instead of concrete index. This commit fixes this by using the alias from the request. Closes elastic#40771
Pinging @elastic/es-security |
Pinging @elastic/es-core-features |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am game with the main code change.
I would love if you'd add an IntegTest
so that we don't catch authorization errors in Rest tests...
However, I think we should make the Rollover action use the client for index and alias creation. WDYT?
When same alias points to multiple indices we can write to only one index with `is_write_index` value `true`. The special handling for PutMappingRequest filtered out such aliases making the request unauthorized. The check has been modified to consider write index flag and only when the requested index matches with the one with write index alias. Closes elastic#40831
…or alias" This reverts commit 5733363.
Hi @albertzaharovits, Thanks for your comment
I did not see one for Rollover Action but I can add unit test to cover this. Thanks.
Could you please elaborate? as I did not understand what you have proposed here. Thank you. |
Ah, I'm sorry, I mean |
@talevy Are you able to review the (1 line) change to the transport action? It looks like you were the last person to make actual functional changes to that action, and I'd appreciate a review from someone who knows that code better than we do. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still need to test my comments, but I figure sending this out earlier is better. let me know if
what I said makes sense!
...r/src/main/java/org/elasticsearch/action/admin/indices/rollover/TransportRolloverAction.java
Show resolved
Hide resolved
x-pack/plugin/src/test/resources/rest-api-spec/test/security/authz/31_rollover_using_alias.yml
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just ran this check to assert my theory:
PUT alpha
{
"aliases": {
"my_alias": {
"is_write_index": true
}
}
}
PUT beta
{
"aliases": {
"my_alias": {
"is_write_index": false
}
}
}
PUT alpha/_doc/1?refresh
{
"foo": "bar"
}
PUT beta/_doc/1?refresh
{
"foo": "bar"
}
POST /my_alias/_rollover/new_index?dry_run
{
"conditions": {
"max_docs": 2
}
}
this returns an incorrect rollover evaluation (the write index "alpha" only has one document):
{
"acknowledged" : false,
"shards_acknowledged" : false,
"old_index" : "alpha",
"new_index" : "new_index",
"rolled_over" : false,
"dry_run" : true,
"conditions" : {
"[max_docs: 2]" : true
}
}
I think the additional change that would make this work would be this:
@@ -249,7 +249,7 @@ public class TransportRolloverAction extends TransportMasterNodeAction<RolloverR
static Map<String, Boolean> evaluateConditions(final Collection<Condition<?>> conditions, final IndexMetaData metaData,
final IndicesStatsResponse statsResponse) {
- return evaluateConditions(conditions, statsResponse.getPrimaries().getDocs(), metaData);
+ return evaluateConditions(conditions, statsResponse.getIndex(metaData.getIndex().getName()).getPrimaries().getDocs(), metaData);
}
The issue is that the code was looking at the total DocStats of all the indices returned in the stats-response. This change should make it so that we are looking at the sourceIndex only. The non-docstats condition (index age) is done on the correct IndexMetaData of the sourceIndex, so this should not be a problem.
When alias is used, we retrieve all the stats (including write + read indexes) so the earlier commit, made the stats evaluation on collective stats instead of considering only source index.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes to Rollover look good to me. I'm not against
getting another opinion regarding my comments around
performance concerns.
Due to the difficulty of controlling time for the additional test in RolloverIT,
what do you think of moving these additional tests as unit tests
in TransportRolloverActionTests
. That way we can control the
state of the indices such that only the write-index meets the
relevant criteria (max_age, max_docs, max_size).
- Add UT instead of IT to minimize impact on build times.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…40774) In `TransportRolloverAction` before doing rollover we resolve source index name (write index) from the alias in the rollover request. Before evaluating the conditions and executing rollover action, we retrieve stats, but to do so we used the source index name resolved from the alias instead of alias from the index. This fails when the user is assigned a role with index privilege on the alias instead of the concrete index. This commit fixes this by using the alias from the request. After this change, verified that when we retrieve all the stats (including write + read indexes) we are considering only source index. Closes elastic#40771
…40774) In `TransportRolloverAction` before doing rollover we resolve source index name (write index) from the alias in the rollover request. Before evaluating the conditions and executing rollover action, we retrieve stats, but to do so we used the source index name resolved from the alias instead of alias from the index. This fails when the user is assigned a role with index privilege on the alias instead of the concrete index. This commit fixes this by using the alias from the request. After this change, verified that when we retrieve all the stats (including write + read indexes) we are considering only source index. Closes elastic#40771
…41284) In `TransportRolloverAction` before doing rollover we resolve source index name (write index) from the alias in the rollover request. Before evaluating the conditions and executing rollover action, we retrieve stats, but to do so we used the source index name resolved from the alias instead of alias from the index. This fails when the user is assigned a role with index privilege on the alias instead of the concrete index. This commit fixes this by using the alias from the request. After this change, verified that when we retrieve all the stats (including write + read indexes) we are considering only source index. Closes #40771
…41285) In `TransportRolloverAction` before doing rollover we resolve source index name (write index) from the alias in the rollover request. Before evaluating the conditions and executing rollover action, we retrieve stats, but to do so we used the source index name resolved from the alias instead of alias from the index. This fails when the user is assigned a role with index privilege on the alias instead of the concrete index. This commit fixes this by using the alias from the request. After this change, verified that when we retrieve all the stats (including write + read indexes) we are considering only source index. Closes #40771
…40774) In `TransportRolloverAction` before doing rollover we resolve source index name (write index) from the alias in the rollover request. Before evaluating the conditions and executing rollover action, we retrieve stats, but to do so we used the source index name resolved from the alias instead of alias from the index. This fails when the user is assigned a role with index privilege on the alias instead of the concrete index. This commit fixes this by using the alias from the request. After this change, verified that when we retrieve all the stats (including write + read indexes) we are considering only source index. Closes elastic#40771
…41286) In `TransportRolloverAction` before doing rollover we resolve source index name (write index) from the alias in the rollover request. Before evaluating the conditions and executing rollover action, we retrieve stats, but to do so we used the source index name resolved from the alias instead of alias from the index. This fails when the user is assigned a role with index privilege on the alias instead of the concrete index. This commit fixes this by using the alias from the request. After this change, verified that when we retrieve all the stats (including write + read indexes) we are considering only source index. Closes #40771
…40774) In `TransportRolloverAction` before doing rollover we resolve source index name (write index) from the alias in the rollover request. Before evaluating the conditions and executing rollover action, we retrieve stats, but to do so we used the source index name resolved from the alias instead of alias from the index. This fails when the user is assigned a role with index privilege on the alias instead of the concrete index. This commit fixes this by using the alias from the request. After this change, verified that when we retrieve all the stats (including write + read indexes) we are considering only source index. Closes elastic#40771
In
TransportRolloverAction
before doing rollover we resolvesource index name (write index) from the alias in the rollover request.
Before evaluating the conditions and executing rollover action, we
retrieve stats, but to do so we used the source index name
resolved from the alias instead of alias from the index.
This fails when the user is assigned a role with index privilege on the
alias instead of the concrete index. This commit fixes this by using
the alias from the request.
Closes #40771