AutoDateHistogram should not merge buckets with a final reduce context #62731
Labels
:Analytics/Aggregations
Aggregations
>bug
Team:Analytics
Meta label for analytical engine team (ESQL/Aggs/Geo)
Comments
jimczi
added
>bug
:Analytics/Geo
|
Pinging @elastic/es-analytics-geo (:Analytics/Geo) |
elasticmachine
added
the
Team:Analytics
jimczi
added a commit
to jimczi/elasticsearch
that referenced
this issue
Sep 21, 2020
This commit ensures that the final order of the terms aggregations is registered correctly after the final reduce. This bug was introduced in elastic#62028 which is not released yet so this PR is marked as a non-issue. This issue was discovered when running a terms aggregation under an auto-date histogram. In such a case, the auto-date histogram may run multiple final reduce to merge buckets together. This change makes sure that running multiple final reduces doesn't create duplicates but it doesn't fix the fact that the final reduce may prune the list of terms prematurely. This other bug is tracked separately in elastic#62731.
jimczi
added a commit
that referenced
this issue
Sep 21, 2020
This commit ensures that the final order of the terms aggregations is registered correctly after the final reduce. This bug was introduced in #62028 which is not released yet so this PR is marked as a non-issue. This issue was discovered when running a terms aggregation under an auto-date histogram. In such a case, the auto-date histogram may run multiple final reduce to merge buckets together. This change makes sure that running multiple final reduces doesn't create duplicates but it doesn't fix the fact that the final reduce may prune the list of terms prematurely. This other bug is tracked separately in #62731.
jimczi
added a commit
that referenced
this issue
Sep 21, 2020
This commit ensures that the final order of the terms aggregations is registered correctly after the final reduce. This bug was introduced in #62028 which is not released yet so this PR is marked as a non-issue. This issue was discovered when running a terms aggregation under an auto-date histogram. In such a case, the auto-date histogram may run multiple final reduce to merge buckets together. This change makes sure that running multiple final reduces doesn't create duplicates but it doesn't fix the fact that the final reduce may prune the list of terms prematurely. This other bug is tracked separately in #62731.
iverase
added
:Analytics/Aggregations
15 tasks
|
Pinging @elastic/es-analytics-geo (Team:Analytics) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The AutoDateHistogram can perform final reduce multiple times when it needs to merge buckets together.
If a
termsaggregation is nested under an auto-date histogram, running multiple final reduce may result in a loss of precision since each final reduce will prune the list of terms prematurely. So if thetermsaggregation is sorted by count, the counts and ordering of the terms within the bucket may be wrong due to the merges that the auto-date histogram performed.We should add the ability to run partial reduce as part of a final reduce in order to allow the merging of buckets in any parent aggregation. That would allow the
termsaggregation to preserve all buckets when merging parent buckets incrementally in the auto-date histogram.The text was updated successfully, but these errors were encountered: