Skip to content

Save memory when date_histogram is not on top #56921

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 19, 2020
Merged

Save memory when date_histogram is not on top #56921

merged 2 commits into from
May 19, 2020

Conversation

nik9000
Copy link
Member

@nik9000 nik9000 commented May 18, 2020

When date_histogram is a sub-aggregator it used to allocate a bunch of
objects for every one of it's parent's buckets. This uses the data
structures that we built in #55873 rework the date_histogram
aggregator instead of all of the allocation.

Part of #56487.

@nik9000
Save memory when date_histogram is not on top
09e35fc 
When `date_histogram` is a sub-aggregator it used to allocate a bunch of
objects for every one of it's parent's buckets. This uses the data
structures that we built in elastic#55873 rework the `date_histogram`
aggregator instead of all of the allocation.

Part of elastic#56487
@nik9000 nik9000 requested a review from talevy May 18, 2020 19:58
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (:Analytics/Aggregations)

@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label May 18, 2020
@nik9000
Copy link
Member Author

nik9000 commented May 18, 2020 •
edited
Loading

Unlike when I did this for significant_term (#56789) or terms (#55873) this doesn't really make the agg go any faster. The performance change is pretty much a wash so far as I can tell. It is a little worse when there are maybe 5 or 10 owning buckets and a little better when there are hundreds. But it is something we can build on!

This was referenced May 18, 2020
Closed
Merged
@nik9000 nik9000 merged commit bea2341 into elastic:master May 19, 2020
@nik9000
Copy link
Member Author

nik9000 commented May 19, 2020

Thanks @polyfractal !

nik9000 added a commit to nik9000/elasticsearch that referenced this pull request May 19, 2020
@nik9000
Save memory when date_histogram is not on top (elastic#56921)
272f9ee 
When `date_histogram` is a sub-aggregator it used to allocate a bunch of
objects for every one of it's parent's buckets. This uses the data
structures that we built in elastic#55873 rework the `date_histogram`
aggregator instead of all of the allocation.

Part of elastic#56487
@nik9000
Copy link
Member Author

nik9000 commented May 19, 2020

Thanks @talevy !

nik9000 added a commit that referenced this pull request May 19, 2020
@nik9000
Save memory when date_histogram is not on top (#56921) (#56960)
8b9c4eb 
When `date_histogram` is a sub-aggregator it used to allocate a bunch of
objects for every one of it's parent's buckets. This uses the data
structures that we built in #55873 rework the `date_histogram`
aggregator instead of all of the allocation.

Part of #56487
nik9000 added a commit to nik9000/elasticsearch that referenced this pull request May 19, 2020
@nik9000
Update skip after backport of elastic#56921
b93107d
nik9000 added a commit that referenced this pull request May 20, 2020
@nik9000
Update skip after backport of #56921 (#56974)
7790e81
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@talevy talevy talevy approved these changes

Assignees
No one assigned
Labels
:Analytics/Aggregations Aggregations >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v7.9.0 v8.0.0-alpha1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@nik9000 @elasticmachine @talevy @jakelandis