Skip to content

[CI] TermsAggregatorTests.testManyUniqueTerms failure #69413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
matriv opened this issue Feb 23, 2021 · 2 comments · Fixed by #69476
Closed

[CI] TermsAggregatorTests.testManyUniqueTerms failure #69413

matriv opened this issue Feb 23, 2021 · 2 comments · Fixed by #69476
Assignees
@nik9000
Labels
:Analytics/Aggregations Aggregations Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) >test-failure Triaged test failures from CI

Comments

@matriv
Copy link
Contributor

matriv commented Feb 23, 2021

Build scan:
https://gradle-enterprise.elastic.co/s/zs7wpexyuupcg

Repro line:

./gradlew ':server:test' --tests "org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorTests.testManyUniqueTerms" \
  -Dtests.seed=928BECEE0B850D70 \
  -Dtests.security.manager=true \
  -Dtests.locale=ar-SD \
  -Dtests.timezone=America/Indiana/Knox \
  -Druntime.java=8

Reproduces locally?:
YES

Applicable branches:
7.12

Failure history:
https://build-stats.elastic.co/app/kibana#/discover?_g=(refreshInterval:(pause:!t,value:0),time:(from:now-90d,mode:quick,to:now))&_a=(columns:!(_source),index:e58bf320-7efd-11e8-bf69-63c8ef516157,interval:auto,query:(language:lucene,query:testManyUniqueTerms),sort:!(time,desc))

Failure excerpt:

10:46:22   2> java.lang.AssertionError: 
10:46:22     Expected: <[b007, b107, b207, b307, b407, b507, b607, b707, b000, b001]>
10:46:22          but: was <[b007, b107, b207, b307, b407, b507, b707, b000, b001, b002]>
10:46:22         at __randomizedtesting.SeedInfo.seed([928BECEE0B850D70:E65ADE16B0EDEB46]:0)
10:46:22         at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
10:46:22         at org.junit.Assert.assertThat(Assert.java:956)
10:46:22         at org.junit.Assert.assertThat(Assert.java:923)
10:46:22         at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorTests.lambda$testManyUniqueTerms$6(TermsAggregatorTests.java:304)
10:46:22         at org.elasticsearch.search.aggregations.AggregatorTestCase.testCase(AggregatorTestCase.java:545)
10:46:22         at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorTests.testManyUniqueTerms(TermsAggregatorTests.java:295)
@matriv matriv added :Analytics/Aggregations Aggregations >test-failure Triaged test failures from CI labels Feb 23, 2021
@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Feb 23, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

@nik9000 nik9000 self-assigned this Feb 23, 2021
@nik9000
Copy link
Member

nik9000 commented Feb 23, 2021

Thanks for filing this! It's certainly my bug. I'll work on a fix this afternoon.

nik9000 added a commit to nik9000/elasticsearch that referenced this issue Feb 23, 2021
@nik9000
Fixup aggs test
2088cf6 
One of the tests that I added in elastic#68871 worked about 99.7% of the time
but on some seeds failed to generate the right buckets because on those
seeds we would collect each segment with its own aggregator and get bad
counts. This "bad counts" problem is known in the terms aggregator - its
the price we pay for distributed work. But we can work around it either
by forcing all the docs into a single segment or by collecting all of
the buckets on the shard. We want to test the code path where don't
collect all buckets on the shard so we opt for the former.

Closes elastic#69413
@nik9000 nik9000 mentioned this issue Feb 23, 2021
Merged
nik9000 added a commit that referenced this issue Feb 23, 2021
@nik9000
Fixup aggs test (#69476)
21edf4d 
One of the tests that I added in #68871 worked about 99.7% of the time
but on some seeds failed to generate the right buckets because on those
seeds we would collect each segment with its own aggregator and get bad
counts. This "bad counts" problem is known in the terms aggregator - its
the price we pay for distributed work. But we can work around it either
by forcing all the docs into a single segment or by collecting all of
the buckets on the shard. We want to test the code path where don't
collect all buckets on the shard so we opt for the former.

Closes #69413
nik9000 added a commit that referenced this issue Feb 23, 2021
@nik9000
Fixup aggs test (#69476)
4971ab1 
One of the tests that I added in #68871 worked about 99.7% of the time
but on some seeds failed to generate the right buckets because on those
seeds we would collect each segment with its own aggregator and get bad
counts. This "bad counts" problem is known in the terms aggregator - its
the price we pay for distributed work. But we can work around it either
by forcing all the docs into a single segment or by collecting all of
the buckets on the shard. We want to test the code path where don't
collect all buckets on the shard so we opt for the former.

Closes #69413
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nik9000 nik9000

Labels
:Analytics/Aggregations Aggregations Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixup aggs test nik9000/elasticsearch
3 participants
@nik9000 @matriv @elasticmachine