Comprehensively test supported/unsupported field type:agg combinations #52493

polyfractal · 2020-02-18T21:37:09Z

This adds a test to AggregatorTestCase that allows us to programmatically verify that an aggregator supports or does not support all available field types. It fetches the list of registered field type parsers, creates a MappedFieldType from the parser and then attempts to run a basic agg against the field.

A supplied list of supported VSTypes are then compared against the output (success or exception) and suceeds or fails the test accordingly.

Still a WIP, need to tidy up, add javadocs, add another "main" agg or two (histo, etc). Putting it up in case folks want to have a look before I spend more time on it.

Note: to make this work, the PR cherry-picks over the getValuesSourceType() component of #51503. This will be added in eventually from the VS refactor branch, but is technically "unused" code outside of the tests until then.

Note 2: does not support field types added from a module at the moment.

This adds a test to AggregatorTestCase that allows us to programmatically verify that an aggregator supports or does not support a particular field type. It fetches the list of registered field type parsers, creates a MappedFieldType from the parser and then attempts to run a basic agg against the field. A supplied list of supported VSTypes are then compared against the output (success or exception) and suceeds or fails the test accordingly.

elasticmachine · 2020-02-18T21:37:12Z

Pinging @elastic/es-analytics-geo (:Analytics/Aggregations)

polyfractal · 2020-02-18T21:38:10Z

test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java

+        return Collections.emptyList();
+    }
+
+    public void testSupportedFieldTypes() throws IOException {


The meat of the PR is here.

polyfractal · 2020-02-18T21:38:53Z

server/src/main/java/org/elasticsearch/index/mapper/RangeFieldMapper.java

@@ -92,14 +94,6 @@ public RangeFieldType fieldType() {
            return (RangeFieldType)fieldType;
        }

-        @Override
-        public Builder docValues(boolean docValues) {


Note: we think this is a bug, I'll open a separate PR fixing this

test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java

server/src/main/java/org/elasticsearch/search/aggregations/support/CoreValuesSourceType.java

test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java

polyfractal · 2020-02-19T18:29:43Z

Tidied up, added javadocs, and cleaned up the comments that were left.

For posterity, we also decided to remove the CoreValuesSourceTypes that weren't in use (IP, BOOLEAN, DATE) and return the appropriate type today (BYTES/NUMERIC) so that we didn't have dead code hanging around.

not-napoleon

Looks good, I left two small comments. Really excited to get this in!

test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java

…ions/AggregatorTestCase.java Co-Authored-By: Mark Tozzi <[email protected]>

polyfractal · 2020-02-20T17:17:38Z

@elasticmachine run elasticsearch-ci/1

elastic#52493) This adds a test to AggregatorTestCase that allows us to programmatically verify that an aggregator supports or does not support a particular field type. It fetches the list of registered field type parsers, creates a MappedFieldType from the parser and then attempts to run a basic agg against the field. A supplied list of supported VSTypes are then compared against the output (success or exception) and suceeds or fails the test accordingly. Co-Authored-By: Mark Tozzi <[email protected]> * Skip fields that are not aggregatable

…nations (#54451) * Comprehensively test supported/unsupported field type:agg combinations (#52493) This adds a test to AggregatorTestCase that allows us to programmatically verify that an aggregator supports or does not support a particular field type. It fetches the list of registered field type parsers, creates a MappedFieldType from the parser and then attempts to run a basic agg against the field. A supplied list of supported VSTypes are then compared against the output (success or exception) and suceeds or fails the test accordingly. Co-Authored-By: Mark Tozzi <[email protected]> * Skip fields that are not aggregatable * Use newIndexSearcher() to avoid incompatible readers (#52723) Lucene's `newSearcher()` can generate readers like ParallelCompositeReader which we can't use. We need to instead use our helper `newIndexSearcher`

$polyfractal$

not-napoleon and others added 2 commits February 18, 2020 16:30

$@polyfractal$

Put values source types on fields (elastic#51503)

5ad9a96

$@polyfractal$ polyfractal added >test Issues or PRs that are addressing/adding tests WIP :Analytics/Aggregations Aggregations labels Feb 18, 2020

$polyfractal$

polyfractal commented Feb 18, 2020

View reviewed changes