Skip to content

Commit 326d696

Browse files
authored
Support offset in composite aggs (#50609)
Adds support for the `offset` parameter to the `date_histogram` source of composite aggs. The `offset` parameter is supported by the normal `date_histogram` aggregation and is useful for folks that need to measure things from, say, 6am one day to 6am the next day. This is implemented by creating a new `Rounding` that knows how to handle offsets and delegates to other rounding implementations. That implementation doesn't fully implement the `Rounding` contract, namely `nextRoundingValue`. That method isn't used by composite aggs so I can't be sure that any implementation that I add will be correct. I propose to leave it throwing `UnsupportedOperationException` until I need it. Closes #48757
1 parent e090b9b commit 326d696

File tree

12 files changed

+372
-47
lines changed

12 files changed

+372
-47
lines changed

docs/reference/aggregations/bucket/composite-aggregation.asciidoc

+66
Original file line numberDiff line numberDiff line change
@@ -287,6 +287,72 @@ Time zones may either be specified as an ISO 8601 UTC offset (e.g. `+01:00` or
287287
`-08:00`) or as a timezone id, an identifier used in the TZ database like
288288
`America/Los_Angeles`.
289289

290+
*Offset*
291+
292+
include::datehistogram-aggregation.asciidoc[tag=offset-explanation]
293+
294+
[source,console,id=composite-aggregation-datehistogram-offset-example]
295+
----
296+
PUT my_index/_doc/1?refresh
297+
{
298+
"date": "2015-10-01T05:30:00Z"
299+
}
300+
301+
PUT my_index/_doc/2?refresh
302+
{
303+
"date": "2015-10-01T06:30:00Z"
304+
}
305+
306+
GET my_index/_search?size=0
307+
{
308+
"aggs": {
309+
"my_buckets": {
310+
"composite" : {
311+
"sources" : [
312+
{
313+
"date": {
314+
"date_histogram" : {
315+
"field": "date",
316+
"calendar_interval": "day",
317+
"offset": "+6h",
318+
"format": "iso8601"
319+
}
320+
}
321+
}
322+
]
323+
}
324+
}
325+
}
326+
}
327+
----
328+
329+
include::datehistogram-aggregation.asciidoc[tag=offset-result-intro]
330+
331+
[source,console-result]
332+
----
333+
{
334+
...
335+
"aggregations": {
336+
"my_buckets": {
337+
"after_key": { "date": "2015-10-01T06:00:00.000Z" },
338+
"buckets": [
339+
{
340+
"key": { "date": "2015-09-30T06:00:00.000Z" },
341+
"doc_count": 1
342+
},
343+
{
344+
"key": { "date": "2015-10-01T06:00:00.000Z" },
345+
"doc_count": 1
346+
}
347+
]
348+
}
349+
}
350+
}
351+
----
352+
// TESTRESPONSE[s/\.\.\./"took": $body.took,"timed_out": false,"_shards": $body._shards,"hits": $body.hits,/]
353+
354+
include::datehistogram-aggregation.asciidoc[tag=offset-note]
355+
290356
===== Mixing different values source
291357

292358
The `sources` parameter accepts an array of values source.

docs/reference/aggregations/bucket/datehistogram-aggregation.asciidoc

+8-1
Original file line numberDiff line numberDiff line change
@@ -461,16 +461,19 @@ the bucket covering that day will only hold data for 23 hours instead of the usu
461461
where you'll have only a 11h bucket on the morning of 27 March when the DST shift
462462
happens.
463463

464+
[[search-aggregations-bucket-datehistogram-offset]]
464465
===== Offset
465466

467+
// tag::offset-explanation[]
466468
Use the `offset` parameter to change the start value of each bucket by the
467469
specified positive (`+`) or negative offset (`-`) duration, such as `1h` for
468470
an hour, or `1d` for a day. See <<time-units>> for more possible time
469471
duration options.
470472

471473
For example, when using an interval of `day`, each bucket runs from midnight
472-
to midnight. Setting the `offset` parameter to `+6h` changes each bucket
474+
to midnight. Setting the `offset` parameter to `+6h` changes each bucket
473475
to run from 6am to 6am:
476+
// end::offset-explanation[]
474477

475478
[source,console,id=datehistogram-aggregation-offset-example]
476479
-----------------------------
@@ -498,8 +501,10 @@ GET my_index/_search?size=0
498501
}
499502
-----------------------------
500503

504+
// tag::offset-result-intro[]
501505
Instead of a single bucket starting at midnight, the above request groups the
502506
documents into buckets starting at 6am:
507+
// end::offset-result-intro[]
503508

504509
[source,console-result]
505510
-----------------------------
@@ -525,8 +530,10 @@ documents into buckets starting at 6am:
525530
-----------------------------
526531
// TESTRESPONSE[s/\.\.\./"took": $body.took,"timed_out": false,"_shards": $body._shards,"hits": $body.hits,/]
527532

533+
// tag::offset-note[]
528534
NOTE: The start `offset` of each bucket is calculated after `time_zone`
529535
adjustments have been made.
536+
// end::offset-note[]
530537

531538
===== Keyed Response
532539

rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/230_composite.yml

+38-4
Original file line numberDiff line numberDiff line change
@@ -243,7 +243,7 @@ setup:
243243
---
244244
"Composite aggregation with format":
245245
- skip:
246-
version: " - 7.1.99"
246+
version: " - 7.99.99" # after BWC merged revert to 7.1.99
247247
reason: calendar_interval introduced in 7.2.0
248248
features: warnings
249249

@@ -309,7 +309,7 @@ setup:
309309
---
310310
"Composite aggregation with format and calendar_interval":
311311
- skip:
312-
version: " - 7.1.99"
312+
version: " - 7.99.99" # after BWC merged revert to 7.1.99
313313
reason: calendar_interval introduced in 7.2.0
314314

315315
- do:
@@ -367,6 +367,40 @@ setup:
367367
- match: { aggregations.test.buckets.0.key.date: "2017-10-21" }
368368
- match: { aggregations.test.buckets.0.doc_count: 1 }
369369

370+
---
371+
"Composite aggregation with date_histogram offset":
372+
- skip:
373+
version: " - 7.99.99" # after BWC merged revert to 7.5.99
374+
reason: offset introduced in 8.0.0
375+
376+
- do:
377+
search:
378+
rest_total_hits_as_int: true
379+
index: test
380+
body:
381+
aggregations:
382+
test:
383+
composite:
384+
sources: [
385+
{
386+
"date": {
387+
"date_histogram": {
388+
"field": "date",
389+
"calendar_interval": "1d",
390+
"offset": "4h",
391+
"format": "iso8601" # Format makes the comparisons a little more obvious
392+
}
393+
}
394+
}
395+
]
396+
397+
- match: {hits.total: 6}
398+
- length: { aggregations.test.buckets: 2 }
399+
- match: { aggregations.test.buckets.0.key.date: "2017-10-19T04:00:00.000Z" }
400+
- match: { aggregations.test.buckets.0.doc_count: 1 }
401+
- match: { aggregations.test.buckets.1.key.date: "2017-10-21T04:00:00.000Z" }
402+
- match: { aggregations.test.buckets.1.doc_count: 1 }
403+
370404
---
371405
"Composite aggregation with after_key in the response":
372406
- do:
@@ -667,7 +701,6 @@ setup:
667701
reason: geotile_grid is not supported until 7.5.0
668702
- do:
669703
search:
670-
rest_total_hits_as_int: true
671704
index: test
672705
body:
673706
aggregations:
@@ -690,7 +723,8 @@ setup:
690723
]
691724
after: { "geo": "12/730/1590", "kw": "foo" }
692725

693-
- match: {hits.total: 6}
726+
- match: { hits.total.value: 6 }
727+
- match: { hits.total.relation: "eq" }
694728
- length: { aggregations.test.buckets: 3 }
695729
- match: { aggregations.test.buckets.0.key.geo: "12/1236/1533" }
696730
- match: { aggregations.test.buckets.0.key.kw: "bar" }

0 commit comments

Comments
 (0)