Skip to content

[7.x] Adds support for geo-bounds filtering in geogrid aggregations #50996

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 14, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,62 @@ var bbox = geohash.decode_bbox('u17');
--------------------------------------------------
// NOTCONSOLE

==== Requests with additional bounding box filtering

The `geohash_grid` aggregation supports an optional `bounds` parameter
that restricts the points considered to those that fall within the
bounds provided. The `bounds` parameter accepts the bounding box in
all the same <<query-dsl-geo-bounding-box-query-accepted-formats,accepted formats>> of the
bounds specified in the Geo Bounding Box Query. This bounding box can be used with or
without an additional `geo_bounding_box` query filtering the points prior to aggregating.
It is an independent bounding box that can intersect with, be equal to, or be disjoint
to any additional `geo_bounding_box` queries defined in the context of the aggregation.

[source,console,id=geohashgrid-aggregation-with-bounds]
--------------------------------------------------
POST /museums/_search?size=0
{
"aggregations" : {
"tiles-in-bounds" : {
"geohash_grid" : {
"field" : "location",
"precision" : 8,
"bounds": {
"top_left" : "53.4375, 4.21875",
"bottom_right" : "52.03125, 5.625"
}
}
}
}
}
--------------------------------------------------
// TEST[continued]

[source,console-result]
--------------------------------------------------
{
...
"aggregations" : {
"tiles-in-bounds" : {
"buckets" : [
{
"key" : "u173zy3j",
"doc_count" : 1
},
{
"key" : "u173zvfz",
"doc_count" : 1
},
{
"key" : "u173zt90",
"doc_count" : 1
}
]
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\./"took": $body.took,"_shards": $body._shards,"hits":$body.hits,"timed_out":false,/]

==== Cell dimensions at the equator
The table below shows the metric dimensions for cells covered by various string lengths of geohash.
Expand Down Expand Up @@ -230,6 +286,8 @@ precision:: Optional. The string length of the geohashes used to define
to precision levels higher than the supported 12 levels,
(e.g. for distances <5.6cm) the value is rejected.

bounds: Optional. The bounding box to filter the points in the bucket.

size:: Optional. The maximum number of geohash buckets to return
(defaults to 10,000). When results are trimmed, buckets are
prioritised based on the volumes of documents they contain.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,62 @@ POST /museums/_search?size=0
--------------------------------------------------
// TESTRESPONSE[s/\.\.\./"took": $body.took,"_shards": $body._shards,"hits":$body.hits,"timed_out":false,/]

==== Requests with additional bounding box filtering

The `geotile_grid` aggregation supports an optional `bounds` parameter
that restricts the points considered to those that fall within the
bounds provided. The `bounds` parameter accepts the bounding box in
all the same <<query-dsl-geo-bounding-box-query-accepted-formats,accepted formats>> of the
bounds specified in the Geo Bounding Box Query. This bounding box can be used with or
without an additional `geo_bounding_box` query filtering the points prior to aggregating.
It is an independent bounding box that can intersect with, be equal to, or be disjoint
to any additional `geo_bounding_box` queries defined in the context of the aggregation.

[source,console,id=geotilegrid-aggregation-with-bounds]
--------------------------------------------------
POST /museums/_search?size=0
{
"aggregations" : {
"tiles-in-bounds" : {
"geotile_grid" : {
"field" : "location",
"precision" : 22,
"bounds": {
"top_left" : "52.4, 4.9",
"bottom_right" : "52.3, 5.0"
}
}
}
}
}
--------------------------------------------------
// TEST[continued]

[source,console-result]
--------------------------------------------------
{
...
"aggregations" : {
"tiles-in-bounds" : {
"buckets" : [
{
"key" : "22/2154412/1378379",
"doc_count" : 1
},
{
"key" : "22/2154385/1378332",
"doc_count" : 1
},
{
"key" : "22/2154259/1378425",
"doc_count" : 1
}
]
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\./"took": $body.took,"_shards": $body._shards,"hits":$body.hits,"timed_out":false,/]

==== Options

Expand All @@ -172,6 +228,8 @@ precision:: Optional. The integer zoom of the key used to define
cells/buckets in the results. Defaults to 7.
Values outside of [0,29] will be rejected.

bounds: Optional. The bounding box to filter the points in the bucket.

size:: Optional. The maximum number of geohash buckets to return
(defaults to 10,000). When results are trimmed, buckets are
prioritised based on the volumes of documents they contain.
Expand Down
1 change: 1 addition & 0 deletions docs/reference/query-dsl/geo-bounding-box-query.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ be executed in memory or indexed. See <<geo-bbox-type,Type>> below for further d
Default is `memory`.
|=======================================================================

[[query-dsl-geo-bounding-box-query-accepted-formats]]
[float]
==== Accepted Formats

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,11 @@ public GeoBoundingBox(StreamInput input) throws IOException {
this.bottomRight = input.readGeoPoint();
}

public boolean isUnbounded() {
return Double.isNaN(topLeft.lon()) || Double.isNaN(topLeft.lat())
|| Double.isNaN(bottomRight.lon()) || Double.isNaN(bottomRight.lat());
}

public GeoPoint topLeft() {
return topLeft;
}
Expand Down Expand Up @@ -120,6 +125,26 @@ public XContentBuilder toXContentFragment(XContentBuilder builder, boolean build
return builder;
}

/**
* If the bounding box crosses the date-line (left greater-than right) then the
* longitude of the point need only to be higher than the left or lower
* than the right. Otherwise, it must be both.
*
* @param lon the longitude of the point
* @param lat the latitude of the point
* @return whether the point (lon, lat) is in the specified bounding box
*/
public boolean pointInBounds(double lon, double lat) {
if (lat >= bottom() && lat <= top()) {
if (left() <= right()) {
return lon >= left() && lon <= right();
} else {
return lon >= left() || lon <= right();
}
}
return false;
}

@Override
public void writeTo(StreamOutput out) throws IOException {
out.writeGeoPoint(topLeft);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ public static boolean isValidLongitude(double longitude) {
return true;
}


/**
* Calculate the width (in meters) of geohash cells at a specific level
* @param level geohash level must be greater or equal to zero
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,10 @@

package org.elasticsearch.search.aggregations.bucket.composite;

import org.elasticsearch.Version;
import org.elasticsearch.common.ParseField;
import org.elasticsearch.common.geo.GeoBoundingBox;
import org.elasticsearch.common.geo.GeoPoint;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.xcontent.ObjectParser;
Expand All @@ -45,6 +48,8 @@ public class GeoTileGridValuesSourceBuilder extends CompositeValuesSourceBuilder
static {
PARSER = new ObjectParser<>(GeoTileGridValuesSourceBuilder.TYPE);
PARSER.declareInt(GeoTileGridValuesSourceBuilder::precision, new ParseField("precision"));
PARSER.declareField(((p, builder, context) -> builder.geoBoundingBox(GeoBoundingBox.parseBoundingBox(p))),
GeoBoundingBox.BOUNDS_FIELD, ObjectParser.ValueType.OBJECT);
CompositeValuesSourceParserHelper.declareValuesSourceFields(PARSER, ValueType.NUMERIC);
}

Expand All @@ -53,6 +58,7 @@ static GeoTileGridValuesSourceBuilder parse(String name, XContentParser parser)
}

private int precision = GeoTileGridAggregationBuilder.DEFAULT_PRECISION;
private GeoBoundingBox geoBoundingBox = new GeoBoundingBox(new GeoPoint(Double.NaN, Double.NaN), new GeoPoint(Double.NaN, Double.NaN));

GeoTileGridValuesSourceBuilder(String name) {
super(name);
Expand All @@ -61,13 +67,21 @@ static GeoTileGridValuesSourceBuilder parse(String name, XContentParser parser)
GeoTileGridValuesSourceBuilder(StreamInput in) throws IOException {
super(in);
this.precision = in.readInt();
if (in.getVersion().onOrAfter(Version.V_7_6_0)) {
this.geoBoundingBox = new GeoBoundingBox(in);
}
}

public GeoTileGridValuesSourceBuilder precision(int precision) {
this.precision = GeoTileUtils.checkPrecisionRange(precision);
return this;
}

public GeoTileGridValuesSourceBuilder geoBoundingBox(GeoBoundingBox geoBoundingBox) {
this.geoBoundingBox = geoBoundingBox;
return this;
}

@Override
public GeoTileGridValuesSourceBuilder format(String format) {
throw new IllegalArgumentException("[format] is not supported for [" + TYPE + "]");
Expand All @@ -76,21 +90,31 @@ public GeoTileGridValuesSourceBuilder format(String format) {
@Override
protected void innerWriteTo(StreamOutput out) throws IOException {
out.writeInt(precision);
if (out.getVersion().onOrAfter(Version.V_7_6_0)) {
geoBoundingBox.writeTo(out);
}
}

@Override
protected void doXContentBody(XContentBuilder builder, Params params) throws IOException {
builder.field("precision", precision);
if (geoBoundingBox.isUnbounded() == false) {
geoBoundingBox.toXContent(builder, params);
}
}

@Override
String type() {
return TYPE;
}

GeoBoundingBox geoBoundingBox() {
return geoBoundingBox;
}

@Override
public int hashCode() {
return Objects.hash(super.hashCode(), precision);
return Objects.hash(super.hashCode(), precision, geoBoundingBox);
}

@Override
Expand All @@ -99,7 +123,8 @@ public boolean equals(Object obj) {
if (obj == null || getClass() != obj.getClass()) return false;
if (super.equals(obj) == false) return false;
GeoTileGridValuesSourceBuilder other = (GeoTileGridValuesSourceBuilder) obj;
return precision == other.precision;
return Objects.equals(precision,other.precision)
&& Objects.equals(geoBoundingBox, other.geoBoundingBox);
}

@Override
Expand All @@ -112,7 +137,7 @@ protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardCon
ValuesSource.GeoPoint geoPoint = (ValuesSource.GeoPoint) orig;
// is specified in the builder.
final MappedFieldType fieldType = config.fieldContext() != null ? config.fieldContext().fieldType() : null;
CellIdSource cellIdSource = new CellIdSource(geoPoint, precision, GeoTileUtils::longEncode);
CellIdSource cellIdSource = new CellIdSource(geoPoint, precision, geoBoundingBox, GeoTileUtils::longEncode);
return new CompositeValuesSourceConfig(name, fieldType, cellIdSource, DocValueFormat.GEOTILE, order(),
missingBucket(), script() != null);
} else {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.search.aggregations.bucket.geogrid;

import org.elasticsearch.common.geo.GeoBoundingBox;
import org.elasticsearch.index.fielddata.MultiGeoPointValues;

/**
* Class representing {@link CellValues} whose values are filtered
* according to whether they are within the specified {@link GeoBoundingBox}.
*
* The specified bounding box is assumed to be bounded.
*/
class BoundedCellValues extends CellValues {

private final GeoBoundingBox geoBoundingBox;

protected BoundedCellValues(MultiGeoPointValues geoValues, int precision, CellIdSource.GeoPointLongEncoder encoder,
GeoBoundingBox geoBoundingBox) {
super(geoValues, precision, encoder);
this.geoBoundingBox = geoBoundingBox;
}


@Override
int advanceValue(org.elasticsearch.common.geo.GeoPoint target, int valuesIdx) {
if (geoBoundingBox.pointInBounds(target.getLon(), target.getLat())) {
values[valuesIdx] = encoder.encode(target.getLon(), target.getLat(), precision);
return valuesIdx + 1;
}
return valuesIdx;
}
}
Loading