Skip to content

[GEO] Add CRS Support to Geo Field Mappers and QueryProcessors #47250

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 24 commits into from

Conversation

nknize
Copy link
Contributor

@nknize nknize commented Sep 27, 2019

This PR adds support for indexing and searching geo_shape data in its native coordinate reference system (CRS) without having to reproject to WGS84.

The following is an example mapping configuration for indexing in UTM Zone 14 N using the EPSG CRS definition:

“properties” : {
    “location” : {
        “type” : “geo_shape”, 
        "crs": {
            "type": "name",
            "properties": {
                "name": "EPSG:32614"  // UTM Zone 14N
            }
        }
    }
}

Users pass GeoJSON with coordinates in the native UTM Zone 14N projection which will index without loss of precision or having to reproject to WGS84.
Query coordinates must be passed in the same coordinate reference system.
All projections are handled using a new Proj4J third party dependency.
This feature is licensed Gold.

todo:

  • add docs
  • check impact on SQL plugin

relates #23221

@nknize nknize added >feature WIP :Analytics/Geo Indexing, search aggregations of geo points and shapes v8.0.0 7x labels Sep 27, 2019
@nknize nknize requested review from rjernst and imotov September 27, 2019 21:23
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo

@nknize nknize removed the WIP label Oct 7, 2019
@nknize
Copy link
Contributor Author

nknize commented Oct 7, 2019

Pending CI this should be ready for review

Copy link
Contributor

@imotov imotov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WRT GeoSql, supporting CRS can be a bit tricky. In GeoSql all supported CRS are described in the SPATIAL_REF_SYS table and the ID under which the CRS shows up in this table is then used everywhere in the interface to identify the CRS.

So, we will probably need to find a way to generate a comprehensive list of CRSes with stable numeric IDs, which might be a bit tricky since we generate this list dynamically. We will not be able to expose this list as a table, unless we are willing to create an internal index with them. So, instead we can add similar functionality as a command SHOW_SPATIAL_REF_SYS or something like this.

The second issues is that in GeoSql CRS id is an attribute of the geometry, so we will need to find a way to pass associate this information with the data that we get from elasticsearch.

The third issue is that we will need a way to perform a transformation of a geometry from its original CRS into another CRS specified by CRS's numeric ID, which means we will need to add a painless whitelisted method that will take 2 parameters: a CRS id and Geometry and return another Geometry converted from its original CRS into specified CRS.

@imotov imotov requested a review from iverase October 11, 2019 13:39
@nknize nknize added the WIP label Oct 11, 2019
@polyfractal polyfractal removed the 7x label Dec 12, 2019
Copy link
Member

@rjernst rjernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nknize I jsut realized I had a couple very minor comments pending here for months. Is this PR something still being worked on that needs review?

@nknize
Copy link
Contributor Author

nknize commented Feb 27, 2020

Thank you @rjernst! Yes this is still being worked but it's blocked by ongoing lucene work. I can close the PR and reopen once the lucene work is done, or we can merge this under the "more dev in master" philosophy and open a separate PR to integrate the Lucene portion once ready?

@rjernst
Copy link
Member

rjernst commented Feb 28, 2020

My preference would be close (or hold the PR open if you prefer) over pushing unfinished work to master. I only pinged here because I noticed I was marked to review. :)

@nknize nknize changed the title [GEO] Add CRS Support for Indexing and Searching in Native Projections [GEO] Add CRS Support to Geo Field Mappers and QueryProcessors Mar 17, 2020
@rjernst rjernst added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label May 4, 2020
@nknize nknize closed this Mar 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Geo Indexing, search aggregations of geo points and shapes >feature release highlight Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.0.0-alpha1 WIP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants