Shape field type cannot handle polygons with high precision coordinates

[filipski]

The bug is an outcome of https://discuss.elastic.co/t/coordinates-precision/285434

Elasticsearch version (bin/elasticsearch --version): 7.12.1

OS version (uname -a if on a Unix-like system): Linux 5.4.0-86-generic #97~18.04.1-Ubuntu SMP Sat Sep 18 03:11:22 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:

Steps to reproduce:

1. Try indexing the following valid polygon, which has no intersections when checked with full 6-digit decimals in the coordinates:

"geometry": {
        "coordinates": [
          [
            [
              386.415622,
              565.6094
            ],
            [
              513.024902,
              538.960938
            ],
            [
              619.571655,
              520.635315
            ],
            [
              767.199768,
              506.289001
            ],
            [
              907.886047,
              497.958893
            ],
            [
              1099.212891,
              498.901306
            ],
            [
              1195.205688,
              507.045593
            ],
            [
              1203.445435,
              505.58371
            ],
            [
              1203.573026,
              501.947307
            ],
            [
              1203.57312,
              501.944641
            ],
            [
              1190.453547,
              500.902579
            ],
            [
              1190.288452,
              504.786316
            ],
            [
              1139.45752,
              500.216583
            ],
            [
              1139.355469,
              494.291016
            ],
            [
              1124.537231,
              493.161346
            ],
            [
              1124.670044,
              496.816101
            ],
            [
              1121.94751,
              496.698059
            ],
            [
              1122.118594,
              492.650836
            ],
            [
              1101.510376,
              491.718231
            ],
            [
              1101.712769,
              495.619873
            ],
            [
              1098.656128,
              495.619873
            ],
            [
              1098.757252,
              491.293826
            ],
            [
              974.669443,
              489.195082
            ],
            [
              974.698608,
              490.455017
            ],
            [
              971.798035,
              490.760345
            ],
            [
              971.773067,
              488.879509
            ],
            [
              921.286339,
              489.272402
            ],
            [
              921.419617,
              493.355591
            ],
            [
              918.366394,
              493.355591
            ],
            [
              918.199252,
              489.296426
            ],
            [
              903.612801,
              489.409939
            ],
            [
              903.546936,
              490.068665
            ],
            [
              885.618896,
              489.967926
            ],
            [
              885.629896,
              489.549884
            ],
            [
              746.015381,
              496.311676
            ],
            [
              745.867432,
              498.244385
            ],
            [
              746.673218,
              506.301941
            ],
            [
              726.327942,
              508.114868
            ],
            [
              725.670191,
              500.222176
            ],
            [
              706.98449,
              501.543385
            ],
            [
              707.3927,
              506.301941
            ],
            [
              704.773987,
              506.503357
            ],
            [
              704.409956,
              501.725423
            ],
            [
              608.29303,
              512.598511
            ],
            [
              520.062988,
              528.713562
            ],
            [
              500.450024,
              531.96851
            ],
            [
              501.859924,
              540.114563
            ],
            [
              499.569977,
              540.57251
            ],
            [
              498.184334,
              532.605034
            ],
            [
              463.674513,
              542.300228
            ],
            [
              463.945648,
              543.781555
            ],
            [
              459.161255,
              545.642151
            ],
            [
              458.832333,
              543.660591
            ],
            [
              405.887587,
              558.534896
            ],
            [
              388.898793,
              564.702453
            ],
            [
              388.531563,
              564.835771
            ],
            [
              386.37985,
              565.616922
            ],
            [
              386.379814,
              565.616935
            ],
            [
              386.379814,
              565.616937
            ],
            [
              386.381351,
              565.616614
            ],
            [
              386.381356,
              565.616612
            ],
            [
              386.393477,
              565.614061
            ],
            [
              386.415622,
              565.6094
            ]
          ]
        ],
        "type": "Polygon"
      }

2. Check the error log from Elastic and observe the 5-digit precision and rounding issues causing self-intersections in the polygon with those updated coordinates

Provide logs (if relevant):

{'timestamp': '2021-09-27T14:50:46.427812Z', 'status': 400, 'error': 'Bad Request', 'message': '{"hint":"Ensure that this is shape has no self-intersections. If it has, split it into several sub-shapes in such a way that no self-intersections remain.","reason":"Unable to Tessellate shape [[386.41562, 565.6094] [513.0249, 538.96094] [619.57166, 520.6353] [767.19977, 506.289] [907.88605, 497.9589] [1099.2129, 498.9013] [1195.2057, 507.0456] [1203.4454, 505.5837] [1203.573, 501.9473] [1203.5731, 501.94464] [1190.4535, 500.9026] [1190.2885, 504.78632] [1139.4575, 500.21658] [1139.3555, 494.29102] [1124.5372, 493.16135] [1124.67, 496.8161] [1121.9475, 496.69806] [1122.1187, 492.65085] [1101.5104, 491.71823] [1101.7128, 495.61987] [1098.6561, 495.61987] [1098.7572, 491.29382] [974.66943, 489.19507] [974.6986, 490.45502] [971.79803, 490.76035] [971.7731, 488.87952] [921.2863, 489.2724] [921.4196, 493.3556] [918.3664, 493.3556] [918.1993, 489.29642] [903.6128, 489.40994] [903.54694, 490.06866] [885.6189, 489.96793] [885.6299, 489.5499] [746.0154, 496.31168] [745.86743, 498.2444] [746.6732, 506.30194] [726.32794, 508.11487] [725.67017, 500.22217] [706.9845, 501.5434] [707.3927, 506.30194] [704.774, 506.50336] [704.41, 501.72543] [608.293, 512.5985] [520.063, 528.71356] [500.45, 531.9685] [501.85992, 540.11456] [499.56998, 540.5725] [498.18433, 532.60504] [463.6745, 542.30023] [463.94565, 543.78156] [459.16125, 545.64215] [458.83234, 543.6606] [405.88757, 558.5349] [388.8988, 564.70245] [388.53156, 564.83575] [386.37985, 565.61694] [386.37982, 565.61694] [386.37982, 565.61694] [386.38135, 565.61664] [386.38135, 565.61664] [386.39346, 565.6141] [386.41562, 565.6094] ]. Possible malformed shape detected.","type":"illegal_argument_exception"}', 'type': 'ApiException'}

Intersections introduced by rounding:

[388.53156, 564.83575],[386.37985, 565.61694]
[386.37982, 565.61694],[386.38135, 565.61664]

while original values of coordinates of those segments are:

[388.531563, 564.835771], [386.37985, 565.616922]
[386.37985, 565.616922], [386.381351, 565.616614]

[elasticmachine]

Pinging @elastic/es-analytics-geo (Team:Analytics)

[badjr]

@filipski can you include the command you used to index the polygon?

[filipski]

The call is made with help of:
org.elasticsearch.client.RestHighLevelClient.update(request, RequestOptions.DEFAULT)
Where the request is the class:
org.elasticsearch.action.update.UpdateRequest()
The values sent to elastic is encoded to bytes with help of:
com.fasterxml.jackson.databind.ObjectMapper.writeValueAsBytes(entity)
Where the argument "entity" is a class that represent the object we want to save in the es-database, it contains (among other data) a map of coordinates made with help of the Java Point-class with Doubles as values representing the polygon.

[filipski]

Is that enough to get you going?

[imotov]

Is that enough to get you going?

Yes, it reproduces in master:

DELETE test-index

PUT test-index
{
  "mappings": {
    "properties": {
      "geometry": {
        "type": "shape"
      }
    }
  }
}

PUT test-index/_doc/1
{
  "geometry": {
    "coordinates": [
      [
        [
          386.415622,
          565.6094
        ],
        [
          513.024902,
          538.960938
        ],
        [
          619.571655,
          520.635315
        ],
        [
          767.199768,
          506.289001
        ],
        [
          907.886047,
          497.958893
        ],
        [
          1099.212891,
          498.901306
        ],
        [
          1195.205688,
          507.045593
        ],
        [
          1203.445435,
          505.58371
        ],
        [
          1203.573026,
          501.947307
        ],
        [
          1203.57312,
          501.944641
        ],
        [
          1190.453547,
          500.902579
        ],
        [
          1190.288452,
          504.786316
        ],
        [
          1139.45752,
          500.216583
        ],
        [
          1139.355469,
          494.291016
        ],
        [
          1124.537231,
          493.161346
        ],
        [
          1124.670044,
          496.816101
        ],
        [
          1121.94751,
          496.698059
        ],
        [
          1122.118594,
          492.650836
        ],
        [
          1101.510376,
          491.718231
        ],
        [
          1101.712769,
          495.619873
        ],
        [
          1098.656128,
          495.619873
        ],
        [
          1098.757252,
          491.293826
        ],
        [
          974.669443,
          489.195082
        ],
        [
          974.698608,
          490.455017
        ],
        [
          971.798035,
          490.760345
        ],
        [
          971.773067,
          488.879509
        ],
        [
          921.286339,
          489.272402
        ],
        [
          921.419617,
          493.355591
        ],
        [
          918.366394,
          493.355591
        ],
        [
          918.199252,
          489.296426
        ],
        [
          903.612801,
          489.409939
        ],
        [
          903.546936,
          490.068665
        ],
        [
          885.618896,
          489.967926
        ],
        [
          885.629896,
          489.549884
        ],
        [
          746.015381,
          496.311676
        ],
        [
          745.867432,
          498.244385
        ],
        [
          746.673218,
          506.301941
        ],
        [
          726.327942,
          508.114868
        ],
        [
          725.670191,
          500.222176
        ],
        [
          706.98449,
          501.543385
        ],
        [
          707.3927,
          506.301941
        ],
        [
          704.773987,
          506.503357
        ],
        [
          704.409956,
          501.725423
        ],
        [
          608.29303,
          512.598511
        ],
        [
          520.062988,
          528.713562
        ],
        [
          500.450024,
          531.96851
        ],
        [
          501.859924,
          540.114563
        ],
        [
          499.569977,
          540.57251
        ],
        [
          498.184334,
          532.605034
        ],
        [
          463.674513,
          542.300228
        ],
        [
          463.945648,
          543.781555
        ],
        [
          459.161255,
          545.642151
        ],
        [
          458.832333,
          543.660591
        ],
        [
          405.887587,
          558.534896
        ],
        [
          388.898793,
          564.702453
        ],
        [
          388.531563,
          564.835771
        ],
        [
          386.37985,
          565.616922
        ],
        [
          386.379814,
          565.616935
        ],
        [
          386.379814,
          565.616937
        ],
        [
          386.381351,
          565.616614
        ],
        [
          386.381356,
          565.616612
        ],
        [
          386.393477,
          565.614061
        ],
        [
          386.415622,
          565.6094
        ]
      ]
    ],
    "type": "Polygon"
  }
}

the error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "failed to parse field [geometry] of type [shape]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "failed to parse field [geometry] of type [shape]",
    "caused_by" : {
      "type" : "illegal_argument_exception",
      "reason" : "Unable to Tessellate shape [[386.41562, 565.6094] [513.0249, 538.96094] [619.57166, 520.6353] [767.19977, 506.289] [907.88605, 497.9589] [1099.2129, 498.9013] [1195.2057, 507.0456] [1203.4454, 505.5837] [1203.573, 501.9473] [1203.5731, 501.94464] [1190.4535, 500.9026] [1190.2885, 504.78632] [1139.4575, 500.21658] [1139.3555, 494.29102] [1124.5372, 493.16135] [1124.67, 496.8161] [1121.9475, 496.69806] [1122.1187, 492.65085] [1101.5104, 491.71823] [1101.7128, 495.61987] [1098.6561, 495.61987] [1098.7572, 491.29382] [974.66943, 489.19507] [974.6986, 490.45502] [971.79803, 490.76035] [971.7731, 488.87952] [921.2863, 489.2724] [921.4196, 493.3556] [918.3664, 493.3556] [918.1993, 489.29642] [903.6128, 489.40994] [903.54694, 490.06866] [885.6189, 489.96793] [885.6299, 489.5499] [746.0154, 496.31168] [745.86743, 498.2444] [746.6732, 506.30194] [726.32794, 508.11487] [725.67017, 500.22217] [706.9845, 501.5434] [707.3927, 506.30194] [704.774, 506.50336] [704.41, 501.72543] [608.293, 512.5985] [520.063, 528.71356] [500.45, 531.9685] [501.85992, 540.11456] [499.56998, 540.5725] [498.18433, 532.60504] [463.6745, 542.30023] [463.94565, 543.78156] [459.16125, 545.64215] [458.83234, 543.6606] [405.88757, 558.5349] [388.8988, 564.70245] [388.53156, 564.83575] [386.37985, 565.61694] [386.37982, 565.61694] [386.37982, 565.61694] [386.38135, 565.61664] [386.38135, 565.61664] [386.39346, 565.6141] [386.41562, 565.6094] ]. Possible malformed shape detected."
    }
  },
  "status" : 400
}

[iverase]

Just to set the right expectations, this error is expected as the provided polygon self-intersects on the float space and this index works on that space.

When I asked you to open an issue was not for making this polygon indexable but for improving the error message which is not very helpful at the moment.

[iverase]

The error message should be more clear now and there is nothing to here as shape type works on the float space.