Skip to content

range query on dates can see small numbers of millis as a year #63680

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nik9000 opened this issue Oct 14, 2020 · 4 comments · Fixed by #63692
Closed

range query on dates can see small numbers of millis as a year #63680

nik9000 opened this issue Oct 14, 2020 · 4 comments · Fixed by #63692
Assignees
Labels
>bug :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team

Comments

@nik9000
Copy link
Member

nik9000 commented Oct 14, 2020

If you write a date range query with numeric parameters we can interpret them as a year. Check it out:

curl -XDELETE -uelastic:password localhost:9200/test
curl -XPOST -HContent-Type:application/json -uelastic:password localhost:9200/test/_doc?refresh -d'{"d": "1970-01-01T00:00:01.010Z"}'
curl -XPOST -HContent-Type:application/json -uelastic:password localhost:9200/test/_doc?refresh -d'{"d": "1967-01-01T00:00:00.000Z"}'
curl -XPOST -HContent-Type:application/json -uelastic:password localhost:9200/test/_search?pretty -d'{
  "query": {
    "range": {
      "d": {
        "gt": 1000
      }
    }
  }
}'

The 1000 gets turned into a String internally and then we parse it as the year 1000.

@nik9000 nik9000 added >bug :Search/Search Search-related issues that do not fall into other categories needs:triage Requires assignment of a team area label labels Oct 14, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (:Search/Search)

@cbuescher
Copy link
Member

I think we should force parsing with "epoch_millis" when the argument of to or from is a number. Working on a fix.

@cbuescher cbuescher removed the needs:triage Requires assignment of a team area label label Oct 14, 2020
@nik9000
Copy link
Member Author

nik9000 commented Oct 14, 2020

I think maybe this an 8.0 breaking change kind of thing though.

@cbuescher
Copy link
Member

For the record: numeric values smaller than 1000 are currently interpreted as is by "strict_date_optional_time||epoch_millis" (meaning they stay as they are), four digit numbers like 1000, 1555, 1968 are converte to milliseconds since epoch (negative and positive after 1970) and five digit numbers are again interpreted as is (e.g. 10000 stays like that, meaning 10000ms since 1970). This is truly odd and we certainly improve the situation by always parse ints with "epoch_millis" here.

cbuescher pushed a commit to cbuescher/elasticsearch that referenced this issue Oct 14, 2020
Currently, if you write a date range query with numeric 'to' or 'from' bounds,
they can be interpreted as years if no format is provided. We use
"strict_date_optional_time||epoch_millis" in this case that can interpret inputs
like 1000 as the year 1000 for example. We should change this to always
interpret and parse numbers in this case with the second option "epoch_millis"
if no other formatter was provided.

Closes elastic#63680
cbuescher pushed a commit that referenced this issue Dec 1, 2020
Currently, if you write a date range query with numeric 'to' or 'from' bounds,
they can be interpreted as years if no format is provided. We use
"strict_date_optional_time||epoch_millis" in this case that can interpret inputs
like 1000 as the year 1000 for example. 
This PR change this to always interpret and parse numbers with the "epoch_millis"
parser if no other formatter was provided.

Closes #63680
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants