Skip to content

id slicer will not work with elasticsearch 6 #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jsnoble opened this issue Sep 20, 2018 · 1 comment
Closed

id slicer will not work with elasticsearch 6 #12

jsnoble opened this issue Sep 20, 2018 · 1 comment

Comments

@jsnoble
Copy link
Member

jsnoble commented Sep 20, 2018

{jsonble}
They are depreciating how _uid operates in this version, and it starting to use a _id search instead which only allows for simple queries like term searches. This will also affect our elasticsearch_reader if we use the capability to further split a time segment further down.

docs:
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/mapping-uid-field.html
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/mapping-id-field.html

for that last one pay attention to the note posted towards the top of the page and the paragraph just past it.

{kstaken}
This appears to be the root of the problem: elastic/elasticsearch#18154

And where they decided to not worry about what this breaks: elastic/elasticsearch#24247

Just so that we maintain an option for working in this regard we may need to look at sliced scroll queries and evaluate just how much risk there is from the state held server side on active indices. https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html

Also if this were applied as a subslicing technique for further division of an indivisible date range query, the impact of the state held server side would be relatively small and not held for the entire duration of a reader operation. It would only have to be held for the duration of the subslicing.

In cases where an index is static using the scroll API should be perfectly fine so it may still give us a reasonable set of options for when range slicing doesn't work.

@jsnoble
Copy link
Member Author

jsnoble commented Jan 10, 2020

this has been reworked to slice on a field, not the _id anymore

@jsnoble jsnoble closed this as completed Jan 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant