-
Notifications
You must be signed in to change notification settings - Fork 25.2k
Reindex source types disregarded in 7.x #49580
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reindex source types disregarded in 7.x #49580
Conversation
Clarify that types in source index are disregarded. Closes elastic#48460
Pinging @elastic/es-distributed (:Distributed/Reindex) |
Pinging @elastic/es-docs (>docs) |
docs/reference/docs/reindex.asciidoc
Outdated
NOTE: Types in source indices are always ignored, also when not specifying a | ||
destination `type`. If explicitly specifying destination `type`, the specified | ||
type must match the type in the destination index or be either unspecified or | ||
the special value `_doc`. See <<removal-of-types>> for further details. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I agree to 1, moving it to a definition of the type
parameter makes sense.
About 2, while they might be trying to preserve their types, they were in reality just expecting an error in this case and I wanted to clarify that the source type is disregarded completely now. In older versions, ES would copy over the source type to the destination too, but now that dest have only one type (which in 7.x can still be freely specified) copying over the type makes little to no sense. The source could in principle still have multiple types if specifying multiple indices with different types.
I commented on this inline too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this PR @henningandersen.
I think the first commit was a bit unclear so I pushed f669009 and a8087be to try to address.
Feel free to revert or adjust those as you see fit.
user feedback: Thanks for this. This could have helped us. |
docs/reference/docs/reindex.asciidoc
Outdated
@@ -484,6 +484,18 @@ Use in conjunction with `max_docs` to control what documents are reindexed. | |||
Set to a list to reindex select fields. | |||
Defaults to `true`. | |||
|
|||
`type`::: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should leave out this part of the change, since it solely concerns the source type and this is just a filter on the source. I think adding it here increases the risk of it being used, so would kind of prefer not to add this now.
It is also unrelated to the issue, since regardless of specifying this filter, the type from source is disregarded when indexing into dest.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm okay with removing this. As you mentioned, this isn't the main problem.
docs/reference/docs/reindex.asciidoc
Outdated
+ | ||
[WARNING] | ||
==== | ||
By default, all reindexed documents are assigned the `_doc` document type. To |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The original problem here really was that the user expected an error. I think that is somewhat lost in this new version and I prefer the original text (here) instead. WDYT?
By default, all reindexed documents are assigned the `_doc` document type. To | |
Types in source indices are always ignored, also when not specifying a | |
destination `type`. If explicitly specifying destination `type`, the specified | |
type must match the type in the destination index or be either unspecified or | |
the special value `_doc`. See <<removal-of-types>> for further details. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 That works for me.
docs/reference/docs/reindex.asciidoc
Outdated
NOTE: Types in source indices are always ignored, also when not specifying a | ||
destination `type`. If explicitly specifying destination `type`, the specified | ||
type must match the type in the destination index or be either unspecified or | ||
the special value `_doc`. See <<removal-of-types>> for further details. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I agree to 1, moving it to a definition of the type
parameter makes sense.
About 2, while they might be trying to preserve their types, they were in reality just expecting an error in this case and I wanted to clarify that the source type is disregarded completely now. In older versions, ES would copy over the source type to the destination too, but now that dest have only one type (which in 7.x can still be freely specified) copying over the type makes little to no sense. The source could in principle still have multiple types if specifying multiple indices with different types.
I commented on this inline too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for looking into this @jrodewig , I have a few comments to your changes. I do not mind carrying out the final changes, but thought we could agree on them here first.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @henningandersen. I agree with your suggestions. Thanks for seeing this one through.
Remove source type description and clarify that source type is disregarded completely.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Thanks again for the iteration @henningandersen.
Thanks for reviewing @jrodewig |
Clarify that types in source index are disregarded.
Clarify that types in source index are disregarded.
Closes #48460