Should reindex-from-remote allow indexing lots of indices into local indexes with the same name? #22920
Comments
|
Personally I wouldn't want to use this feature. If I had many indices that I wanted to reindex-from-remote I'd write a bash script to reindex them one by one. That way I could see progress and parallelize them or whatever. I've been known to write some pretty fancy (still horrible) bash scripts for this kind of thing.... |
|
I like the ease of use of reindexing all indices to a new cluster, but I think i'm persuaded by your argument:
|
|
Talked with a small group - I'll update the docs with the bash script I'd write. |
|
@nik9000 Did the bash script end up in the docs? The use case we just had for this is migrating a local Elasticsearch cluster with daily indices to Elastic Cloud. I personally think there would be value in making this use case as simple as possible and not require an external script to work. What is the reason the script works with a new index but fails with the exact same one? |
No. I forgot about this issue. The nice thing about an external script is that it can be made to be fairly pause-able and resume-able, both of which are harder with reindex. If you want to dump many external indices into the same target index that'll work. But reindex doesn't have a "same_as_source" option. You can use a script to get it, but I figure if you are loading many indices you are better of doing it with something you have more control over. |
lcawl
added
:Distributed Indexing/CRUD
|
@nik9000 are you still planning to do this? if not, it's fine. Please close the issue. |
Folks tend to want to be able to make a single `_reindex` call to migrate many indices. You *can* do that and we even have an example of how to do that in the docs but it isn't always a good idea. This change adds some advice to the docs: generally you want to make one reindex call per index. Closes elastic#22920
Folks tend to want to be able to make a single `_reindex` call to migrate many indices. You *can* do that and we even have an example of how to do that in the docs but it isn't always a good idea. This change adds some advice to the docs: generally you want to make one reindex call per index. Closes #22920
Folks tend to want to be able to make a single `_reindex` call to migrate many indices. You *can* do that and we even have an example of how to do that in the docs but it isn't always a good idea. This change adds some advice to the docs: generally you want to make one reindex call per index. Closes #22920
Folks tend to want to be able to make a single `_reindex` call to migrate many indices. You *can* do that and we even have an example of how to do that in the docs but it isn't always a good idea. This change adds some advice to the docs: generally you want to make one reindex call per index. Closes #22920
Right now reindex-from-remote only indexes into a single destination index by default. You can use this technique to make it write into indices with a different name, but you can't get it to write into exactly the same index name. The script doesn't support that. Should we do something to make reindex-from-remote support writing to the same target indexes?
The text was updated successfully, but these errors were encountered: