Skip to content

Docs: HLRC: refactor bulk, migrate and reindex apis #35413

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 16, 2018
Merged

Docs: HLRC: refactor bulk, migrate and reindex apis #35413

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3fbfbc9
refactor bulk and reindex apis
pgomulka Nov 8, 2018
ae42d3e
migration API
pgomulka Nov 14, 2018
2632fbf
fix missing file
pgomulka Nov 15, 2018
d77d598
break lines
pgomulka Nov 15, 2018
210aa98
Merge branch 'master' into docs/improve-migration-bulk-index
pgomulka Nov 15, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions client/rest-high-level/src/test/java/org/elasticsearch/client/ReindexIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,15 +106,19 @@ public void testReindexTask() throws IOException, InterruptedException {
);
}
{
ReindexRequest reindexRequest = new ReindexRequest();
// tag::submit-reindex-task
ReindexRequest reindexRequest = new ReindexRequest(); // <1>
reindexRequest.setSourceIndices(sourceIndex);
reindexRequest.setDestIndex(destinationIndex);
reindexRequest.setSourceQuery(new IdsQueryBuilder().addIds("1").types("type"));
reindexRequest.setRefresh(true);

TaskSubmissionResponse reindexSubmission = highLevelClient().submitReindexTask(reindexRequest, RequestOptions.DEFAULT);
TaskSubmissionResponse reindexSubmission = highLevelClient()
.submitReindexTask(reindexRequest, RequestOptions.DEFAULT); // <2>

String taskId = reindexSubmission.getTask(); // <3>
// end::submit-reindex-task

BooleanSupplier hasUpgradeCompleted = checkCompletionStatus(reindexSubmission.getTask());
BooleanSupplier hasUpgradeCompleted = checkCompletionStatus(taskId);
awaitBusy(hasUpgradeCompleted);
}
}
Expand Down
14 changes: 7 additions & 7 deletions docs/java-rest/high-level/document/bulk.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,9 @@ And different operation types can be added to the same +{request}+:
--------------------------------------------------
include-tagged::{doc-tests-file}[{api}-request-with-mixed-operations]
--------------------------------------------------
<1> Adds a `DeleteRequest` to the `BulkRequest`. See <<{upid}-delete>>
<1> Adds a `DeleteRequest` to the +{request}+. See <<{upid}-delete>>
for more information on how to build `DeleteRequest`.
<2> Adds an `UpdateRequest` to the `BulkRequest`. See <<{upid}-update>>
<2> Adds an `UpdateRequest` to the +{request}+. See <<{upid}-update>>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

for more information on how to build `UpdateRequest`.
<3> Adds an `IndexRequest` using the SMILE format

Expand Down Expand Up @@ -72,22 +72,22 @@ the index/update/delete operations.

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/CRUDDocumentationIT.java[bulk-request-pipeline]
include-tagged::{doc-tests-file}[{api}-request-pipeline]
--------------------------------------------------
<1> Global pipelineId used on all sub requests, unless overridden on a sub request

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/CRUDDocumentationIT.java[bulk-request-routing]
include-tagged::{doc-tests-file}[{api}-request-routing]
--------------------------------------------------
<1> Global routingId used on all sub requests, unless overridden on a sub request

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/CRUDDocumentationIT.java[bulk-request-index-type]
include-tagged::{doc-tests-file}[{api}-request-index-type]
--------------------------------------------------
<1> A bulk request with global index and type used on all sub requests, unless overridden on a sub request.
Both parameters are @Nullable and can only be set during BulkRequest creation.
Both parameters are @Nullable and can only be set during +{request}+ creation.

include::../execution.asciidoc[]

Expand Down Expand Up @@ -167,7 +167,7 @@ actions currently added (defaults to 1000, use -1 to disable it)
actions currently added (defaults to 5Mb, use -1 to disable it)
<3> Set the number of concurrent requests allowed to be executed
(default to 1, use 0 to only allow the execution of a single request)
<4> Set a flush interval flushing any `BulkRequest` pending if the
<4> Set a flush interval flushing any +{request}+ pending if the
interval passes (defaults to not set)
<5> Set a constant back off policy that initially waits for 1 second
and retries up to 3 times. See `BackoffPolicy.noBackoff()`,
Expand Down
17 changes: 15 additions & 2 deletions docs/java-rest/high-level/document/reindex.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
[id="{upid}-{api}-request"]
==== Reindex Request

A +{request} can be used to copy documents from one or more indexes into a
A +{request}+ can be used to copy documents from one or more indexes into a
destination index.

It requires an existing source index and a target index which may or may not exist pre-request. Reindex does not attempt
Expand Down Expand Up @@ -100,7 +100,7 @@ include-tagged::{doc-tests-file}[{api}-request-sort]
<1> add descending sort to`field1`
<2> add ascending sort to `field2`

+{request} also supports a `script` that modifies the document. It allows you to
+{request}+ also supports a `script` that modifies the document. It allows you to
also change the document's metadata. The following example illustrates that.

["source","java",subs="attributes,callouts,macros"]
Expand Down Expand Up @@ -157,6 +157,19 @@ include-tagged::{doc-tests-file}[{api}-request-refresh]

include::../execution.asciidoc[]

[id="{upid}-{api}-task-submission"]
==== Reindex task submission
It is also possible to submit a +{request}+ and not wait for it completion with the use of Task API. This is an equivalent of a REST request
with wait_for_completion flag set to false.

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{hlrc-tests}/ReindexIT.java[submit-reindex-task]
--------------------------------------------------
<1> A +{request}+ is constructed the same way as for the synchronous method
<2> A submit method returns a `TaskSubmissionResponse` which contains a task identifier.
<3> The task identifier can be used to get `response` from a completed task.

[id="{upid}-{api}-response"]
==== Reindex Response

Expand Down
1 change: 1 addition & 0 deletions docs/java-rest/high-level/index.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ the same response objects.
--

:doc-tests: {docdir}/../../client/rest-high-level/src/test/java/org/elasticsearch/client/documentation
:hlrc-tests: {docdir}/../../client/rest-high-level/src/test/java/org/elasticsearch/client

include::getting-started.asciidoc[]
include::supported-apis.asciidoc[]
Expand Down
34 changes: 20 additions & 14 deletions docs/java-rest/high-level/migration/upgrade.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,22 @@
--
:api: upgrade
:request: IndexUpgradeRequest
:response: BulkByScrollResponse
:submit_response: IndexUpgradeSubmissionResponse
:doc-tests-file: {doc-tests}/MigrationClientDocumentationIT.java
--

[[java-rest-high-migration-upgrade]]
=== Migration Upgrade

[[java-rest-high-migraton-upgrade-request]]
==== Index Upgrade Request

An `IndexUpgradeRequest` requires an index argument. Only one index at the time should be upgraded:
An +{request}+ requires an index argument. Only one index at the time should be upgraded:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/MigrationClientDocumentationIT.java[upgrade-request]
include-tagged::{doc-tests-file}[{api}-request]
--------------------------------------------------
<1> Create a new request instance

Expand All @@ -17,39 +25,37 @@ include-tagged::{doc-tests}/MigrationClientDocumentationIT.java[upgrade-request]

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/MigrationClientDocumentationIT.java[upgrade-execute]
include-tagged::{doc-tests-file}[{api}-execute]
--------------------------------------------------

[[java-rest-high-migration-upgrade-response]]
==== Response
The returned `BulkByScrollResponse` contains information about the executed operation
The returned +{response}+ contains information about the executed operation


[[java-rest-high-migraton-async-upgrade-request]]
==== Asynchronous Execution

The asynchronous execution of a upgrade request requires both the `IndexUpgradeRequest`
The asynchronous execution of an upgrade request requires both the +{request}+
instance and an `ActionListener` instance to be passed to the asynchronous
method:

A typical listener for `BulkResponse` looks like:

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/MigrationClientDocumentationIT.java[upgrade-async-listener]
include-tagged::{doc-tests-file}[{api}-async-listener]
--------------------------------------------------
<1> Called when the execution is successfully completed. The response is
provided as an argument and contains a list of individual results for each
operation that was executed. Note that one or more operations might have
failed while the others have been successfully executed.
<2> Called when the whole `IndexUpgradeRequest` fails. In this case the raised
<2> Called when the whole +{request}+ fails. In this case the raised
exception is provided as an argument and no operation has been executed.

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/MigrationClientDocumentationIT.java[upgrade-async-execute]
include-tagged::{doc-tests-file}[{api}-async-execute]
--------------------------------------------------
<1> The `IndexUpgradeRequest` to execute and the `ActionListener` to use when
<1> The +{request}+ to execute and the `ActionListener` to use when
the execution completes

The asynchronous method does not block and returns immediately. Once it is
Expand All @@ -59,11 +65,11 @@ it failed.


=== Migration Upgrade with Task API
Submission of upgrade request task will requires the `IndexUpgradeRequest` and will return
`IndexUpgradeSubmissionResponse`. The `IndexUpgradeSubmissionResponse` can later be use to fetch
Submission of upgrade request task will requires the +{request}+ and will return
+{submit_response}+. The +{submit_response}+ can later be use to fetch
TaskId and query the Task API for results.

["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/MigrationClientDocumentationIT.java[upgrade-task-api]
include-tagged::{doc-tests-file}[{api}-task-api]
--------------------------------------------------