Skip to content

[7.x] [DOCS] Add data streams to bulk, delete, and index API docs (#58340) #58434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 23, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 18 additions & 9 deletions docs/reference/docs/bulk.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ POST _bulk

`POST /_bulk`

`POST /<index>/_bulk`
`POST /<target>/_bulk`

[[docs-bulk-api-desc]]
==== {api-description-title}
Expand All @@ -47,8 +47,12 @@ optional_source\n

The `index` and `create` actions expect a source on the next line,
and have the same semantics as the `op_type` parameter in the standard index API:
create fails if a document with the same name already exists in the index,
index adds or replaces a document as necessary.
`create` fails if a document with the same ID already exists in the target,
`index` adds or replaces a document as necessary.

NOTE: <<data-streams,Data streams>> support only the `create` action. To update
or delete a document in a data stream, you must target the backing index
containing the document. See <<update-delete-docs-in-a-data-stream>>.

`update` expects that the partial doc, upsert,
and script and its options are specified on the next line.
Expand All @@ -67,8 +71,8 @@ the `Content-Type` header should be set to `application/x-ndjson`.
Because this format uses literal `\n`'s as delimiters,
make sure that the JSON actions and sources are not pretty printed.

If you specify an index in the request URI,
it is used for any actions that don't explicitly specify an index.
If you provide a `<target>` in the request path,
it is used for any actions that don't explicitly specify an `_index` argument.

A note on the format: The idea here is to make processing of this as
fast as possible. As some of the actions are redirected to other
Expand Down Expand Up @@ -192,8 +196,10 @@ See <<url-access-control>>.
[[docs-bulk-api-path-params]]
==== {api-path-parms-title}

`<index>`::
(Optional, string) Name of the index to perform the bulk actions against.
`<target>`::
(Optional, string)
Name of the data stream, index, or index alias to perform bulk actions
on.

[[docs-bulk-api-query-params]]
==== {api-query-parms-title}
Expand Down Expand Up @@ -321,7 +327,8 @@ operation.
=====
`_index`::
(string)
The index name or alias associated with the operation.
Name of the index associated with the operation. If the operation targeted a
data stream, this is the backing index into which the document was written.

`_type`::
(string)
Expand Down Expand Up @@ -415,7 +422,9 @@ ID of the shard associated with the failed operation.

`index`::
(string)
The index name or alias associated with the failed operation.
Name of the index associated with the failed operation. If the operation
targeted a data stream, this is the backing index into which the document was
attempted to be written.
======
=====
====
Expand Down
4 changes: 4 additions & 0 deletions docs/reference/docs/delete.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ Removes a JSON document from the specified index.
You use DELETE to remove a document from an index. You must specify the
index name and document ID.

NOTE: You cannot send deletion requests directly to a data stream. To delete a
document in a data stream, you must target the backing index containing the
document. See <<update-delete-docs-in-a-data-stream>>.

[float]
[[optimistic-concurrency-control-delete]]
===== Optimistic concurrency control
Expand Down
56 changes: 41 additions & 15 deletions docs/reference/docs/index_.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,27 +6,40 @@

IMPORTANT: See <<removal-of-types>>.

Adds a JSON document to the specified index and makes
it searchable. If the document already exists,
updates the document and increments its version.
Adds a JSON document to the specified data stream or index and makes
it searchable. If the target is an index and the document already exists,
the request updates the document and increments its version.

NOTE: You cannot send update requests for existing documents to a data stream.
To update a document in a data stream, you must target the backing index
containing the document. See <<update-delete-docs-in-a-data-stream>>.

[[docs-index-api-request]]
==== {api-request-title}

`PUT /<index>/_doc/<_id>`
`PUT /<target>/_doc/<_id>`

`POST /<index>/_doc/`
`POST /<target>/_doc/`

`PUT /<index>/_create/<_id>`
`PUT /<target>/_create/<_id>`

`POST /<index>/_create/<_id>`
`POST /<target>/_create/<_id>`

[[docs-index-api-path-params]]
==== {api-path-parms-title}

`<index>`::
(Required, string) Name of the target index. By default, the index is created
automatically if it doesn't exist. For more information, see <<index-creation>>.
`<target>`::
(Required, string) Name of the data stream or index to target.
+
If the target doesn't exist and matches the name or wildcard (`*`) pattern of a
<<create-a-data-stream-template,composable template with a `data_stream`
definition>>, this request creates the data stream. See
<<set-up-a-data-stream>>.
+
If the target doesn't exist and doesn't match a data stream template,
this request creates the index.
+
You can check for existing targets using the resolve index API.

`<_id>`::
(Optional, string) Unique identifier for the document. Required if you are
Expand All @@ -48,6 +61,9 @@ if it does not already exist (_put if absent_). If a document with the specified
`_id` already exists, the indexing operation will fail. Same as using the
`<index>/_create` endpoint. Valid values: `index`, `create`.
If document id is specified, it defaults to `index`. Otherwise, it defaults to `create`.
+
NOTE: If the request targets a data stream, an `op_type` of `create` is
required. See <<add-documents-to-a-data-stream>>.

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=pipeline]

Expand Down Expand Up @@ -125,12 +141,19 @@ You can index a new JSON document with the `_doc` or `_create` resource. Using
exist. To update an existing document, you must use the `_doc` resource.

[[index-creation]]
===== Create indices automatically
===== Automatically create data streams and indices

If request's target doesn't exist and matches a
<<create-a-data-stream-template,composable template with a `data_stream`
definition>>, the index operation automatically creates the data stream. See
<<set-up-a-data-stream>>.

If the specified index does not already exist, by default the index operation
automatically creates it and applies any configured
<<indices-templates,index templates>>. If no mapping exists, the index operation
creates a dynamic mapping. By default, new fields and objects are
If the target doesn't exist and doesn't match a data stream template,
the operation automatically creates the index and applies any matching
<<indices-templates,index templates>>.

If no mapping exists, the index operation
creates a dynamic mapping. By default, new fields and objects are
automatically added to the mapping if needed. For more information about field
mapping, see <<mapping,mapping>> and the <<indices-put-mapping,put mapping>> API.

Expand All @@ -142,6 +165,9 @@ automatic creation of indices that match specified patterns, or set it to
comma-separated list of patterns you want to allow, or prefix each pattern with
`+` or `-` to indicate whether it should be allowed or blocked. When a list is
specified, the default behaviour is to disallow.
+
IMPORTANT: The `action.auto_create_index` setting only affects the automatic
creation of indices. It does not affect the creation of data streams.

[source,console]
--------------------------------------------------
Expand Down
4 changes: 2 additions & 2 deletions docs/reference/rest-api/common-parms.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -452,8 +452,8 @@ end::index-total[]
tag::bulk-index[]
`_index`::
(Optional, string)
The name of the target index.
Required if not specified as a path parameter.
Name of the data stream, index, or index alias to perform the action on. This
parameter is required if a `<target>` is not specified in the request path.
end::bulk-index[]

tag::index-metric[]
Expand Down