Skip to content

Commit b769f76

Browse files
authored
Update rollover index API docs for data streams (#55551)
1 parent 83f0ee2 commit b769f76

File tree

1 file changed

+128
-48
lines changed

1 file changed

+128
-48
lines changed

docs/reference/indices/rollover-index.asciidoc

Lines changed: 128 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,12 @@
44
<titleabbrev>Rollover index</titleabbrev>
55
++++
66

7-
Assigns an <<indices-aliases, index alias>> to a new index
8-
when the alias's existing index meets a condition you provide.
7+
Creates a new index for a rollover target when the target's existing index meets
8+
a condition you provide. A rollover target can be either an
9+
<<indices-aliases, index alias>> or a
10+
<<indices-create-data-stream, data stream>>. When targeting an alias, the alias
11+
is updated to point to the new index. When targeting a data stream, the new
12+
index becomes the data stream's write index and its generation is incremented.
913

1014
[source,console]
1115
----
@@ -25,42 +29,47 @@ POST /alias1/_rollover/twitter
2529
==== {api-request-title}
2630

2731

28-
`POST /<alias>/_rollover/<target-index>`
32+
`POST /<rollover-target>/_rollover/<target-index>`
2933

30-
`POST /<alias>/_rollover/`
34+
`POST /<rollover-target>/_rollover/`
3135

3236

3337
[[rollover-index-api-desc]]
3438
==== {api-description-title}
3539

36-
The rollover index API rolls an <<indices-aliases, alias>> to a new index when
37-
the existing index meets a condition you provide. You can use this API to retire
38-
an index that becomes too large or too old.
40+
The rollover index API rolls a rollover target to a new index when the existing
41+
index meets a condition you provide. You can use this API to retire an index
42+
that becomes too large or too old.
3943

4044
NOTE: To roll over an index, a condition must be met *when you call the API*.
4145
{es} does not monitor the index after you receive an API response. To
4246
automatically roll over indices when a condition is met, you can use {es}'s
4347
<<index-lifecycle-management, index lifecycle management (ILM) policies>>.
4448

45-
The rollover index API accepts a single alias name
49+
The rollover index API accepts a rollover target name
4650
and a list of `conditions`.
4751

48-
If the specified alias points to a single index,
52+
If the specified rollover target is an alias pointing to a single index,
4953
the rollover request:
5054

5155
. Creates a new index
5256
. Adds the alias to the new index
5357
. Removes the alias from the original index
5458

55-
If the specified alias points to multiple indices,
59+
If the specified rollover target is an alias pointing to multiple indices,
5660
one of these indices must have `is_write_index` set to `true`.
57-
In this case,
58-
the rollover request:
61+
In this case, the rollover request:
5962

6063
. Creates a new index
6164
. Sets `is_write_index` to `true` for the new index
6265
. Sets `is_write_index` to `false` for the original index
6366

67+
If the specified rollover target is a data stream, the rollover request:
68+
69+
. Creates a new index
70+
. Adds the new index as a backing index and the write index on the data stream
71+
. Increments the `generation` attribute of the data stream
72+
6473

6574
[[rollover-wait-active-shards]]
6675
===== Wait for active shards
@@ -73,10 +82,10 @@ index creation applies to the rollover action.
7382
[[rollover-index-api-path-params]]
7483
==== {api-path-parms-title}
7584

76-
`<alias>`::
77-
(Required, string)
78-
Name of the existing index alias
79-
to assign to the target index.
85+
`<rollover-target>`::
86+
(Required*, string)
87+
Name of the existing index alias or data stream on which to
88+
perform the rollover.
8089

8190

8291
`<target-index>`::
@@ -87,19 +96,18 @@ Name of the target index to create and assign the index alias.
8796

8897
include::{docdir}/indices/create-index.asciidoc[tag=index-name-reqs]
8998

90-
*This parameter is not required
91-
if the alias is assigned to an index name that ends with `-` and a number,
92-
such as `logs-000001`.
93-
In this case,
94-
the name of the new index follows the same pattern,
95-
incrementing the number.
96-
For example,
97-
`logs-000001` increments to `logs-000002`.
98-
This number is zero-padded with a length of 6,
99-
regardless of the prior index name.
100-
101-
If the existing index for the alias does not match this pattern,
102-
this parameter is required.
99+
*This parameter is not permitted if `rollover-target` is a data stream. In
100+
that case, the new index name will be in the form `<rollover-target>-000001`
101+
where the zero-padded number of length 6 is the generation of the data stream.
102+
103+
If `rollover-target` is an alias that is assigned to an index name that ends
104+
with `-` and a number such as `logs-000001`. In this case, the name of the new
105+
index follows the same pattern and increments the number. For example,
106+
`logs-000001` increments to `logs-000002`. This number is zero-padded with a
107+
length of 6, regardless of the prior index name.
108+
109+
If the existing index for the alias does not match this pattern, this parameter
110+
is required.
103111
--
104112

105113

@@ -126,8 +134,9 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=aliases]
126134
`conditions`::
127135
+
128136
--
129-
(Required, object)
130-
Set of conditions the index alias's existing index must met to roll over.
137+
(Optional, object)
138+
If supplied, the set of conditions the rollover target's existing index must
139+
meet to roll over. If omitted, the rollover will be performed unconditionally.
131140

132141
Parameters include:
133142

@@ -143,12 +152,12 @@ The document count does *not* include documents in replica shards.
143152

144153
`max_size`::
145154
(Optional, <<byte-units, byte units>>)
146-
Maximum index size.
147-
This is the total size of all primary shards in the index.
155+
Maximum index size.
156+
This is the total size of all primary shards in the index.
148157
Replicas are not counted toward the maximum index size.
149158

150-
TIP: To see the current index size, use the <<cat-indices, _cat indices>> API.
151-
The `pri.store.size` value shows the combined size of all primary shards.
159+
TIP: To see the current index size, use the <<cat-indices, _cat indices>> API.
160+
The `pri.store.size` value shows the combined size of all primary shards.
152161
--
153162

154163
include::{docdir}/rest-api/common-parms.asciidoc[tag=mappings]
@@ -212,13 +221,82 @@ The API returns the following response:
212221
<2> Whether the rollover was dry run.
213222
<3> The result of each condition.
214223

224+
225+
[[rollover-data-stream-ex]]
226+
===== Roll over a data stream
227+
228+
[source,console]
229+
--------------------------------------------------
230+
PUT /_data_stream/my-data-stream <1>
231+
{
232+
"timestamp_field": "date"
233+
}
234+
235+
# Add > 1000 documents to my-data-stream
236+
237+
POST /my-data-stream/_rollover <2>
238+
{
239+
"conditions" : {
240+
"max_age": "7d",
241+
"max_docs": 1000,
242+
"max_size": "5gb"
243+
}
244+
}
245+
--------------------------------------------------
246+
// TEST[setup:huge_twitter]
247+
// TEST[s/# Add > 1000 documents to my-data-stream/POST _reindex?refresh\n{"source":{"index":"twitter"},"dest":{"index":"my-data-stream-000001"}}/]
248+
<1> Creates a data stream called `my-data-stream` with one initial backing index
249+
named `my-data-stream-000001`.
250+
<2> This request creates a new backing index, `my-data-stream-000002`, and adds
251+
it as the write index for the `my-data-stream` data stream if the current
252+
write index meets at least one of the following conditions:
253+
+
254+
--
255+
* The index was created 7 or more days ago.
256+
* The index has an index size of 5GB or greater.
257+
* The index contains 1,000 or more documents.
258+
--
259+
260+
The API returns the following response:
261+
262+
[source,console-result]
263+
--------------------------------------------------
264+
{
265+
"acknowledged": true,
266+
"shards_acknowledged": true,
267+
"old_index": "my-data-stream-000001", <1>
268+
"new_index": "my-data-stream-000002", <2>
269+
"rolled_over": true, <3>
270+
"dry_run": false, <4>
271+
"conditions": { <5>
272+
"[max_age: 7d]": false,
273+
"[max_docs: 1000]": true,
274+
"[max_size: 5gb]": false,
275+
}
276+
}
277+
--------------------------------------------------
278+
279+
<1> The previous write index for the data stream.
280+
<2> The new write index for the data stream.
281+
<3> Whether the index was rolled over.
282+
<4> Whether the rollover was dry run.
283+
<5> The result of each condition.
284+
285+
////
286+
[source,console]
287+
-----------------------------------
288+
DELETE /_data_stream/my-data-stream
289+
-----------------------------------
290+
// TEST[continued]
291+
////
292+
215293
[[rollover-index-settings-ex]]
216294
===== Specify settings for the target index
217295

218296
The settings, mappings, and aliases for the new index are taken from any
219-
matching <<indices-templates,index templates>>. Additionally, you can specify
220-
`settings`, `mappings`, and `aliases` in the body of the request, just like the
221-
<<indices-create-index,create index>> API. Values specified in the request
297+
matching <<indices-templates,index templates>>. If the rollover target is an index
298+
alias, you can specify `settings`, `mappings`, and `aliases` in the body of the request
299+
just like the <<indices-create-index,create index>> API. Values specified in the request
222300
override any values set in matching index templates. For example, the following
223301
`rollover` request overrides the `index.number_of_shards` setting:
224302

@@ -247,11 +325,14 @@ POST /logs_write/_rollover
247325

248326
[[rollover-index-specify-index-ex]]
249327
===== Specify a target index name
328+
If the rollover target is a data stream, you cannot specify a target index
329+
name. The new index name will always be in the form `<rollover-target>-000001`
330+
where the zero-padded number of length 6 is the generation of the data stream.
250331

251-
If the name of the existing index ends with `-` and a number -- e.g.
252-
`logs-000001` -- then the name of the new index will follow the same pattern,
253-
incrementing the number (`logs-000002`). The number is zero-padded with a length
254-
of 6, regardless of the old index name.
332+
If the rollover target is an index alias and the name of the existing index ends
333+
with `-` and a number -- e.g. `logs-000001` -- then the name of the new index will
334+
follow the same pattern, incrementing the number (`logs-000002`). The number is
335+
zero-padded with a length of 6, regardless of the old index name.
255336

256337
If the old name doesn't match this pattern then you must specify the name for
257338
the new index as follows:
@@ -273,12 +354,11 @@ POST /my_alias/_rollover/my_new_index_name
273354
[[_using_date_math_with_the_rollover_api]]
274355
===== Use date math with a rollover
275356

276-
It can be useful to use <<date-math-index-names,date math>> to name the
277-
rollover index according to the date that the index rolled over, e.g.
357+
If the rollover target is an index alias, it can be useful to use <<date-math-index-names,date math>>
358+
to name the rollover index according to the date that the index rolled over, e.g.
278359
`logstash-2016.02.03`. The rollover API supports date math, but requires the
279-
index name to end with a dash followed by a number, e.g.
280-
`logstash-2016.02.03-1` which is incremented every time the index is rolled
281-
over. For instance:
360+
index name to end with a dash followed by a number, e.g. `logstash-2016.02.03-1`
361+
which is incremented every time the index is rolled over. For instance:
282362

283363
[source,console]
284364
--------------------------------------------------
@@ -371,7 +451,7 @@ POST /logs_write/_rollover?dry_run
371451
[[indices-rollover-is-write-index]]
372452
===== Roll over a write index
373453

374-
The rollover alias when rolling over a write index that has `is_write_index` explicitly set to `true` is not
454+
If the rollover target is an index alias for a write index that has `is_write_index` explicitly set to `true`, it is not
375455
swapped during rollover actions. Since having an alias point to multiple indices is ambiguous in distinguishing
376456
which is the correct write index to roll over, it is not valid to rollover an alias that points to multiple indices.
377457
For this reason, the default behavior is to swap which index is being pointed to by the write-oriented alias. This

0 commit comments

Comments
 (0)