Skip to content

Commit 6a8e101

Browse files
committed
[DOCS] Add throttling based on configuration parameter (#56653)
1 parent fb5e632 commit 6a8e101

File tree

5 files changed

+55
-46
lines changed

5 files changed

+55
-46
lines changed

docs/reference/rest-api/common-parms.asciidoc

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -940,15 +940,13 @@ default value is `60s`.
940940
end::sync-time-delay[]
941941

942942
tag::transform-settings[]
943-
Defines the settings of the {transform}.
943+
Defines optional {transform} settings.
944944
end::transform-settings[]
945945

946946
tag::transform-settings-docs-per-second[]
947-
Set `docs-per-second` to throttle the rate transform is issuing search and
948-
index requests. Throttling slows down search requests by adding a wait time
949-
between searches. For the number of docs, the number of input documents is taken.
950-
Throttling can be disabled by setting it to `null`.
951-
The default is no throttling, value `null`.
947+
Specifies a limit on the number of input documents per second. This setting
948+
throttles the {transform} by adding a wait time between search requests. The
949+
default value is `null`, which disables throttling.
952950
end::transform-settings-docs-per-second[]
953951

954952
tag::transform-settings-max-page-search-size[]

docs/reference/transform/apis/put-transform.asciidoc

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,23 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-group-by]
128128
====
129129
//End pivot
130130

131+
//Begin settings
132+
`settings`::
133+
(Optional, object)
134+
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings]
135+
+
136+
.Properties of `settings`
137+
[%collapsible%open]
138+
====
139+
`docs_per_second`:::
140+
(Optional, float)
141+
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-docs-per-second]
142+
`max_page_search_size`:::
143+
(Optional, integer)
144+
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-max-page-search-size]
145+
====
146+
//End settings
147+
131148
//Begin source
132149
`source`::
133150
(Required, object)
@@ -185,23 +202,6 @@ delays.
185202
====
186203
//End sync
187204

188-
//Begin settings
189-
`settings`::
190-
(Optional, object)
191-
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings]
192-
+
193-
.Properties of `settings`
194-
[%collapsible%open]
195-
====
196-
`docs_per_second`:::
197-
(Optional, float)
198-
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-docs-per-second]
199-
`max_page_search_size`:::
200-
(Optional, integer)
201-
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-max-page-search-size]
202-
====
203-
//End settings
204-
205205

206206
[[put-transform-example]]
207207
==== {api-examples-title}

docs/reference/transform/apis/update-transform.asciidoc

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,23 @@ include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-pipeline]
102102
(Optional, <<time-units, time units>>)
103103
include::{docdir}/rest-api/common-parms.asciidoc[tag=frequency]
104104

105+
//Begin settings
106+
`settings`::
107+
(Optional, object)
108+
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings]
109+
+
110+
.Properties of `settings`
111+
[%collapsible%open]
112+
====
113+
`docs_per_second`:::
114+
(Optional, float)
115+
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-docs-per-second]
116+
`max_page_search_size`:::
117+
(Optional, integer)
118+
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-max-page-search-size]
119+
====
120+
//End settings
121+
105122
//Begin source
106123
`source`::
107124
(Optional, object)
@@ -159,23 +176,6 @@ delays.
159176
====
160177
//End sync
161178

162-
//Begin settings
163-
`settings`::
164-
(Optional, object)
165-
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings]
166-
+
167-
.Properties of `settings`
168-
[%collapsible%open]
169-
====
170-
`docs_per_second`:::
171-
(Optional, float)
172-
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-docs-per-second]
173-
`max_page_search_size`:::
174-
(Optional, integer)
175-
include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-settings-max-page-search-size]
176-
====
177-
//End settings
178-
179179
[[update-transform-example]]
180180
==== {api-examples-title}
181181

docs/reference/transform/checkpoints.asciidoc

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -49,11 +49,7 @@ requires enough resources to support both the composite aggregation search and
4949
the indexing of its results.
5050

5151
TIP: If the cluster experiences unsuitable performance degradation due to the
52-
{transform}, stop the {transform}. Consider whether you can apply a
53-
source query to the {transform} to reduce the scope of data it
54-
processes. Also consider whether the cluster has sufficient resources in place
55-
to support both the composite aggregation search and the indexing of its
56-
results.
52+
{transform}, stop the {transform} and refer to <<transform-performance>>.
5753

5854
[discrete]
5955
[[ml-transform-checkpoint-errors]]

docs/reference/transform/overview.asciidoc

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,4 +72,19 @@ If your {transform} must process a lot of historic data, it has high resource
7272
usage initially--particularly during the first checkpoint.
7373

7474
For better performance, make sure that your search aggregations and queries are
75-
optimized and that your {transform} is processing only necessary data.
75+
optimized and that your {transform} is processing only necessary data. Consider
76+
whether you can apply a source query to the {transform} to reduce the scope of
77+
data it processes. Also consider whether the cluster has sufficient resources in
78+
place to support both the composite aggregation search and the indexing of its
79+
results.
80+
81+
If you prefer to spread out the impact on your cluster (at the cost of a slower
82+
{transform}), you can throttle the rate at which it performs search and index
83+
requests. Set the `docs_per_second` limit when you <<put-transform,create>> or
84+
<<update-transform,update>> your {transform}. If you want to calculate the
85+
current rate, use the following information from the
86+
{ref}/get-transform-stats.html[get {transform} stats API]:
87+
```
88+
documents_processed / search_time_in_ms * 1000
89+
```
90+

0 commit comments

Comments
 (0)