Skip to content

Commit e4118ac

Browse files
authored
Refine size-your-shards wording (#89081) (#89160)
Clarify that the limits in the docs are absolute maxima that will avoid things just breaking but won't necessarily give great performance.
1 parent 9c1bbb4 commit e4118ac

File tree

1 file changed

+22
-7
lines changed

1 file changed

+22
-7
lines changed

docs/reference/how-to/size-your-shards.asciidoc

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -175,17 +175,25 @@ index prirep shard store
175175

176176
[discrete]
177177
[[shard-count-recommendation]]
178-
==== Aim for 3000 indices or fewer per GB of heap memory on each master node
178+
==== Master-eligible nodes should have at least 1GB of heap per 3000 indices
179179

180180
The number of indices a master node can manage is proportional to its heap
181181
size. The exact amount of heap memory needed for each index depends on various
182182
factors such as the size of the mapping and the number of shards per index.
183183

184-
As a general rule of thumb, you should aim for 3000 indices or fewer per GB of
185-
heap on master nodes. For example, if your cluster contains 12000 indices then
186-
each dedicated master node should have at least 4GB of heap. For non-dedicated
187-
master nodes, the same rule holds and should be added to the heap requirements
188-
of the other roles of each node.
184+
As a general rule of thumb, you should have fewer than 3000 indices per GB of
185+
heap on master nodes. For example, if your cluster has dedicated master nodes
186+
with 4GB of heap each then you should have fewer than 12000 indices. If your
187+
master nodes are not dedicated master nodes then the same sizing guidance
188+
applies: you should reserve at least 1GB of heap on each master-eligible node
189+
for every 3000 indices in your cluster.
190+
191+
Note that this rule defines the absolute maximum number of indices that a
192+
master node can manage, but does not guarantee the performance of searches or
193+
indexing involving this many indices. You must also ensure that your data nodes
194+
have adequate resources for your workload and that your overall sharding
195+
strategy meets all your performance requirements. See also
196+
<<single-thread-per-shard>> and <<each-shard-has-overhead>>.
189197

190198
To check the configured size of each node's heap, use the <<cat-nodes,cat nodes
191199
API>>.
@@ -207,7 +215,7 @@ GET _cat/shards?v=true
207215

208216
[discrete]
209217
[[field-count-recommendation]]
210-
==== Allow 1kB of heap per field per index on data nodes, plus overheads
218+
==== Data nodes should have at least 1kB of heap per field per index, plus overheads
211219

212220
The exact resource usage of each mapped field depends on its type, but a rule
213221
of thumb is to allow for approximately 1kB of heap overhead per mapped field
@@ -222,6 +230,13 @@ For example, if a data node holds shards from 1000 indices, each containing
222230
of heap for the fields and another 0.5GB of heap for its workload and other
223231
overheads, and therefore this node will need a heap size of at least 4.5GB.
224232

233+
Note that this rule defines the absolute maximum number of indices that a data
234+
node can manage, but does not guarantee the performance of searches or indexing
235+
involving this many indices. You must also ensure that your data nodes have
236+
adequate resources for your workload and that your overall sharding strategy
237+
meets all your performance requirements. See also <<single-thread-per-shard>>
238+
and <<each-shard-has-overhead>>.
239+
225240
[discrete]
226241
[[avoid-node-hotspots]]
227242
==== Avoid node hotspots

0 commit comments

Comments
 (0)