Skip to content

Commit fa5e430

Browse files
mburke5678mburke5678
authored
Merge pull request #20032 from mburke5678/issue-13036-4x
Issues: Edit suggested in file admin_guide/overcommit.adoc 4x
2 parents e86da16 + 2dde483 commit fa5e430

4 files changed

+60
-51
lines changed

‎logging/config/cluster-logging-collector.adoc

+2
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,8 @@ include::modules/cluster-logging-collector-log-location.adoc[leveloffset=+1]
2929

3030
include::modules/cluster-logging-collector-throttling.adoc[leveloffset=+1]
3131

32+
include::modules/nodes-cluster-overcommit-buffer-chunk.adoc[leveloffset=+1]
33+
3234
include::modules/cluster-logging-collector-json.adoc[leveloffset=+1]
3335

3436
include::modules/cluster-logging-collector-undefined.adoc[leveloffset=+1]

‎modules/nodes-cluster-overcommit-buffer-chunk.adoc

+56
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * nodes/nodes-cluster-overcommit.adoc
4+
// * cluster-logging-collector.adoc
5+
6+
7+
[id="understandin-fluentd-buffering_{context}"]
8+
= Understanding Buffer Chunk Limiting for Fluentd
9+
10+
If the Fluentd logger is unable to keep up with a high number of logs, it will need
11+
to switch to file buffering to reduce memory usage and prevent data loss.
12+
13+
Fluentd file buffering stores records in _chunks_. Chunks are stored in _buffers_.
14+
15+
The Fluentd `buffer_chunk_limit` is determined by the environment variable
16+
`BUFFER_SIZE_LIMIT`, which has the default value `8m`. The file buffer size per
17+
output is determined by the environment variable `FILE_BUFFER_LIMIT`, which has
18+
the default value `256Mi`. The permanent volume size must be larger than
19+
`FILE_BUFFER_LIMIT` multiplied by the output.
20+
21+
On the Fluentd pods, permanent volume */var/lib/fluentd* should be
22+
prepared by the PVC or hostmount, for example. That area is then used for the
23+
file buffers.
24+
25+
The `buffer_type` and `buffer_path` are configured in the Fluentd configuration files as
26+
follows:
27+
28+
----
29+
$ egrep "buffer_type|buffer_path" *.conf
30+
output-es-config.conf:
31+
buffer_type file
32+
buffer_path `/var/lib/fluentd/buffer-output-es-config`
33+
output-es-ops-config.conf:
34+
buffer_type file
35+
buffer_path `/var/lib/fluentd/buffer-output-es-ops-config`
36+
----
37+
38+
The Fluentd `buffer_queue_limit` is the value of the variable `BUFFER_QUEUE_LIMIT`. This value is `32` by default.
39+
40+
The environment variable `BUFFER_QUEUE_LIMIT` is calculated as `(FILE_BUFFER_LIMIT / (number_of_outputs * BUFFER_SIZE_LIMIT))`.
41+
42+
If the `BUFFER_QUEUE_LIMIT` variable has the default set of values:
43+
44+
* `FILE_BUFFER_LIMIT = 256Mi`
45+
* `number_of_outputs = 1`
46+
* `BUFFER_SIZE_LIMIT = 8Mi`
47+
48+
The value of `buffer_queue_limit` will be `32`. To change the `buffer_queue_limit`, you must change the value of `FILE_BUFFER_LIMIT`.
49+
50+
In this formula, `number_of_outputs` is `1` if all the logs are sent to a single resource, and it is incremented by `1` for each additional resource. For example, the value of `number_of_outputs` is:
51+
52+
* `1` - if all logs are sent to a single Elasticsearch pod
53+
* `2` - if application logs are sent to an Elasticsearch pod and ops logs are sent to
54+
another Elasticsearch pod
55+
* `4` - if application logs are sent to an Elasticsearch pod, ops logs are sent to
56+
another Elasticsearch pod, and both of them are forwarded to other Fluentd instances

‎modules/nodes-cluster-overcommit-resource-requests.adoc

-51
Original file line numberDiff line numberDiff line change
@@ -24,54 +24,3 @@ difference between request and limit determines the level of overcommit;
2424
for instance, if a container is given a memory request of 1Gi and a memory limit
2525
of 2Gi, it is scheduled based on the 1Gi request being available on the node,
2626
but could use up to 2Gi; so it is 200% overcommitted.
27-
28-
[id="understandin-fluentd-buffering_{context}"]
29-
== Understanding Buffer Chunk Limiting for Fluentd
30-
31-
If the Fluentd logger is unable to keep up with a high number of logs, it will need
32-
to switch to file buffering to reduce memory usage and prevent data loss.
33-
34-
Fluentd file buffering stores records in _chunks_. Chunks are stored in _buffers_.
35-
36-
The Fluentd `buffer_chunk_limit` is determined by the environment variable
37-
`BUFFER_SIZE_LIMIT`, which has the default value `8m`. The file buffer size per
38-
output is determined by the environment variable `FILE_BUFFER_LIMIT`, which has
39-
the default value `256Mi`. The permanent volume size must be larger than
40-
`FILE_BUFFER_LIMIT` multiplied by the output.
41-
42-
On the Fluentd pods, permanent volume */var/lib/fluentd* should be
43-
prepared by the PVC or hostmount, for example. That area is then used for the
44-
file buffers.
45-
46-
The `buffer_type` and `buffer_path` are configured in the Fluentd configuration files as
47-
follows:
48-
49-
----
50-
$ egrep "buffer_type|buffer_path" *.conf
51-
output-es-config.conf:
52-
buffer_type file
53-
buffer_path `/var/lib/fluentd/buffer-output-es-config`
54-
output-es-ops-config.conf:
55-
buffer_type file
56-
buffer_path `/var/lib/fluentd/buffer-output-es-ops-config`
57-
----
58-
59-
The Fluentd `buffer_queue_limit` is the value of the variable `BUFFER_QUEUE_LIMIT`. This value is `32` by default.
60-
61-
The environment variable `BUFFER_QUEUE_LIMIT` is calculated as `(FILE_BUFFER_LIMIT / (number_of_outputs * BUFFER_SIZE_LIMIT))`.
62-
63-
If the `BUFFER_QUEUE_LIMIT` variable has the default set of values:
64-
65-
* `FILE_BUFFER_LIMIT = 256Mi`
66-
* `number_of_outputs = 1`
67-
* `BUFFER_SIZE_LIMIT = 8Mi`
68-
69-
The value of `buffer_queue_limit` will be `32`. To change the `buffer_queue_limit`, you must change the value of `FILE_BUFFER_LIMIT`.
70-
71-
In this formula, `number_of_outputs` is `1` if all the logs are sent to a single resource, and it is incremented by `1` for each additional resource. For example, the value of `number_of_outputs` is:
72-
73-
* `1` - if all logs are sent to a single Elasticsearch pod
74-
* `2` - if application logs are sent to an Elasticsearch pod and ops logs are sent to
75-
another Elasticsearch pod
76-
* `4` - if application logs are sent to an Elasticsearch pod, ops logs are sent to
77-
another Elasticsearch pod, and both of them are forwarded to other Fluentd instances

‎nodes/clusters/nodes-cluster-overcommit.adoc

+2
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,8 @@ include::modules/nodes-cluster-overcommit-about.adoc[leveloffset=+1]
2828

2929
include::modules/nodes-cluster-overcommit-resource-requests.adoc[leveloffset=+1]
3030

31+
include::modules/nodes-cluster-overcommit-buffer-chunk.adoc[leveloffset=+2]
32+
3133
include::modules/nodes-cluster-overcommit-resources-containers.adoc[leveloffset=+1]
3234

3335
include::modules/nodes-cluster-overcommit-qos-about.adoc[leveloffset=+1]

0 commit comments

Comments
 (0)