You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: ydb/docs/en/core/development/load-actors-storage.md
+69-18Lines changed: 69 additions & 18 deletions
Original file line number
Diff line number
Diff line change
@@ -2,33 +2,58 @@
2
2
3
3
Tests the read/write performance to and from Distributed Storage. The load is generated on Distributed Storage directly without using any tablet and Query Processor layers. When testing write performance, the actor writes data to the specified VDisk group. To test read performance, the actor first writes data to the specified VDisk group and then reads the data. After the load is removed, all the data written by the actor is deleted.
4
4
5
-
You can generate two types of load:
5
+
You can generate three types of load:
6
6
7
-
*_Continuous_: The actor ensures the specified number of requests are run concurrently. To generate a continuous load, set a zero interval between requests (e.g., `WriteIntervals: { Weight: 1.0 Uniform: { MinUs: 0 MaxUs: 0 } }`), while keeping the `MaxInFlightWriteRequests` parameter value different from zero.
8
-
*_Interval_: The actor runs requests at preset intervals. To generate an interval load, set a non-zero interval between requests, e.g., `WriteIntervals: { Weight: 1.0 Uniform: { MinUs: 50000 MaxUs: 50000 } }`. Maximum number of in-flight requests is set by the `InFlightReads` parameter. If its value is `0`, their number is unlimited.
7
+
*_Continuous_: The actor ensures the specified number of requests are run concurrently. To generate a continuous load, set a zero interval between requests (e.g., `WriteIntervals: { Weight: 1.0 Uniform: { MinUs: 0 MaxUs: 0 } }`), while keeping the `MaxInFlightWriteRequests` parameter value different from zero and the `WriteHardRateDispatcher` parameter being absent.
8
+
*_Interval_: The actor runs requests at preset intervals. To generate an interval load, set a non-zero interval between requests, e.g., `WriteIntervals: { Weight: 1.0 Uniform: { MinUs: 50000 MaxUs: 50000 } }` and don't set the `WriteHardRateDispatcher` parameter. Maximum number of in-flight requests is set by the `InFlightWrites` parameter. If its value is `0`, their number is unlimited.
9
+
*_Hard rate_: The actor runs requests at certain intervals, the value of which is adjusted to maintain preset request rate per second. If the duration of the load is limited by `LoadDuration` than the request rate may differ at the moment of start and the moment of finishing the load and will alter gradually throughout all the main load cycle. To generate a load of this type, set the [parameters of hard rate load](#hardRare) (parameter `WriteHardRateDispatcher`). Note that if this parameter is set, the hard rate type of load will be launched, regardless the value of the `WriteIntervals` parameter. Maximum number of in-flight requests is set by the `InFlightWrites` parameter. If its value is `0`, their number is unlimited.
9
10
10
11
## Actor parameters {#options}
11
12
12
13
{% include [load-actors-params](../_includes/load-actors-params.md) %}
13
14
14
15
| Parameter | Description |
15
16
--- | ---
16
-
|` DurationSeconds`| Load duration. |
17
-
|` Tablets`| The load is generated on behalf of a tablet with the following parameters:<ul><li>` TabletId`: Tablet ID. It must be unique for each load actor.</li><li>` Channel`: Tablet channel.</li><li>` GroupId`: ID of the VDisk group to get loaded.</li><li>` Generation`: Tablet generation.</li></ul> |
18
-
|` WriteSizes`| Size of the data to write. It is selected randomly for each request from the `Min`-`Max` range. You can set multiple `WriteSizes` ranges, in which case a value from a specific range will be selected based on its `Weight`. |
19
-
|` WriteIntervals`| Setting up the [parameters for probabilistic distribution](#params) of intervals between the records loaded at intervals (in milliseconds). You can set multiple `WriteIntervals` ranges, in which case a value from a specific range will be selected based on its `Weight`. |
20
-
|` MaxInFlightWriteRequests`| Maximum number of simultaneously processed write requests. |
21
-
|` ReadSizes`| Size of the data to read. It is selected randomly for each request from the `Min`-`Max` range. You can set multiple `ReadSizes` ranges, in which case a value from a specific range will be selected based on its `Weight`. |
22
-
|` ReadIntervals`| Setting up the [parameters for probabilistic distribution](#params) of intervals between the queries loaded by intervals (in milliseconds). You can set multiple `ReadIntervals` ranges, in which case a value from a specific range will be selected based on its `Weight`. |
23
-
|` MaxInFlightReadRequests`| Maximum number of simultaneously processed read requests. |
24
-
|` FlushIntervals`| Setting up the [parameters for probabilistic distribution](#params) of intervals (in microseconds) between the queries used to delete data written by the StorageLoad actor. You can set multiple `FlushIntervals` ranges, in which case a value from a specific range will be selected based on its `Weight`. |
25
-
|` PutHandleClass`| Class of data writes to the disk subsystem. If the `TabletLog` value is set, the write operation has the highest priority. |
26
-
|` GetHandleClass`| Class of data reads from the disk subsystem. If the `FastRead` is set, the read operation is performed with the highest speed possible. |
17
+
|`DurationSeconds`| Load duration. The timer is started upon completing the initial data allocation. |
18
+
|`Tablets`| The load is generated on behalf of a tablet with the following parameters:<ul><li>`TabletId`: Tablet ID. It must be unique for each load actor across all the cluster nodes. This parameter and `TabletName` are mutually exclusive.</li><li>`TabletName`: Tablet name. If the parameter is set, tablets' IDs will be assigned automatically, tablets launched on the same node with the same name will be given the same ID, tablets launched on different nodes will be given different IDs.</li><li>`Channel`: Tablet channel.</li><li>`GroupId`: ID of the storage group to get loaded.</li><li>`Generation`: Tablet generation.</li></ul> |
19
+
|`WriteSizes`| Size of the data to write. It is selected randomly for each request from the `Min`-`Max` range. You can set multiple `WriteSizes` ranges, in which case a value from a specific range will be selected based on its `Weight`. |
20
+
|`WriteHardRateDispatcher`| Setting up the [parameters of load with hard rate](#hardRate) for write requests. If this parameter is set than the value of `WriteIntervals` is ignored. |
21
+
|`WriteIntervals`| Setting up the [parameters for probabilistic distribution](#params) of intervals between the records loaded at intervals (in milliseconds). You can set multiple `WriteIntervals` ranges, in which case a value from a specific range will be selected based on its `Weight`. |
22
+
|`MaxInFlightWriteRequests`| Maximum number of simultaneously processed write requests. |
23
+
|`ReadSizes`| Size of the data to read. It is selected randomly for each request from the `Min`-`Max` range. You can set multiple `ReadSizes` ranges, in which case a value from a specific range will be selected based on its `Weight`. |
24
+
|`WriteHardRateDispatcher`| Setting up the [parameters of load with hard rate](#hardRate) for read requests. If this parameter is set than the value of `ReadIntervals` is ignored. |
25
+
|`ReadIntervals`| Setting up the [parameters for probabilistic distribution](#params) of intervals between the queries loaded by intervals (in milliseconds). You can set multiple `ReadIntervals` ranges, in which case a value from a specific range will be selected based on its `Weight`. |
26
+
|`MaxInFlightReadRequests`| Maximum number of simultaneously processed read requests. |
27
+
|`FlushIntervals`| Setting up the [parameters for probabilistic distribution](#params) of intervals (in microseconds) between the queries used to delete data written by the write requests in the main load cycle of the StorageLoad actor. You can set multiple `FlushIntervals` ranges, in which case a value from a specific range will be selected based on its `Weight`. Only one flush request will be processed simultaneously. |
28
+
|`PutHandleClass`| Class of data writes to the disk subsystem. If the `TabletLog` value is set, the write operation has the highest priority. |
29
+
|`GetHandleClass`| Class of data reads from the disk subsystem. If the `FastRead` is set, the read operation is performed with the highest speed possible. |
30
+
|`Initial allocation`| Setting up the [parameters for initial data allocation](#initialAllocation). It defines the amount of data to be written before the start of the main load cycle which can later be read by read requests along with the data written in the main load cycle. |
27
31
28
32
### Parameters of probabilistic distribution {#params}
29
33
30
34
{% include [load-actors-params](../_includes/load-actors-interval.md) %}
31
35
36
+
### Parameters of load with hard rate {#hardRate}
37
+
38
+
| Parameter | Description |
39
+
--- | ---
40
+
|`RequestRateAtStart`| Requests per second at the moment of load start. If load duration limit is not set then the request rate will remain the same and equal to the value of this parameter. |
41
+
|`RequestRateOnFinish`| Requests per second at the moment of load finish. |
42
+
43
+
### Parameters of initial data allocation {#initialAllocation}
44
+
45
+
| Parameter | Description |
46
+
--- | ---
47
+
|`TotalSize`| Total size of allocated data. This parameter and `BlobsNumber` are mutually exclusive. |
48
+
|`BlobsNumber`| Total number of allocated blobs. |
49
+
|`BlobSizes`| Size of the blobs to write. It is selected randomly for each request from the `Min`-`Max` range. You can set multiple `WriteSizes` ranges, in which case a value from a specific range will be selected based on its `Weight`. |
50
+
|`MaxWritesInFlight`| Maximum number of simultaneously processed write requests. If this parameter is not set then the number of simultaneously processed requests is not limited. |
51
+
|`MaxWriteBytesInFlight`| Maximum number of total amount of simultaneously processed write requests' data. If this parameter is not set then the total amount of simultaneously processed write requests' data is not limited. |
52
+
|`PutHandleClass`| Class of data writes to the disk subsystem. |
53
+
|`DelayAfterCompletionSec`| The amount of time in seconds the actor will wait upon completing the initial data allocation before starting the main load cycle. If its value is `0` or not set the load will start immediately after the completion of the data allocaion. |
54
+
55
+
{% include [load-actors-params](../_includes/load-actors-interval.md) %}
56
+
32
57
## Examples {#examples}
33
58
34
59
### Write load {#write}
@@ -51,8 +76,8 @@ StorageLoad: {
51
76
52
77
When viewing test results, the following values should be of most interest to you:
53
78
54
-
*`Writes per second`: Number of writes per second, e.g., `28690.29`.
55
-
*`Speed@ 100%`: 100 percentile of write speed in MB/s, e.g., `108.84`.
79
+
*`Writes per second`: Number of writes per second, e.g., `28690.29`.
80
+
*`Speed@ 100%`: 100 percentile of write speed in MB/s, e.g., `108.84`.
56
81
57
82
### Read load {#read}
58
83
@@ -76,7 +101,33 @@ StorageLoad: {
76
101
}
77
102
}
78
103
```
79
-
80
104
When viewing test results, the following value should be of most interest to you:
81
105
82
-
*` ReadSpeed@ 100%`: 100 percentile of read speed in MB/s, e.g., `60.86`.
106
+
*`ReadSpeed@ 100%`: 100 percentile of read speed in MB/s, e.g., `60.86`.
107
+
108
+
### Read only {#readonly}
109
+
110
+
Before the start of the main load cycle the `1 GB` data block of blobs with sizes between `1 MB` and `5 MB` is allocated. To avoid overloading the system with write requests the number of simultaneously processed requests is limited by the value of `5`. After completing the initial data aloocation the main cycle is launched, which consists of read requests sent with increasing rate: from `10` to `50` requests per second, the rate increase gradually for `300` seconds.
Calculated percentiles will only represent the requests of the main load cycle and won't include write requests sent during the initial data allocation. The graphs in Monitoring should be of interest, for example, they allow to trace the request latency degradation caused by the increasing load.
0 commit comments