Pre-merge changes based on comments

Author: anton-bobkov

ydb/docs/en/core/concepts/glossary.md

@@ -101,6 +101,16 @@ Together, these mechanisms allow {{ ydb-short-name }} to provide [strict consist
 
 The implementation of distributed transactions is covered in a separate article [{#T}](../contributor/datashard-distributed-txs.md), while below there's a list of several [related terms](#distributed-transaction-implementation).
 
+### Interactive transactions {#interactive-transaction}
+
+The term **interactive transactions** refers to transactions that are split into multiple queries and involve data processing by an application between these queries. For example:
+
+1. Select some data.
+1. Process the selected data in the application.
+1. Update some data in the database.
+1. Commit the transaction in a separate query.
+
+
 ### Multi-version concurrency control {#mvcc}
 
 [**Multi-version concurrency control**](https://en.wikipedia.org/wiki/Multiversion_concurrency_control) or **MVCC** is a method {{ ydb-short-name }} used to allow multiple concurrent transactions to access the database simultaneously without interfering with each other. It is described in more detail in a separate article [{#T}](mvcc.md).

ydb/docs/en/core/dev/troubleshooting/performance/queries/_includes/overloaded-errors.md

@@ -1,9 +1,23 @@
-1. Open the **DB overview** Grafana dashboard.
+1. Open the **[DB overview](../../../../../reference/observability/metrics/grafana-dashboards.md#dboverview)** Grafana dashboard.
 
 1. In the **API details** section, see if the **Soft errors (retriable)** chart shows any spikes in the rate of queries with the `OVERLOADED` status.
 
     ![](../_assets/soft-errors.png)
 
-1. In the Grafana **DB status** dashboard, see if the number of sessions in the **Session count by host** chart exceeded the 1000 limit.
+1. To check if the spikes in overloaded errors were caused by exceeding the limit of 15000 queries in table partition queues:
+
+    1. In the [Embedded UI](../../../../../reference/embedded-ui/index.md), go to the **Databases** tab and click on the database.
+
+    1. On the **Navigation** tab, ensure the required database is selected.
+
+    1. Open the **Diagnostics** tab.
+
+    1. Open the **Top shards** tab.
+
+    1. In the **Immediate** and **Historical** tabs, sort the shards by the **InFlightTxCount** column and see if the top values reach the 15000 limit.
+
+1. To check if the spikes in overloaded errors were caused by tablet splits and merges, see [{#T}](../../schemas/splits-merges.md).
+
+1. To check if the spikes in overloaded errors were caused by exceeding the 1000 limit of open sessions, in the Grafana **DB status** dashboard, see the **Session count by host** chart.
 
 1. See the [overloaded shards](../../schemas/overloaded-shards.md) issue.

ydb/docs/en/core/dev/troubleshooting/performance/queries/_includes/transaction-lock-invalidation.md

@@ -1,4 +1,4 @@
-1. Open the **DB overview** Grafana dashboard.
+1. Open the **[DB overview](../../../../../reference/observability/metrics/grafana-dashboards.md#dboverview)** Grafana dashboard.
 
 1. See if the **Transaction Locks Invalidation** chart shows any spikes.
 

ydb/docs/en/core/dev/troubleshooting/performance/queries/transaction-lock-invalidation.md

@@ -1,6 +1,6 @@
 # Transaction lock invalidation
 
-Each transaction in {{ ydb-short-name }} uses [optimistic locking](https://en.wikipedia.org/wiki/Optimistic_concurrency_control) to ensure that no other transaction has modified the data it has read or changed. If the locks check reveals conflicting modifications, the committing transaction rolls back and must be restarted. In this case, {{ ydb-short-name }} returns a **transaction locks invalidated** error. Restarting a significant share of transactions can degrade your application's performance.
+{{ ydb-short-name }} uses [optimistic locking](https://en.wikipedia.org/wiki/Optimistic_concurrency_control) to find conflicts with other transactions being executed. If the locks check during the commit phase reveals conflicting modifications, the committing transaction rolls back and must be restarted. In this case, {{ ydb-short-name }} returns a **transaction locks invalidated** error. Restarting a significant share of transactions can degrade your application's performance.
 
 {% note info %}
 
@@ -16,13 +16,14 @@ The YDB SDK provides a built-in mechanism for handling temporary failures. For m
 
 ## Recommendations
 
-The longer a transaction lasts, the higher the likelihood of encountering a **transaction locks invalidated** error.
+Consider the following recommendations:
 
-If possible, avoid interactive transactions. For example, try to avoid the following pattern:
+- The longer a transaction lasts, the higher the likelihood of encountering a **transaction locks invalidated** error.
 
-1. Select some data.
-1. Process the selected data in the application.
-1. Update some data in the database.
-1. Commit the transaction in a separate query.
+    If possible, avoid [interactive transactions](../../../../concepts/glossary.md#interactive-transaction). A better approach is to use a single YQL query with `begin;` and `commit;` to select data, update data, and commit the transaction.
 
-A better approach is to use a single YQL query to select data, update data, and commit the transaction.
+    If you do need interactive transactions, append `commit;` to the last query in the transaction.
+
+- Analyze the range of primary keys where conflicting modifications occur, and try to change the application logic to reduce the number of conflicts.
+
+    For example, if a single row with a total balance value is frequently updated, split this row into a hundred rows and calculate the total balance as a sum of these rows. This will drastically reduce the number of **transaction locks invalidated** errors.

ydb/docs/en/core/dev/troubleshooting/performance/schemas/_assets/describe.png

@@ -1,44 +1,80 @@
-1. Analyze the **Overloaded shard count** chart in the **DB overview** Grafana dashboard.
+1. Use the Embedded UI or Grafana to see if the {{ ydb-short-name }} nodes are overloaded:
 
-    ![](../_assets/overloaded-shards-dashboard.png)
+    - In the **[DB overview](../../../../../reference/observability/metrics/grafana-dashboards.md#dboverview)** Grafana dashboard, analyze the **Overloaded shard count** chart.
 
-    The chart indicates whether the {{ ydb-short-name }} cluster has overloaded shards, but it does not specify which table's shards are overloaded.
+        ![](../_assets/overloaded-shards-dashboard.png)
 
-2. To identify the table with overloaded shards, follow these steps:
+        The chart indicates whether the {{ ydb-short-name }} cluster has overloaded shards, but it does not specify which table's shards are overloaded.
 
-    1. In the [Embedded UI](../../../../../reference/embedded-ui/index.md), go to the **Databases** tab and click on the database.
+        {% note tip %}
 
-    2. On the **Navigation** tab, ensure the required database is selected.
+        Use Grafana to set up alert notifications when {{ ydb-short-name }} data shards get overloaded.
 
-    3. Open the **Diagnostics** tab.
+        {% endnote %}
 
-    4. Open the **Top shards** tab.
 
-    5. In the **Immediate** and **Historical** tabs, sort the shards by the **CPUCores** column and analyze the information.
+    - In the [Embedded UI](../../../../../reference/embedded-ui/index.md):
 
-    ![](../_assets/partitions-by-cpu.png)
+        1. Go to the **Databases** tab and click on the database.
 
-    Additionally, the information about overloaded shards is provided as a system table. For more information, see [{#T}](../../../../system-views.md#top-overload-partitions).
+        1. On the **Navigation** tab, ensure the required database is selected.
 
-    {% endnote %}
+        1. Open the **Diagnostics** tab.
 
-3. To pinpoint the schema issue, follow these steps:
+        1. Open the **Top shards** tab.
 
-    1. Retrieve information about the problematic table using the [{{ ydb-short-name }} CLI](../../../../../reference/ydb-cli/index.md). Run the following command:
+        1. In the **Immediate** and **Historical** tabs, sort the shards by the **CPUCores** column and analyze the information.
 
-        ```bash
-        ydb scheme describe <table_name>
-        ```
+        ![](../_assets/partitions-by-cpu.png)
 
-    2. In the command output, analyze the **Auto partitioning settings**:
+        Additionally, the information about overloaded shards is provided as a system table. For more information, see [{#T}](../../../../system-views.md#top-overload-partitions).
 
-        * `Partitioning by size`
-        * `Partitioning by load`
-        * `Max partitions count`
+1. To pinpoint the schema issue, use the [Embedded UI](../../../../../reference/embedded-ui/index.md) or [{{ ydb-short-name }} CLI](../../../../../reference/ydb-cli/index.md):
 
-        If the table does not have these options, see [Recommendations for table configuration](../overloaded-shards.md#table-config).
+    - In the [Embedded UI](../../../../../reference/embedded-ui/index.md):
 
-4. Analyze whether primary key values increment monotonically:
+        1. On the **Databases** tab, click on the database.
+
+        1. On the **Navigation** tab, select the required table.
+
+        1. Open the **Diagnostics** tab.
+
+        1. On the **Describe** tab, navigate to `root > PathDescription > Table > PartitionConfig > PartitioningPolicy`.
+
+            ![Describe](../_assets/describe.png)
+
+        1. Analyze the **PartitioningPolicy** values:
+
+            - `SizeToSplit`
+            - `SplitByLoadSettings`
+            - `MaxPartitionsCount`
+
+            If the table does not have these options, see [Recommendations for table configuration](../overloaded-shards.md#table-config).
+
+        {% note info %}
+
+        You can also find this information on the **Diagnostics > Info** tab.
+
+        {% endnote %}
+
+
+    - In the [{{ ydb-short-name }} CLI](../../../../../reference/ydb-cli/index.md):
+
+        1. To retrieve information about the problematic table, run the following command:
+
+            ```bash
+            ydb scheme describe <table_name>
+            ```
+
+        2. In the command output, analyze the **Auto partitioning settings**:
+
+            - `Partitioning by size`
+            - `Partitioning by load`
+            - `Max partitions count`
+
+            If the table does not have these options, see [Recommendations for table configuration](../overloaded-shards.md#table-config).
+
+1. Analyze whether primary key values increment monotonically:
 
     - Check the data type of the primary key column. `Serial` data types are used for autoincrementing values.
 

ydb/docs/en/core/dev/troubleshooting/performance/schemas/_includes/overloaded-shards-diagnostics.md

@@ -27,10 +27,24 @@ Consider the following solutions to address shard overload:
 
 * If the problematic table is not partitioned by load, enable partitioning by load.
 
+    {% note tip %}
+
+    A table is not partitioned by load, if you see the `Partitioning by load: false` line on the **Diagnostics > Info** tab in the **Embedded UI** or the  `ydb scheme describe` command output.
+
+    {% endnote %}
+
 * If the table has reached the maximum number of partitions, increase the partition limit.
 
+    {% note tip %}
+
+    To determine the number of partitions in the table, see the `PartCount` value on the **Diagnostics > Info** tab in the **Embedded UI**.
+
+    {% endnote %}
+
+
 Both operations can be performed by executing an [`ALTER TABLE ... SET`](../../../../yql/reference/syntax/alter_table/set.md) query.
 
+
 ### For the imbalanced primary key {#pk-recommendations}
 
 Consider modifying the primary key to distribute the load evenly across table partitions. You cannot change the primary key of an existing table. To do that, you will have to create a new table with the modified primary key and then migrate the data to the new table.

ydb/docs/en/core/dev/troubleshooting/performance/schemas/overloaded-shards.md

@@ -6,7 +6,7 @@
 
 {% endif %}
 
-Each [row-oriented table](../../../../concepts/datamodel/table.md#row-oriented-tables) partition in {{ ydb-short-name }} is processed by a [data shard](../../../../concepts/glossary.md#data-shard) tablet. {{ ydb-short-name }} supports automatic splitting and merging of data shards which allows it to seamlessly adapt to changes in workloads. However, these operations are not free and might have a short-term negative impact on query latencies.
+Each [row-oriented table](../../../../concepts/datamodel/table.md#row-oriented-tables) partition in {{ ydb-short-name }} is processed by a [data shard](../../../../concepts/glossary.md#data-shard) tablet. {{ ydb-short-name }} supports automatic [splitting and merging](../../../../concepts/datamodel/table.md#partitioning) of data shards which allows it to seamlessly adapt to changes in workloads. However, these operations are not free and might have a short-term negative impact on query latencies.
 
 When {{ ydb-short-name }} splits a partition, it replaces the original partition with two new partitions covering the same range of primary keys. Now, two data shards process the range of primary keys that was previously handled by a single data shard, thereby adding more computing resources for the table.
 
@@ -27,6 +27,6 @@ When configuring [table partitioning](../../../../concepts/datamodel/table.md#pa
 
 ## Recommendations
 
-If the user load on {{ ydb-short-name }} has not changed, consider adjusting the gap between the min and max limits for the number of table partitions to the recommended 20% difference.
+If the user load on {{ ydb-short-name }} has not changed, consider adjusting the gap between the min and max limits for the number of table partitions to the recommended 20% difference. Use the [`ALTER TABLE table_name SET (key = value)`](../../../../yql/reference/syntax/alter_table/set.md) YQL statement to update the [`AUTO_PARTITIONING_MIN_PARTITIONS_COUNT`](../../../../concepts/datamodel/table.md#auto_partitioning_min_partitions_count) and [`AUTO_PARTITIONING_MAX_PARTITIONS_COUNT`](../../../../concepts/datamodel/table.md#auto_partitioning_max_partitions_count) parameters.
 
 If you want to avoid splitting and merging data shards, you can set the min limit to the max limit value or disable partitioning by load.

ydb/docs/en/core/dev/troubleshooting/performance/schemas/splits-merges.md

@@ -2,9 +2,14 @@
 
 Synchronized clocks are critical for distributed databases. If system clocks on the {{ ydb-short-name }} servers drift excessively, distributed transactions will experience increased latencies.
 
-If a {{ ydb-short-name }} cluster with multiple [coordinators](../../../../concepts/glossary.md#coordinator), planned transactions are merged by [mediators](../../../../concepts/glossary.md#mediator) before being sent off for execution.
+{% note alert %}
 
-If the system clocks of the nodes running the coordinator tablets differ, transaction latencies increase by the time difference between the fastest and slowest system clocks. This occurs because a transaction planned on a node with a faster system clock can only be executed once the coordinator with the slowest clock reaches the same time.
+It is important to keep system clocks on the {{ ydb-short-name }} servers in sync, to avoid high latencies.
+
+{% endnote %}
+
+
+If the system clocks of the nodes running the [coordinator](../../../../concepts/glossary.md#coordinator) tablets differ, transaction latencies increase by the time difference between the fastest and slowest system clocks. This occurs because a transaction planned on a node with a faster system clock can only be executed once the coordinator with the slowest clock reaches the same time.
 
 Furthermore, if the system clock drift exceeds 30 seconds, {{ ydb-short-name }} will refuse to process distributed transactions. Before coordinators start planning a transaction, affected [Data shards](../../../../concepts/glossary.md#data-shard) determine an acceptable range of timestamps for the transaction. The start of this range is the current time of the mediator tablet's clock, while the 30-second planning timeout determines the end. If the coordinator's system clock exceeds this time range, it cannot plan a distributed transaction, resulting in errors for such queries.
 

ydb/docs/en/core/dev/troubleshooting/performance/system/system-clock-drift.md

@@ -2,15 +2,12 @@
 
 {{ ydb-short-name }} automatically balances the load by moving tablets from overloaded nodes to other nodes. This process is managed by [Hive](../../../../concepts/glossary.md#hive). When Hive moves tablets, queries affecting those tablets might experience increased latencies while they wait for the tablet to get initialized on the new node.
 
-[//]: # (This information is taken from a draft topic Concepts > Hive.)
-[//]: # (TODO: When the above-mentioned topic is merged, remove the info from here and add a link.)
-
 {{ ydb-short-name }} considers usage of the following hardware resources for balancing nodes:
 
 - CPU
 - Memory
 - Network
-- [Counter](*counter)
+- [Count](*count)
 
 Autobalancing occurs in the following cases:
 
@@ -86,6 +83,5 @@ Adjust Hive balancer settings:
     {% endnote %}
 
 
-
-[*counter]: A virtual resource is used for balancing tablets that lack other hardware resource metrics (such as CPU, memory, or network) and for column shards. If a tablet uses this resource, its value is always set to 1.
+[*count]: Count is a virtual resource for distributing tablets of the same type evenly between nodes.