Skip to content

YDBDOCS-633: continue reorganizing docs #3353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Apr 3, 2024
Merged
2 changes: 1 addition & 1 deletion ydb/docs/en/core/administration/upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ All minor versions within a major version are compatible for updates. Major vers
* X.Y.* → X.Y+2.*: Update is impossible, major versions are inconsistent.
* X.Y.* → X.Y-2.*: Update is impossible, major versions are inconsistent.

A list of available versions can be found on the [download page](https://ydb.tech/en/docs/downloads/). The YDB release policy is described in more details in the [Release management](../development/manage-releases.md) article of the YDB development documentation.
A list of available versions can be found on the [download page](https://ydb.tech/en/docs/downloads/). The YDB release policy is described in more details in the [Release management](../contributor/manage-releases.md) article of the YDB development documentation.

{% note warning %}

Expand Down
10 changes: 0 additions & 10 deletions ydb/docs/en/core/best_practices/_includes/index.md

This file was deleted.

2 changes: 0 additions & 2 deletions ydb/docs/en/core/best_practices/_includes/schema_design.md

This file was deleted.

2 changes: 0 additions & 2 deletions ydb/docs/en/core/best_practices/_includes/table_sharding.md

This file was deleted.

2 changes: 0 additions & 2 deletions ydb/docs/en/core/best_practices/batch_upload.md

This file was deleted.

1 change: 0 additions & 1 deletion ydb/docs/en/core/best_practices/index.md

This file was deleted.

3 changes: 0 additions & 3 deletions ydb/docs/en/core/best_practices/paging.md

This file was deleted.

6 changes: 0 additions & 6 deletions ydb/docs/en/core/best_practices/pk_scalability.md

This file was deleted.

2 changes: 0 additions & 2 deletions ydb/docs/en/core/best_practices/schema_design.md

This file was deleted.

3 changes: 0 additions & 3 deletions ydb/docs/en/core/best_practices/secondary_indexes.md

This file was deleted.

2 changes: 0 additions & 2 deletions ydb/docs/en/core/best_practices/table_sharding.md

This file was deleted.

2 changes: 0 additions & 2 deletions ydb/docs/en/core/best_practices/timeouts.md

This file was deleted.

24 changes: 0 additions & 24 deletions ydb/docs/en/core/best_practices/toc_i.yaml

This file was deleted.

2 changes: 0 additions & 2 deletions ydb/docs/en/core/best_practices/toc_p.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion ydb/docs/en/core/changelog-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ Release date: May 1, 2023. To update to version **2.3.0**, select the [Downloads
**Features:**

* Added the interactive mode of query execution. To switch to the interactive mode, run [ydb yql](reference/ydb-cli/yql.md) without arguments. This mode is experimental: backward compatibility is not guaranteed yet.
* Added the [ydb index rename](reference/ydb-cli/commands/secondary_index.md#rename) command for [atomic replacement](best_practices/secondary_indexes.md#atomic-index-replacement) or renaming of a secondary index.
* Added the [ydb index rename](reference/ydb-cli/commands/secondary_index.md#rename) command for [atomic replacement](dba/secondary-indexes.md#atomic-index-replacement) or renaming of a secondary index.
* Added the `ydb workload topic` command for generating the load that reads messages from topics and writes messages to topics.
* Added the [--recursive](reference/ydb-cli/commands/dir.md#rmdir-options) option for the `ydb scheme rmdir` command. Use it to delete a directory recursively, with all its content.
* Added support for the `topic` and `coordination node` types in the [ydb scheme describe](reference/ydb-cli/commands/scheme-describe.md) command.
Expand Down
2 changes: 1 addition & 1 deletion ydb/docs/en/core/changelog-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ Release date: May 5, 2023. To update to version 23.1, select the [Downloads](dow
**Functionality:**

* Added [initial table scan](concepts/cdc.md#initial-scan) when creating a CDC changefeed. Now, you can export all the data existing at the time of changefeed creation.
* Added [atomic index replacement](best_practices/secondary_indexes.md#atomic-index-replacement). Now, you can atomically replace one pre-defined index with another. This operation is absolutely transparent for your application. Indexes are replaced seamlessly, with no downtime.
* Added [atomic index replacement](dba/secondary-indexes.md#atomic-index-replacement). Now, you can atomically replace one pre-defined index with another. This operation is absolutely transparent for your application. Indexes are replaced seamlessly, with no downtime.
* Added the [audit log](cluster/audit-log.md): Event stream including data about all the operations on {{ ydb-short-name }} objects.

**Performance:**
Expand Down
2 changes: 1 addition & 1 deletion ydb/docs/en/core/cluster/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@ This section provides information about deploying, configuring, maintaining, mon
* [{#T}](../deploy/index.md).
* [{#T}](../maintenance/embedded_monitoring/index.md).
* [{#T}](../maintenance/manual/index.md).
* [{#T}](../troubleshooting/system_views_cluster.md).
* [{#T}](../devops/manual/system-views.md).
* [{#T}](../administration/monitoring.md).
* [{#T}](../administration/upgrade.md).
2 changes: 1 addition & 1 deletion ydb/docs/en/core/cluster/toc_i.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ items:
- name: Embedded UI
include: { mode: link, path: ../maintenance/embedded_monitoring/toc_p.yaml }
- name: Cluster system views
href: ../troubleshooting/system_views_cluster.md
href: ../devops/manual/system-views.md
- name: Audit log
href: audit-log.md
- name: Short access control notation
Expand Down
2 changes: 1 addition & 1 deletion ydb/docs/en/core/concepts/_includes/scan_query.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ This method of executing queries has the following unique features:

{% node info %}

From the *Scan Queries* interface, you can query [system tables](../../troubleshooting/system_views_db.md).
From the *Scan Queries* interface, you can query [system tables](../../dba/system-views.md).

{% endnote %}

Expand Down
2 changes: 1 addition & 1 deletion ydb/docs/en/core/concepts/_includes/secondary_indexes.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,5 +59,5 @@ A secondary index can be:

## Purpose and use of secondary indexes {#best_practices}

For information about the purpose and use of secondary indexes for app development, see the [recommendations](../../best_practices/secondary_indexes.md).
For information about the purpose and use of secondary indexes for app development, see the [recommendations](../../dba/secondary-indexes.md).

2 changes: 1 addition & 1 deletion ydb/docs/en/core/concepts/cdc.md
Original file line number Diff line number Diff line change
Expand Up @@ -243,4 +243,4 @@ You can add a changefeed to an existing table or erase it using the [ADD CHANGEF

## CDC purpose and use {#best_practices}

For information about using CDC when developing apps, see [best practices](../best_practices/cdc.md).
For information about using CDC when developing apps, see [best practices](../dba/cdc.md).
8 changes: 4 additions & 4 deletions ydb/docs/en/core/concepts/datamodel/_includes/table.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Searching using an index allows you to swiftly locate the required rows without

You can create a row-oriented table through the YDB web interface, CLI, or SDK. Regardless of the method you choose to interact with YDB, it's important to keep in mind the following rule: the table must have at least one primary key column, and it's permissible to create a table consisting solely of primary key columns.

By default, when creating a row-oriented table, all columns are optional and can have `NULL` values. This behavior can be modified by setting the `NOT NULL` conditions for key columns that are part of the primary key. Primary keys are unique, and row-oriented tables are always sorted by this key. This means that point reads by the key, as well as range queries by key or key prefix, are efficiently executed (essentially using an index). It's permissible to create a table consisting solely of key columns. When choosing a key, it's crucial to be careful, so we recommend reviewing the article: ["Choosing a Primary Key for Maximum Performance"](../../../best_practices/pk_scalability.md).
By default, when creating a row-oriented table, all columns are optional and can have `NULL` values. This behavior can be modified by setting the `NOT NULL` conditions for key columns that are part of the primary key. Primary keys are unique, and row-oriented tables are always sorted by this key. This means that point reads by the key, as well as range queries by key or key prefix, are efficiently executed (essentially using an index). It's permissible to create a table consisting solely of key columns. When choosing a key, it's crucial to be careful, so we recommend reviewing the article: ["Choosing a Primary Key for Maximum Performance"](../../../dba/primary-key/row-oriented.md).

### Partitioning row-oriented tables {#partitioning}

Expand Down Expand Up @@ -167,9 +167,9 @@ Column-oriented {{ ydb-short-name }} tables are in the Preview mode.

{% endnote %}

YDB's column-oriented tables stores data of each column separately (independently) from each other. This data storage principle is optimized for handling Online Analytical Processing (OLAP) workloads, as only the columns directly involved in the query are read during its execution. One of the key advantages of this approach is the high data compression ratios since columns often contain repetitive or similar data. A downside, however, is that operations on whole rows become more resource-intensive.
YDB's column-oriented tables store data of each column separately (independently) from each other. This data storage principle is optimized for handling Online Analytical Processing (OLAP) workloads, as only the columns directly involved in the query are read during its execution. One of the key advantages of this approach is the high data compression ratios since columns often contain repetitive or similar data. A downside, however, is that operations on whole rows become more resource-intensive.

At the moment, the main use case for YDB column-oriented tables is writing data with an increasing primary key (for example, event time), analyzing this data, and deleting outdated data based on TTL. The optimal way to add data to YDB column-oriented tables is [batch upload](../../../best_practices/batch_upload.md), performed in MB-sized blocks. Data packet insertion is atomic: data will be written either to all partitions or none.
At the moment, the main use case for YDB column-oriented tables is writing data with an increasing primary key (for example, event time), analyzing this data, and deleting outdated data based on TTL. The optimal way to add data to YDB column-oriented tables is [batch upload](../../../dev/batch-upload.md), performed in MB-sized blocks. Data packet insertion is atomic: data will be written either to all partitions or none.

In most cases, working with YDB column-oriented tables is similar to working with row tables, but there are differences:
* Only `NOT NULL` columns can be used as the primary key.
Expand Down Expand Up @@ -221,7 +221,7 @@ WITH (STORE = COLUMN);
Unlike data partitioning in row-oriented {{ ydb-short-name }} tables, key values are not used to partition data in column-oriented tables. This way, you can uniformly distribute data across all your existing partitions. This kind of partitioning enables you to avoid hotspots at data inserta and speeding up analytical queries that process (that is, read) large amounts of data.


How you select partitioning keys substantially affects the performance of queries to your column-oriented tables. Learn more in [{#T}](../../../best_practices/pk-olap-scalability.md).
How you select partitioning keys substantially affects the performance of queries to your column-oriented tables. Learn more in [{#T}](../../../dba/primary-key/column-oriented.md).

To manage data partitioning, use the `AUTO_PARTITIONING_MIN_PARTITIONS_COUNT` additional parameter. The system ignores other partitioning parameters for column-oriented tables.

Expand Down
2 changes: 0 additions & 2 deletions ydb/docs/en/core/concepts/datatypes.md

This file was deleted.

1 change: 0 additions & 1 deletion ydb/docs/en/core/concepts/toc_i.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ items:
href: auth.md
- name: Data model and schema
include: { path: datamodel/toc_p.yaml, mode: link }
- { name: Data types, href: datatypes.md, hidden: true } # Deprecated
- { name: Transactions, href: transactions.md }
- { name: Secondary indexes, href: secondary_indexes.md }
- { name: Change Data Capture (CDC), href: cdc.md, when: feature_changefeed }
Expand Down
2 changes: 0 additions & 2 deletions ydb/docs/en/core/db/_includes/cloud_remark.md

This file was deleted.

8 changes: 0 additions & 8 deletions ydb/docs/en/core/db/index.md

This file was deleted.

8 changes: 0 additions & 8 deletions ydb/docs/en/core/db/toc_i.yaml

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ CDC is represented as a data schema object: a changefeed that can be added to a

## Reading data from a topic {#read}

You can read data using an [SDK](../reference/ydb-sdk/) or the [{{ ydb-short-name }} CLI](../reference/ydb-cli/). As with any other data schema object, you can access a changefeed using its path that has the following format:
You can read data using an [SDK](../reference/ydb-sdk/index.md) or the [{{ ydb-short-name }} CLI](../reference/ydb-cli/index.md). As with any other data schema object, you can access a changefeed using its path that has the following format:

```txt
path/to/table/changefeed_name
Expand Down
14 changes: 14 additions & 0 deletions ydb/docs/en/core/dba/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# {{ ydb-short-name }} for Database Administrators (DBA)

This section of {{ ydb-short-name }} documentation covers everything you need to know to properly manage contents of {{ ydb-short-name}} databases: [tables](../concepts/datamodel/table.md), [topics](../concepts/topic.md), and other entities.

The creation and management of databases are out of the scope of this section because in {{ ydb-short-name }} clusters these operations are either [executed by DevOps Engineers](../devops/index.md) or automated externally if you are using a managed service.

- [Backup and recovery](backup-and-recovery.md)
- Choosing a primary key for:
- [Row-oriented tables](primary-key/row-oriented.md)
- [Column-oriented tables](primary-key/column-oriented.md)
- [Secondary indices](secondary-indexes.md)
- [Change Data Capture](cdc.md)
- [Database system views](system-views.md)
- [Custom attributes](custom-attributes.md)
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ Column-oriented tables do not support automatic repartitioning at the moment. Th

Example:

When your data stream is 1 GB per second, an analytical table with 1,000 partitions is an optimal choice. Nevertheless, it is not advisable to create tables with an excessive number of partitions: this could raise resource consumption in the cluster and negatively impact the query rate.
When your data stream is 1 GB per second, an analytical table with 1,000 partitions is an optimal choice. Nevertheless, it is not advisable to create tables with an excessive number of partitions: this could raise resource consumption in the cluster and negatively impact the query rate.
6 changes: 6 additions & 0 deletions ydb/docs/en/core/dba/primary-key/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Choosing a primary key

The recommendations for choosing a proper primary key for a table depend on its type:

- [Row-oriented tables](row-oriented.md)
- [Column-oriented tables](column-oriented.md)
Original file line number Diff line number Diff line change
Expand Up @@ -44,4 +44,4 @@ In {{ ydb-short-name }}, all columns, including key ones, may contain a NULL val

## Row size limit {#limit-string}

To achieve high performance, we don't recommend writing rows larger than 8 MB and key columns larger than 2 KB to the DB.
To achieve high performance, we don't recommend writing rows larger than 8 MB and key columns larger than 2 KB to the DB.
7 changes: 7 additions & 0 deletions ydb/docs/en/core/dba/primary-key/toc_p.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
items:
- name: Overview
href: index.md
- name: Row-oriented
href: row-oriented.md
- name: Column-oriented
href: column-oriented.md
Loading