Skip to content

Commit a04691a

Browse files
anton-bobkovanton-bobkov
authored
Importing from PG example update (#15081)
1 parent 4880f2d commit a04691a

File tree

5 files changed

+79
-29
lines changed

5 files changed

+79
-29
lines changed

‎ydb/docs/en/core/postgresql/docker-connect.md

+2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Connecting via PostgreSQL Protocol
22

3+
{% include [./_includes/alert.md](./_includes/alert_preview.md) %}
4+
35
## Running {{ ydb-short-name }} with PostgreSQL compatibility enabled
46

57
Currently, the PostgreSQL compatibility feature is available for testing in the Docker image: `ghcr.io/ydb-platform/local-ydb:nightly`.

‎ydb/docs/en/core/postgresql/functions.md

+2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# PostgreSQL functions
22

3+
{% include [./_includes/alert.md](./_includes/alert_preview.md) %}
4+
35
This section contains PostgreSQL functions supported in the {{ ydb-short-name }} mode of compatibility with PostgreSQL. The original structure of the PostgreSQL documentation and examples of function application are preserved in the section. This article is automatically supplemented and tested.
46

57
{% include [functions](./_includes/functions.md) %}

‎ydb/docs/en/core/postgresql/import.md

+42-20
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Dump data form PostgreSQL
22

3+
{% include [./_includes/alert.md](./_includes/alert_preview.md) %}
4+
35
Data from PostgreSQL can be migrated to {{ ydb-short-name }} using utilities such as [pg_dump](https://www.postgresql.org/docs/current/app-pgdump.html), [psql](https://www.postgresql.org/docs/current/app-psql.html), and [{{ ydb-short-name }} CLI](../reference/ydb-cli/index.md). The [pg_dump](https://www.postgresql.org/docs/current/app-pgdump.html) and [psql](https://www.postgresql.org/docs/current/app-psql.html) utilities are installed with PostgreSQL. [{{ ydb-short-name }} CLI](../reference/ydb-cli/index.md) is {{ ydb-short-name }}'s command-line client, which is [installed separately](../reference/ydb-cli/install.md).
46

57
To do this, you need to:
@@ -26,8 +28,8 @@ The `ydb tools pg-convert` command reads a dump file or standard input created b
2628
* Deleting the `WITH (...)` section in `CREATE TABLE`.
2729
* Commenting out unsupported constructs (optionally):
2830

29-
* `SELECT pg_catalog.set_config.*`
30-
* `ALTER TABLE`
31+
* `SELECT pg_catalog.set_config.*`
32+
* `ALTER TABLE`
3133

3234
If the CLI cannot find a table's primary key, it will automatically create a [BIGSERIAL](https://www.postgresql.org/docs/current/datatype-numeric.html#DATATYPE-SERIAL) column named `__ydb_stub_id` as the primary key.
3335

@@ -60,26 +62,46 @@ As an example, data generated by [pgbench](https://www.postgresql.org/docs/curre
6062

6163
1. Start Docker containers with PostgreSQL and {{ ydb-short-name }}:
6264

63-
```bash
64-
docker run -d --rm -e POSTGRES_USER=root -e POSTGRES_PASSWORD=1234 \
65-
-e POSTGRES_DB=local --name postgres postgres:14
66-
docker run --name ydb-postgres -d --pull always -p 5432:5432 -p 8765:8765 \
67-
-e POSTGRES_USER=root -e POSTGRES_PASSWORD=1234 \
68-
-e YDB_FEATURE_FLAGS=enable_temp_tables \
69-
-e YDB_USE_IN_MEMORY_PDISKS=true \
70-
ghcr.io/ydb-platform/ydb-local:latest
71-
```
65+
```bash
66+
docker run --name postgres_container \
67+
-e POSTGRES_USER=pgroot -e POSTGRES_PASSWORD=1234 \
68+
-e POSTGRES_DB=local \
69+
-p 5433:5433 -d postgres:14 -c 'port=5433'
70+
docker run --name ydb-postgres -d --pull always -p 5432:5432 -p 8765:8765 \
71+
-e POSTGRES_USER=ydbroot -e POSTGRES_PASSWORD=4321 \
72+
-e YDB_FEATURE_FLAGS=enable_temp_tables,enable_table_pg_types \
73+
-e YDB_USE_IN_MEMORY_PDISKS=true \
74+
ghcr.io/ydb-platform/local-ydb:latest
75+
```
76+
77+
Information about the started Docker containers:
78+
79+
#|
80+
|| Database | PostgreSQL | YDB ||
81+
|| Container name | postgres_container | ydb-postgres ||
82+
|| Address | postgres://pgroot:1234@localhost:5433/local | postgresql://ydbroot:4321@localhost:5432/local ||
83+
|| Port | 5433 | 5432 ||
84+
|| User name | pgroot | ydbroot ||
85+
|| Password | 1234 | 4321 ||
86+
|#
7287

7388
2. Generate data through [pgbench](https://www.postgresql.org/docs/current/pgbench.html):
7489

75-
```bash
76-
docker exec postgres pgbench -i postgres://root:1234@localhost/local
77-
```
90+
```bash
91+
docker exec postgres_container pgbench postgres://pgroot:1234@localhost:5433/local -i
92+
```
93+
94+
3. Create a dump of the database using [pg_dump](https://www.postgresql.org/docs/current/app-pgdump.html):
95+
96+
```bash
97+
docker exec postgres_container pg_dump postgres://pgroot:1234@localhost:5433/local --inserts \
98+
--column-inserts --encoding=utf_8 --rows-per-insert=1000 > dump.sql
99+
```
100+
101+
4. Load the dump into {{ ydb-short-name }}:
78102

79-
3. Create a dump of the database using [pg_dump](https://www.postgresql.org/docs/current/app-pgdump.html) and load it into {{ ydb-short-name }}:
103+
```bash
104+
ydb tools pg-convert --ignore-unsupported -i dump.sql | psql postgresql://ydbroot:4321@localhost:5432/local
105+
```
80106

81-
```bash
82-
docker exec postgres pg_dump -U root --format=c --file=/var/lib/postgresql/data/dump.sql local
83-
docker cp postgres:/var/lib/postgresql/data/dump.sql .
84-
cat dump.sql | docker exec -i ydb-postgres psql -U root -d local
85-
```
107+
This command uses {{ ydb-short-name }} CLI to convert the dump.sql file to the format readable by the {{ ydb-short-name }} PostgreSQL compatibility layer. The converted dump file is then redirected to the `psql` utility for loading the data into {{ ydb-short-name }} via PostgreSQL protocol.

‎ydb/docs/ru/core/postgresql/functions.md

+2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# PostgreSQL функции
22

3+
{% include [./_includes/alert.md](./_includes/alert_preview.md) %}
4+
35
В данном разделе содержатся PostgreSQL функции, поддерживаемые в режиме совместимости {{ ydb-short-name }} с PostgreSQL. В разделе сохранены оригинальная структура документации PostgreSQL и примеры применения функций. Данная статья автоматически пополняется и тестируется, поэтому текст представлен только на английском языке.
46

57
{% include [functions](./_includes/functions.md) %}

‎ydb/docs/ru/core/postgresql/import.md

+31-9
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Импорт данных из PostgreSQL
22

3+
{% include [./_includes/alert.md](./_includes/alert_preview.md) %}
4+
35
Данные из PostgreSQL в {{ ydb-name }} можно импортировать различными способами:
46

57
- С помощью [pg-dump](#pg-dump).
@@ -78,25 +80,45 @@
7880
1. Поднять докер-контейнеры с PostgreSQL и {{ ydb-short-name }}:
7981

8082
```bash
81-
docker run -d --rm -e POSTGRES_USER=root -e POSTGRES_PASSWORD=1234 \
82-
-e POSTGRES_DB=local --name postgres postgres:14
83+
docker run --name postgres_container \
84+
-e POSTGRES_USER=pgroot -e POSTGRES_PASSWORD=1234 \
85+
-e POSTGRES_DB=local \
86+
-p 5433:5433 -d postgres:14 -c 'port=5433'
8387
docker run --name ydb-postgres -d --pull always -p 5432:5432 -p 8765:8765 \
84-
-e POSTGRES_USER=root -e POSTGRES_PASSWORD=1234 \
85-
-e YDB_FEATURE_FLAGS=enable_temp_tables \
88+
-e POSTGRES_USER=ydbroot -e POSTGRES_PASSWORD=4321 \
89+
-e YDB_FEATURE_FLAGS=enable_temp_tables,enable_table_pg_types \
8690
-e YDB_USE_IN_MEMORY_PDISKS=true \
87-
ghcr.io/ydb-platform/local-ydb:nightly
91+
ghcr.io/ydb-platform/local-ydb:latest
8892
```
8993

94+
Информация по поднимаемым docker-контейнерам:
95+
96+
#|
97+
|| База данных | PostgreSQL | YDB ||
98+
|| Имя контейнера | postgres_container | ydb-postgres ||
99+
|| Адрес | postgres://pgroot:1234@localhost:5433/local | postgresql://ydbroot:4321@localhost:5432/local ||
100+
|| Порт | 5433 | 5432 ||
101+
|| Имя пользователя | pgroot | ydbroot ||
102+
|| Пароль | 1234 | 4321 ||
103+
|#
104+
90105
2. Сгенерировать данные через [pgbench](https://www.postgresql.org/docs/current/pgbench.html):
91106

92107
```bash
93-
docker exec postgres pgbench postgres://root:1234@localhost/local -i
108+
docker exec postgres_container pgbench postgres://pgroot:1234@localhost:5433/local -i
94109
```
95110

96-
3. Сделать дамп базы через [pg_dump](https://www.postgresql.org/docs/current/app-pgdump.html) и загрузить его в {{ ydb-short-name }}:
111+
3. Сделать дамп базы через [pg_dump](https://www.postgresql.org/docs/current/app-pgdump.html):
97112

98113
```bash
99-
docker exec postgres pg_dump postgres://root:1234@localhost/local --inserts \
114+
docker exec postgres_container pg_dump postgres://pgroot:1234@localhost:5433/local --inserts \
100115
--column-inserts --encoding=utf_8 --rows-per-insert=1000 > dump.sql
101-
ydb tools pg-convert -i dump.sql | psql postgresql://root:1234@localhost/local
102116
```
117+
118+
4. Загрузить дамп базы в {{ ydb-short-name }}:
119+
120+
```bash
121+
ydb tools pg-convert --ignore-unsupported -i dump.sql | psql postgresql://ydbroot:4321@localhost:5432/local
122+
```
123+
124+
Эта команда использует {{ ydb-short-name }} CLI для преобразования файла `dump.sql` в формат, поддерживаемый {{ ydb-short-name }} в режиме совместимости с PostgreSQL. Затем преобразованный файл перенаправляется в утилиту `psql` для загрузки данных в {{ ydb-short-name }} по протоколу PostgreSQL.

0 commit comments

Comments
 (0)