|
1 | | -{% include [no-translation](../../_includes/alerts/no-translation.md) %} |
| 1 | +# Working with ClickHouse databases |
| 2 | + |
| 3 | +This section describes the basic information about working with the external ClickHouse database [ClickHouse](https://clickhouse.com). |
| 4 | + |
| 5 | +To work with the external ClickHouse database, the following steps must be completed: |
| 6 | +1. Create a [secret](../datamodel/secrets.md) containing the password to connect to the database. |
| 7 | + ```sql |
| 8 | + CREATE OBJECT clickhouse_datasource_user_password (TYPE SECRET) WITH (value = "<password>"); |
| 9 | + ``` |
| 10 | +1. Create an [external data source](../datamodel/external_data_source.md) describing the target database inside the ClickHouse cluster. To connect to ClickHouse, you can use either the [native TCP protocol](https://clickhouse.com/docs/en/interfaces/tcp) (`PROTOCOL="NATIVE"`) or the [HTTP protocol](https://clickhouse.com/docs/en/interfaces/http) (`PROTOCOL="HTTP"`). To enable encryption for connections to the external database, use the `USE_TLS="TRUE"` parameter. |
| 11 | + ```sql |
| 12 | + CREATE EXTERNAL DATA SOURCE clickhouse_datasource WITH ( |
| 13 | + SOURCE_TYPE="ClickHouse", |
| 14 | + LOCATION="<host>:<port>", |
| 15 | + DATABASE_NAME="<database>", |
| 16 | + AUTH_METHOD="BASIC", |
| 17 | + LOGIN="<login>", |
| 18 | + PASSWORD_SECRET_NAME="clickhouse_datasource_user_password", |
| 19 | + PROTOCOL="NATIVE", |
| 20 | + USE_TLS="TRUE" |
| 21 | + ); |
| 22 | + ``` |
| 23 | + |
| 24 | +1. {% include [!](_includes/connector_deployment.md) %} |
| 25 | +1. [Execute a query](#query) to the database. |
| 26 | + |
| 27 | + |
| 28 | +## Query syntax {#query} |
| 29 | +To work with ClickHouse, use the following SQL query form: |
| 30 | + |
| 31 | +```sql |
| 32 | +SELECT * FROM clickhouse_datasource.<table_name> |
| 33 | +``` |
| 34 | + |
| 35 | +Where: |
| 36 | +- `clickhouse_datasource` is the identifier of the external data source; |
| 37 | +- `<table_name>` is the table's name within the external data source. |
| 38 | +
|
| 39 | +## Limitations |
| 40 | +
|
| 41 | +There are several limitations when working with ClickHouse clusters: |
| 42 | +
|
| 43 | +1. {% include [!](_includes/supported_requests.md) %} |
| 44 | +1. {% include [!](_includes/datetime_limits.md) %} |
| 45 | +1. {% include [!](_includes/predicate_pushdown.md) %} |
| 46 | +
|
| 47 | +## Supported data types |
| 48 | +
|
| 49 | +By default, ClickHouse columns cannot physically contain `NULL` values. However, users can create tables with columns of optional or [nullable](https://clickhouse.com/docs/en/sql-reference/data-types/nullable) types. The column types displayed in {{ ydb-short-name }} when extracting data from the external ClickHouse database will depend on whether primitive or optional types are used in the ClickHouse table. Due to the previously discussed limitations of {{ ydb-short-name }} types used to store dates and times, all similar ClickHouse types are displayed in {{ ydb-short-name }} as [optional](../../yql/reference/types/optional.md). |
| 50 | +
|
| 51 | +Below are the mapping tables for ClickHouse and {{ ydb-short-name }} types. All other data types, except those listed, are not supported. |
| 52 | +
|
| 53 | +### Primitive data types |
| 54 | +
|
| 55 | +|ClickHouse data type|{{ ydb-full-name }} data type|Notes| |
| 56 | +|---|----|------| |
| 57 | +|`Bool`|`Bool`|| |
| 58 | +|`Int8`|`Int8`|| |
| 59 | +|`UInt8`|`Uint8`|| |
| 60 | +|`Int16`|`Int16`|| |
| 61 | +|`UInt16`|`Uint16`|| |
| 62 | +|`Int32`|`Int32`|| |
| 63 | +|`UInt32`|`Uint32`|| |
| 64 | +|`Int64`|`Int64`|| |
| 65 | +|`UInt64`|`Uint64`|| |
| 66 | +|`Float32`|`Float`|| |
| 67 | +|`Float64`|`Double`|| |
| 68 | +|`Date`|`Date`|| |
| 69 | +|`Date32`|`Optional<Date>`|Valid date range from 1970-01-01 to 2105-12-31. Values outside this range return `NULL`.| |
| 70 | +|`DateTime`|`Optional<DateTime>`|Valid time range from 1970-01-01 00:00:00 to 2105-12-31 23:59:59. Values outside this range return `NULL`.| |
| 71 | +|`DateTime64`|`Optional<Timestamp>`|Valid time range from 1970-01-01 00:00:00 to 2105-12-31 23:59:59. Values outside this range return `NULL`.| |
| 72 | +|`String`|`String`|| |
| 73 | +|`FixedString`|`String`|Null bytes in `FixedString` are transferred to `String` unchanged.| |
| 74 | +
|
| 75 | +### Optional data types |
| 76 | +
|
| 77 | +|ClickHouse data type|{{ ydb-full-name }} data type|Notes| |
| 78 | +|---|----|------| |
| 79 | +|`Nullable(Bool)`|`Optional<Bool>`|| |
| 80 | +|`Nullable(Int8)`|`Optional<Int8>`|| |
| 81 | +|`Nullable(UInt8)`|`Optional<Uint8>`|| |
| 82 | +|`Nullable(Int16)`|`Optional<Int16>`|| |
| 83 | +|`Nullable(UInt16)`|`Optional<Uint16>`|| |
| 84 | +|`Nullable(Int32)`|`Optional<Int32>`|| |
| 85 | +|`Nullable(UInt32)`|`Optional<Uint32>`|| |
| 86 | +|`Nullable(Int64)`|`Optional<Int64>`|| |
| 87 | +|`Nullable(UInt64)`|`Optional<Uint64>`|| |
| 88 | +|`Nullable(Float32)`|`Optional<Float>`|| |
| 89 | +|`Nullable(Float64)`|`Optional<Double>`|| |
| 90 | +|`Nullable(Date)`|`Optional<Date>`|| |
| 91 | +|`Nullable(Date32)`|`Optional<Date>`|Valid date range from 1970-01-01 to 2105-12-31. Values outside this range return `NULL`.| |
| 92 | +|`Nullable(DateTime)`|`Optional<DateTime>`|Valid time range from 1970-01-01 00:00:00 to 2105-12-31 23:59:59. Values outside this range return `NULL`.| |
| 93 | +|`Nullable(DateTime64)`|`Optional<Timestamp>`|Valid time range from 1970-01-01 00:00:00 to 2105-12-31 23:59:59. Values outside this range return `NULL`.| |
| 94 | +|`Nullable(String)`|`Optional<String>`|| |
| 95 | +|`Nullable(FixedString)`|`Optional<String>`|Null bytes in `FixedString` are transferred to `String` unchanged.| |
0 commit comments