Skip to content
This repository was archived by the owner on May 14, 2025. It is now read-only.

Update database section of ref guide #5082

Merged
merged 1 commit into from
Sep 10, 2022
Merged

Update database section of ref guide #5082

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions spring-cloud-dataflow-docs/src/main/asciidoc/configuration-database.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@

[[configuration-database-overview]]
A relational database is used to store stream and task definitions as well as the state of executed tasks.
Spring Cloud Data Flow provides schemas for *MariaDB*, *MySQL*, *Oracle*, *PostgreSQL*, *Db2*, *SQL Server*, and *H2*. The schema is automatically created when the server starts.

NOTE: The JDBC drivers for *MariaDB*, *MySQL* (via the _MariaDB_ driver), *PostgreSQL*, *SQL Server* are available without additional configuration. To use any other database you need to put the corresponding JDBC driver jar on the classpath of the server as described <<#add-custom-driver,here>>.

To configure a database the following properties must be set:

* `spring.datasource.url`
* `spring.datasource.username`
* `spring.datasource.password`
* `spring.datasource.driver-class-name`

The `username` and `password` are the same regardless of the database. However, the `url` and `driver-class-name` vary per database as follows.

[frame="none"]
[cols="a,a,a,a"]
[cols="10%,30%,20%,10%"]
|===
|[.small]#Database#|[.small]#spring.datasource.url#|[.small]#spring.datasource.driver-class-name#|[.small]#Driver included#

|[.small]#MariaDB 10.4+#
|[.small]#jdbc:mariadb://${db-hostname}:${db-port}/${db-name}#
|[.small]#org.mariadb.jdbc.Driver#
|[.small]#Yes#

|[.small]#MySQL 5.7#
|[.small]#jdbc:mysql://${db-hostname}:${db-port}/${db-name}?permitMysqlScheme#
|[.small]#org.mariadb.jdbc.Driver#
|[.small]#Yes#

|[.small]#MySQL 8.0+#
|[.small]#jdbc:mariadb://${db-hostname}:${db-port}/${db-name}?allowPublicKeyRetrieval=true&useSSL=false&autoReconnect=true&permitMysqlScheme#{empty}footnote:[SSL is disabled in this example, adjust accordingly for your environment and requirements]
|[.small]#org.mariadb.jdbc.Driver#
|[.small]#Yes#

|[.small]#PostgresSQL#
|[.small]#jdbc:postgres://${db-hostname}:${db-port}/${db-name}#
|[.small]#org.postgresql.Driver#
|[.small]#Yes#

|[.small]#SQL Server#
|[.small]#jdbc:sqlserver://${db-hostname}:${db-port};databasename=${db-name}&encrypt=false#
|[.small]#com.microsoft.sqlserver.jdbc.SQLServerDriver#
|[.small]#Yes#

|[.small]#DB2#
|[.small]#jdbc:db2://${db-hostname}:${db-port}/{db-name}#
|[.small]#com.ibm.db2.jcc.DB2Driver#
|[.small]#No#

|[.small]#Oracle#
|[.small]#jdbc:oracle:thin:@${db-hostname}:${db-port}/{db-name}#
|[.small]#oracle.jdbc.OracleDriver#
|[.small]#No#
|===

==== H2
When no other database is configured and the *H2* driver has been added to the server classpath then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noob question, Is there a case where we do not include the H2 driver? If they don't specify a datasource dataflow defaults to H2.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So....

The H2 config was split into its own config and made optional in #4802 as it was lighting up CVE scans.

We quickly realized the missing dep. broke running locally w/ java -jar as well as many examples that do the same in the docs. Because of this I made the dependency non-optional here.

As of now the dependency is still provided. We need to decide what we are going to do in this area (provide it or not). This doc is written as if it will be removed. Lets leave it like that for now and we will re-address the docs when we make a decision on H2.

Sound good @cppwfs ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to merge this and we can follow on w/ whatever we need to.

Spring Cloud Data Flow uses an embedded instance of the *H2* database as the default.

NOTE: *H2* is good for development purposes but is not recommended for production use nor is it supported as an external mode.

To use *H2* add the `com.h2database:h2:2.1.214` JDBC driver to the classpath and do not configure any other database.
15 changes: 5 additions & 10 deletions spring-cloud-dataflow-docs/src/main/asciidoc/configuration-kubernetes.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -421,14 +421,13 @@ data:
The password is a base64-encoded value.

[[configuration-kubernetes-rdbms]]
=== Database Configuration
=== Database

Spring Cloud Data Flow provides schemas for H2, MariaDB, Oracle, PostgreSQL, DB2, and SQL Server. The appropriate schema is automatically created when the server starts, provided the right database driver and appropriate credentials are in the classpath.
include::configuration-database.adoc[]

The JDBC drivers for MariaDB, PostgreSQL, SQL Server, and embedded H2 are available out of the box.
If you use any other database, you need to put the corresponding JDBC driver jar on the classpath of the server.
==== Database configuration

For instance, if you use MariaDB in addition to a password in the secrets file, you could provide the following properties in the ConfigMap:
When running in Kubernetes, the database properties are typically set in the ConfigMap. For instance, if you use MariaDB in addition to a password in the secrets file, you could provide the following properties in the ConfigMap:


[source,yaml]
Expand All @@ -441,13 +440,9 @@ data:
username: root
password: ${mariadb-root-password}
driverClassName: org.mariadb.jdbc.Driver
url: jdbc:mariadb://${MARIADB_SERVICE_HOST}:${MARIADB_SERVICE_PORT}/test
driverClassName: org.mariadb.jdbc.Driver
----


For PostgreSQL, you could use the following configuration:

Similarly, for PostgreSQL you could use the following configuration:

[source,yaml]
----
Expand Down
86 changes: 13 additions & 73 deletions spring-cloud-dataflow-docs/src/main/asciidoc/configuration-local.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,91 +29,31 @@ The REST `/about` endpoint provides information on the features that have been e
[[configuration-local-rdbms]]
=== Database

A relational database is used to store stream and task definitions as well as the state of executed tasks.
Spring Cloud Data Flow provides schemas for *H2*, *MariaDB*, *Oracle*, *PostgreSQL*, *Db2*, and *SQL Server*. The schema is automatically created when the server starts.
include::configuration-database.adoc[]

By default, Spring Cloud Data Flow offers an embedded instance of the *H2* database (if the H2 JDBC driver jar is found on the classpath).
The *H2* database is good for development purposes but is not recommended for production use.

NOTE: *H2* database is not supported as an external mode.

The JDBC drivers for *MariaDB*, *PostgreSQL*, *SQL Server* are available without additional configuration.

If you are using the embedded *H2* or any other database, then you need to put the corresponding JDBC driver jar on the classpath of the server.

The database properties can be passed as environment variables or command-line arguments to the Data Flow Server.

==== MariaDB

The following example shows how to define a MariaDB database connection with command Line arguments
==== Database configuration

When running locally, the database properties can be passed as environment variables or command-line arguments to the Data Flow Server. For example, to start the server with MariaDB using command line arguments execute the following command:
[source,bash,subs=attributes]
----
java -jar spring-cloud-dataflow-server/target/spring-cloud-dataflow-server-{project-version}.jar \
--spring.datasource.url=jdbc:mariadb://localhost:3306/mydb \
--spring.datasource.username=<user> \
--spring.datasource.password=<password> \
--spring.datasource.username=user \
--spring.datasource.password=pass \
--spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
----

==== PostgreSQL

The following example shows how to define a PostgreSQL database connection with command line arguments:

[source,bash,subs=attributes]
----
java -jar spring-cloud-dataflow-server/target/spring-cloud-dataflow-server-{project-version}.jar \
--spring.datasource.url=jdbc:postgresql://localhost:5432/mydb \
--spring.datasource.username=<user> \
--spring.datasource.password=<password> \
--spring.datasource.driver-class-name=org.postgresql.Driver
----

==== SQL Server

The following example shows how to define a SQL Server database connection with command line arguments:

[source,bash,subs=attributes]
----
java -jar spring-cloud-dataflow-server/target/spring-cloud-dataflow-server-{project-version}.jar \
--spring.datasource.url='jdbc:sqlserver://localhost:1433;databaseName=mydb&encrypt=false' \
--spring.datasource.username=<user> \
--spring.datasource.password=<password> \
--spring.datasource.driver-class-name=com.microsoft.sqlserver.jdbc.SQLServerDriver
----

==== Db2

The following example shows how to define a Db2 database connection with command line arguments:

Likewise, to start the server with MariaDB using environment variables execute the following command:
[source,bash,subs=attributes]
----
java -jar spring-cloud-dataflow-server/target/spring-cloud-dataflow-server-{project-version}.jar \
--spring.datasource.url=jdbc:db2://localhost:50000/mydb \
--spring.datasource.username=<user> \
--spring.datasource.password=<password> \
--spring.datasource.driver-class-name=com.ibm.db2.jcc.DB2Driver
----

NOTE: Due to licensing restrictions we're unable to bundle Db2 driver. You need to add it to
server's classpath yourself.

==== Oracle

The following example shows how to define a Oracle database connection with command line arguments:
SPRING_DATASOURCE_URL=jdbc:mariadb://localhost:3306/mydb
SPRING_DATASOURCE_USERNAME=user
SPRING_DATASOURCE_PASSWORD=pass
SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.mariadb.jdbc.Driver

[source,bash,subs=attributes]
----
java -jar spring-cloud-dataflow-server/target/spring-cloud-dataflow-server-{project-version}.jar \
--spring.datasource.url=jdbc:oracle:thin:@localhost:1521/MYDB \
--spring.datasource.username=<user> \
--spring.datasource.password=<password> \
--spring.datasource.driver-class-name=oracle.jdbc.OracleDriver
java -jar spring-cloud-dataflow-server/target/spring-cloud-dataflow-server-{project-version}.jar
----

NOTE: Due to licensing restrictions we're unable to bundle Oracle driver. You need to add it to
server's classpath yourself.

[#add-custom-driver]
==== Adding a Custom JDBC Driver
To add a custom driver for the database (for example, Oracle), you should rebuild the Data Flow Server and add the dependency to the Maven `pom.xml` file.
You need to modify the maven `pom.xml` of `spring-cloud-dataflow-server` module.
Expand All @@ -124,7 +64,7 @@ To add a custom JDBC driver dependency for the Spring Cloud Data Flow server:
. Select the tag that corresponds to the version of the server you want to rebuild and clone the github repository.
. Edit the spring-cloud-dataflow-server/pom.xml and, in the `dependencies` section, add the dependency for the database driver required. In the following example , an Oracle driver has been chosen:

[source, xml]
[source,xml]
----
<dependencies>
...
Expand Down