Introduce configuration version 2 #208
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…bump" This reverts commit 7f5d41b.
plcplc
commented
Dec 11, 2023
plcplc
force-pushed
the
plc/issues/NDAT-1065
branch
from
94c84f1 to
26cee2e
Compare
soupi
reviewed
Dec 12, 2023
danieljharvey
approved these changes
Dec 12, 2023
Co-authored-by: Daniel Harvey <[email protected]>
Co-authored-by: Daniel Harvey <[email protected]>
…tgres versions?" This reverts commit 0796fb9.
plcplc
commented
Dec 12, 2023
| @@ -15,7 +15,7 @@ | |||
| }, | |||
| "ndc-postgres-12": { | |||
| "feature": "postgres", | |||
| "filter": "not test(configure_initial_configuration_is_unchanged) and not test(configure_is_idempotent)", | |||
| "filter": "not test(postgres_current_only)", | |||
There was a problem hiding this comment.
@danieljharvey : How does this look to you?
There was a problem hiding this comment.
Looks great - naming it like this explains why we're skipping too which is nice.
| @@ -1,32 +1,46 @@ | |||
| //! Tests that configuration generation has not changed. | |||
| //! | |||
| //! If you have changed it intentionally, run `just generate-chinook-configuration`. | |||
| //! | |||
There was a problem hiding this comment.
👍 Yeah this should stop confusion in future, nice!
github-merge-queue
bot
removed this pull request from the merge queue due to no response for status checks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
This PR adds a new version (
"2") of the deployment configuration data format.This version of the configuration is capable of expressing array types in collections and arguments.
Since this is the first time a new version is introduced there are a lot of changes the only purpose of which is to distinguish between versions.
Only the infrastructure-related shell of the connector is aware of different versions of deployment configurations existing. The core of the connector only works with a single internal version.
This PR is also the one to introduce tests of array types. In hindsight this ought to have been possible in the previous PR that introduced the internal types and transformations (#191).
Note that there is not yet any automated way to upgrade a configuration to a newer version, but this will be introduced shortly.
This PR also adds a changelog entry.
How
The file
version2.rsis a duplicate ofversion1.rs, which has been adapted to use the new data types (incidentally these are just the ones of the internal model).configuration.sqlnow exists asversion1.sqlandversion2.sqlrespectively, since these have different capabilities. This is becauseversion2.sqlintroduces the ability to introspect array types.configuration.rsnow exposesRawConfigurationandConfigurationtypes which are enums of all the supported versions (currently 1 and "2").One big wart on the implementation is that serde and schemars are unable to derive trait implementations for these types correctly, since they only support strings as enum tags, and we used a number literal for version 1.
Once we drop support of version 1 completely we can remove the manually implemented instances.
The various
Connectortrait implementations now explicitly work on the internal representation of a configuration,RuntimeConfiguration.