docs/design: Upgrade Resource State #228

bflad · 2021-11-29T14:30:38Z

Reference: #42

bflad · 2021-11-29T14:36:07Z

Just to jot down some potential proposals so they are out of my mental state/notes:

Implementing something similar to SDKv2 with the top level schema type including version and state upgraders fields
Overloading the underlying schema types (e.g. Attribute) to include state upgrade fields such as "Removed" or "UpgradeState"
(Maybe cursed idea) Overloading the tfsdk struct tag to include versioning information

hashicorp-cla · 2022-03-12T19:13:52Z

All committers have signed the CLA.

Reference: #42

…ate implementation

…nd Block fields

docs/design/upgrade-resource-state.md

ewbankkit · 2022-04-14T18:10:25Z

docs/design/upgrade-resource-state.md

+The framework could allow provider developers to optionally extend their `ResourceType` with a new `UpgradeState` method.
+
+For example in the framework:
+
+```go
+// New type
+type ResourceTypeWithUpgradeState interface {
+    UpgradeState(context.Context, /*...*/) /*...*/
+}
+```
+
+Which would result in the following optional provider implementation:
+
+```go
+func (rt ExampleResourceType) UpgradeState(ctx context.Context, /*...*/) /*...*/ {
+    /* ... */
+}
+```
+
+However, there is an immediate drawback that there would be no access to the provider client during upgrade state operations. This is considered a non-starter due to the goals of this functionality.


Although the headings are different, this text is identical to the following section - New ResourceWithUpgradeState Interface.

Hey sorry, I'm not sure what you mean here. 😅 ResourceType and Resource are very different implementation details. Providers get the opportunity to inject their clients into Resource, but not ResourceType.

My bad 😄

docs/design/upgrade-resource-state.md

Co-authored-by: Kit Ewbank <[email protected]>

docs/design/upgrade-resource-state.md

Reference: #42 Reference: #228 Support provider defined `UpgradeResourceState` RPC handling, by introducing an optional `ResourceWithUpgradeState` interface type, with an `UpgradeState` method. Each underlying state version upgrade implementation is expected to consume the prior state, perform any necessary data manipulations, then respond with the upgraded state. This framework implementation differs from the terraform-plugin-sdk implementation: - State upgraders are specified via a mapping, rather than a slice with underlying version field. This should prevent certain classes of coding issues. - State upgraders must be wholly contained from the prior state version to the current schema version. The framework does not loop through each successive version because attempting to recreate the `tfprotov6.RawState` for each intermediate version request would be very problematic. For example, terraform-plugin-go does not implement functionality for marshalling a `RawState`. Provider developers can use their own coding techniques to reduce code duplications when multiple versions need the same logic. - Specifying the full prior schema is now an optional implementation detail. Working with the lower level data types is more challenging, however this has been a repeated feature request. There are some quirks and potential future enhancements to the framework `UpgradeResourceState` handling: - Past and current versions Terraform CLI will call `UpgradeResourceState` even if the state version matches the current schema version. This implementation keeps the framework's prior logic to roundtrip the existing state into the upgraded state. It may be possible to stop this Terraform CLI behavior with protocol version 6, although the logic would need to remain for backwards compatibility. - It may be possible to help provider developers simplify logic by attempting to automatically populate compatible parts of the upgraded state from the prior state. This can potentially be done at a later time.

bflad · 2022-04-18T18:21:16Z

Proof of concept implementation (and testing examples) for current recommendation: #292

This by no means is meant to represent that other proposals are off the table or the recommendation cannot be changed, however it hopefully gives some additional confidence in the potential implementation.

Reference: #42 Reference: #228 Support provider defined `UpgradeResourceState` RPC handling, by introducing an optional `ResourceWithUpgradeState` interface type, with an `UpgradeState` method. Each underlying state version upgrade implementation is expected to consume the prior state, perform any necessary data manipulations, then respond with the upgraded state. This framework implementation differs from the terraform-plugin-sdk implementation: - State upgraders are specified via a mapping, rather than a slice with underlying version field. This should prevent certain classes of coding issues. - State upgraders must be wholly contained from the prior state version to the current schema version. The framework does not loop through each successive version because attempting to recreate the `tfprotov6.RawState` for each intermediate version request would be very problematic. For example, terraform-plugin-go does not implement functionality for marshalling a `RawState`. Provider developers can use their own coding techniques to reduce code duplications when multiple versions need the same logic. - Specifying the full prior schema is now an optional implementation detail. Working with the lower level data types is more challenging, however this has been a repeated feature request. There are some quirks and potential future enhancements to the framework `UpgradeResourceState` handling: - Past and current versions Terraform CLI will call `UpgradeResourceState` even if the state version matches the current schema version. This implementation keeps the framework's prior logic to roundtrip the existing state into the upgraded state. It may be possible to stop this Terraform CLI behavior with protocol version 6, although the logic would need to remain for backwards compatibility. - It may be possible to help provider developers simplify logic by attempting to automatically populate compatible parts of the upgraded state from the prior state. This can potentially be done at a later time.

bookshelfdave

This was a good read, thanks for describing pros and cons for each proposal!

docs/design/upgrade-resource-state.md

Co-authored-by: Dave Parfitt <[email protected]>

bflad · 2022-04-20T19:26:15Z

If anyone has further feedback, please reach out to me directly. We can also consider different approaches before we reach 1.0.

Reference: #42 Reference: #228 Support provider defined `UpgradeResourceState` RPC handling, by introducing an optional `ResourceWithUpgradeState` interface type, with an `UpgradeState` method. Each underlying state version upgrade implementation is expected to consume the prior state, perform any necessary data manipulations, then respond with the upgraded state. This framework implementation differs from the terraform-plugin-sdk implementation: - State upgraders are specified via a mapping, rather than a slice with underlying version field. This should prevent certain classes of coding issues. - State upgraders must be wholly contained from the prior state version to the current schema version. The framework does not loop through each successive version because attempting to recreate the `tfprotov6.RawState` for each intermediate version request would be very problematic. For example, terraform-plugin-go does not implement functionality for marshalling a `RawState`. Provider developers can use their own coding techniques to reduce code duplications when multiple versions need the same logic. - Specifying the full prior schema is now an optional implementation detail. Working with the lower level data types is more challenging, however this has been a repeated feature request. There are some quirks and potential future enhancements to the framework `UpgradeResourceState` handling: - Past and current versions Terraform CLI will call `UpgradeResourceState` even if the state version matches the current schema version. This implementation keeps the framework's prior logic to roundtrip the existing state into the upgraded state. It may be possible to stop this Terraform CLI behavior with protocol version 6, although the logic would need to remain for backwards compatibility. - It may be possible to help provider developers simplify logic by attempting to automatically populate compatible parts of the upgraded state from the prior state. This can potentially be done at a later time.

github-actions · 2022-05-21T02:18:27Z

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

bflad added design-doc Issues or pull requests about a design document, capturing design ideas and trade-offs. sdkv2-parity Issues tracking feature parity with terraform-plugin-sdk v2 and PRs working towards it. labels Nov 29, 2021

bflad added this to the v0.6.0 milestone Jan 11, 2022

bookshelfdave assigned bflad Jan 24, 2022

bflad modified the milestones: v0.6.0, v0.7.0 Mar 10, 2022

bflad added 2 commits April 13, 2022 14:03

docs/design: Upgrade Resource State background and goals

512cdf7

Reference: #42

docs/design: Some additional context around current UpgradeResourceSt…

49dcbe1

…ate implementation

bflad force-pushed the bflad-docs-design-UpgradeResourceState branch from 40d5e01 to 49dcbe1 Compare April 13, 2022 18:29

docs/design: Proposals, Recommendations, slightly more background

14b254f

bflad marked this pull request as ready for review April 14, 2022 15:25

bflad requested a review from a team as a code owner April 14, 2022 15:25

bflad removed their assignment Apr 14, 2022

docs/design: UpgradeResourceState proposal for additional Attribute a…

819b902

…nd Block fields

ewbankkit reviewed Apr 14, 2022

View reviewed changes