Consider having a waiting interface to satisfy to enable consistant waiting in most resources

[remyleone]

Usually, you have to wait for a resource to reach a certain state before being considered stable. Could it be possible to have a way to define in terraform framework an interface that would enable terraform to know when it is waiting for a resource to be available?

Anyhow it would be useful to have a consistent pattern to wait in a standard fashion.

[bflad]

Hi @remyleone 👋 Thank you for raising this feature request. If you have ideas/proposals around this area, it would be great to see them.

There are potentially (at least) two ways to interpret this request, which it would be helpful to get clarification on. Are you referring to one or both of these?

• A way for the Terraform plan/apply user interface output to show intermediate status updates
• A way for Terraform to better handle changes involving multiple underlying operations

Terraform core and the protocol does not provide any mechanisms to support either of these concepts natively at the moment. The protocol is currently request-response oriented and binary in whether the request was successful or not (e.g. error diagnostics). There is no way for providers to influence the UI during a request at the moment. Core also does not take into consideration whether an operation may have involved multiple underlying steps in the provider. It is dependent on the provider side (e.g. the SDK or provider code itself) to handle any nuance and respond accordingly into the protocol as its designed today.

The first concept has been historically tracked in these issues: hashicorp/terraform-plugin-sdk#109 / hashicorp/terraform-plugin-sdk#145 / hashicorp/terraform#17267

Designing functionality around this must happen with changes to core and protocol, since neither support the concept of streaming with provider operations beyond the plugin-level stdout receiver for logs. Since Terraform acts as a client to the provider as a server, there's no way to push this information in the current RPCs, so its very likely that new RPCs would need to be designed for this purpose.

The second concept has been historically tracked in a few issues. One of those issues being tracked with hashicorp/terraform-plugin-sdk#330 is where an ApplyResourceChange operation to create a resource may have reached a point where it can successfully be refreshed, but since an error diagnostic is returned while waiting for completion (or something similar), core will automatically mark the resource as tainted. Similarly, a historical tracking issue hashicorp/terraform-plugin-sdk#476 goes into details about when ApplyResourceChange operation to update a resource may involve multiple underlying calls and therefore can enter into a half-state that may or may not be appropriately handled by the provider when an error is raised.

Designing functionality around this, whether in concert with changes to core and the protocol, do however require very careful consideration. For example, to properly enable providers to opt out of taint on creation requires some major overhaul in the core logic for how provisioners are handled. Provisioner execution is directly tied to their associated resource instance execution. Supporting a resource state which denotes that its provisioners are still yet to run requires new nodes in the graph and state storage for them. If functionality for this is to be designed in the framework, it will need to account for being generic and handling various edge cases and nuances associated with the way the ecosystem of providers and their APIs are designed. As we discovered over time with the prior SDK, a less than ideal codified abstraction can be worse than providing general design guidance.

Hopefully some of the additional context helps here.

[remyleone]

Thanks for the very detailed answer :) I was thinking about the first interpretation but it could apply to the second interpretation as well. My main goal was to be very explicit about whether or not the plugin is running or blocking because of its wait on external resources. Because it is very oriented towards request/response it is not easy to signal when we start waiting for an operation and when it is over. Maybe having a way in the SDK to signal that we are currently waiting could be helpful to tackle other operations in the meantime.

I understand that it could mean some massive refactoring. In the meantime, it could be useful to have in the documentation an additional example that depicts the system currently waiting. In particular, it could be very helpful to see how as an SDK writer I can use the context to see if I should cancel a wait operation because it already timed out. Ideally, I would like to see also how a graceful timeout could be signaled to the terraform core.

[remyleone]

In the meantime I think there is an issue with the documentation presented here:

https://developer.hashicorp.com/terraform/plugin/framework/resources/timeouts#accessing-timeouts-in-crud-functions

it says to use a timeouts.Create but when I look into the package:

https://pkg.go.dev/github.com/hashicorp/[email protected]/resource/timeouts

I don't find anything that matches this function signature.

[bendbennett]

Hi @remyleone 👋

I believe the documentation was fixed in Document Relationships Between Terraform Commands, Protocol RPCs, and Framework Functionality which will be released as part of the changes for v1.2.0.

[oferca]

Hey @remyleone,
I actually wrote a plugin for this exact purpose - github.com/oferca/tf

Hope you enjoy

[vipinvkmenon]

Nice feature