Introduce "after" Relation in Workflows

[csviri]

Description of the Problem

Workflows now has only the "depends on" relation. What mean if a dependent resource C depends on dependent resource A and B, it will be reconciled if both A and B are reconciled and ready. So both reconcile preconditions needs to hold if any, but also ready post conditions needs to be true for A and B if present.

While this probably covers most of the cases, and semantically feels natural: If a resource depends on other resource (so probably has to do something with those, like using some values from status, or calling their API after deployed) that one needs to be reconciled and ready first. And this might cover most of the cases, but also we can identify cases when this is not sufficient:

1. A resource needs to be after others but not all, only one or just a subset. So for example A would needs to be reconciled after B or C, but only one of those needs to exists (because maybe some precondition not holds for B). So basically a logical "or". Note that a reconcile precondition (RP) for A it is possible to check whether B or C was reconciled, or with what outcome, so it's not simply an "or" but a very generic approach to give a possibility to cover all kind of scenarios.

2. But an other example, B depends on A: A <- B. But we want to have B reconciled even if a precondition for A does not hold. In other words, A is an optional resource, but if present B needs to be reconciled after.

In other words we need a more generic relation, where we can just say, no matters if the previous resource was reconciled or not (because the reconcile precondition not holds).

Definition of after

An after relationship between a resource A and other resources B1 ...Bn . Where A is reconciled or evaulated for reconciliation (like checking preconditions). If all resources B1 ... Bn were evaluated for for reconciliation and/or reconciled - and if reconciled the ready post condition holds.

Combining depends_on and after

after is not a replacement for depends_on it's rather a complementary solution. So should it be possible to describe that C depends on A, but it is after B. So A <- C ; B - C. Well semantically it would mean to reconciled C we needs to reconcile A, but also B (optionally). Of course this could be described with two after relations: A - C ; B - C and a reconcile precondition, where we check if A exists.

So basically we can say these two can be combined; unless if we discover that it makes some semantic mess,( or it's very complex implementation wise. )

Additional Samples / Cases

Consider also the following situation: A <- B - C . B depends on A; C is after B. But the precondition for A does not hold. So B would not be reconciled either, but C will.

Notes

• Should after just processed after a ready condition is true? - probably yes, it based on those cases, that would be probably reasonable still (although, will loose a little bit generality). Alternatively just take into account the reconcile precondition, not the ready conditions.
• Notation: we can use arrow (<-) for depends on and just line/minus sign for after (-) . We read left to right so left happens always first.

[metacosm]

It seems to me that all this could be handled with appropriate pre-conditions… I'm also not sure if the workflow will keep being a DAG if we introduce this kind of relation. Perhaps, it would be simpler to add an optional attribute on the dependent instead of adding a similar-but-not-quite relation which is somewhat confusing.

[csviri]

It seems to me that all this could be handled with appropriate pre-conditions

Actually I'm not sure, the problem is if a precondition not holds the resource is deleted, even if was creatred before. (That is an other issue, maybe to rename reconcile precondition simply to precondition, so it more describes that)

I'm also not sure if the workflow will keep being a DAG if we introduce this kind of relation

It is possible to create a non-dag even now, there is just a detection if it's really a dag, same could be done here.

it would be simpler to add an optional attribute

I don't think we should mix these conditions and ordering into dependent resources. I think the current scope of DR is very well defined and how it works in the standalone mode is also very useful. While the execution can be separatelly addressed by the workflows. Also this is mostly about ordering, just like depends on, but more generic.

similar-but-not-quite relation which is somewhat confusing.

yeah, I agree that might be a little confusing, to have both depends on and after, but would cover more cases.

[csviri]

Just to clarify, I don't necessary mean to implement this now. We should gather some feedback, see more cases that not fit. And see if those could be fixed by this notion.

[csviri]

One sample, what this would fix more nicely is discussed here: #1732 (reply in thread)