Skip to content

feat: activation condition #2105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 21 additions & 10 deletions docs/documentation/workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,12 @@ reconciliation process.
proceeding until the condition checking whether the DR is ready holds true
- **Delete postcondition** - is a condition on a given DR to check if the reconciliation of
dependents can proceed after the DR is supposed to have been deleted
- **Activation condition** - is a special condition, what is mean to describe if the DR
should be used in the workflow. Typical use-case is, to differentiate if the actual
platform is Openshift or vanilla Kubernetes, therefore users can make sure that no
informers are registered for resources witch are platform specific (like Route on Openshift) -
since that would result in an error. But can be used do check like if CertMager is installed or not,
and define the behavior based on that.

## Defining Workflows

Expand Down Expand Up @@ -66,6 +72,7 @@ will only consider the `ConfigMap` deleted until that post-condition becomes `tr
@Dependent(type = ConfigMapDependentResource.class,
reconcilePrecondition = ConfigMapReconcileCondition.class,
deletePostcondition = ConfigMapDeletePostCondition.class,
activationCondition = ConfigMapActivationCondition.class,
dependsOn = DEPLOYMENT_NAME)
})
public class SampleWorkflowReconciler implements Reconciler<WorkflowAllFeatureCustomResource>,
Expand Down Expand Up @@ -165,7 +172,7 @@ executed if a resource is marked for deletion.
## Common Principles

- **As complete as possible execution** - when a workflow is reconciled, it tries to reconcile as
many resources as possible. Thus if an error happens or a ready condition is not met for a
many resources as possible. Thus, if an error happens or a ready condition is not met for a
resources, all the other independent resources will be still reconciled. This is the opposite
to a fail-fast approach. The assumption is that eventually in this way the overall state will
converge faster towards the desired state than would be the case if the reconciliation was
Expand All @@ -187,12 +194,12 @@ demonstrated using examples:
2. Root nodes, i.e. nodes in the graph that do not depend on other nodes are reconciled first,
in a parallel manner.
2. A DR is reconciled if it does not depend on any other DRs, or *ALL* the DRs it depends on are
reconciled and ready. If a DR defines a reconcile pre-condition, then this condition must
become `true` before the DR is reconciled.
reconciled and ready. If a DR defines a reconcile pre-condition and/or an activationCondition,
then these condition must become `true` before the DR is reconciled.
2. A DR is considered *ready* if it got successfully reconciled and any ready post-condition it
might define is `true`.
3. If a DR's reconcile pre-condition is not met, this DR is deleted. All of the DRs that depend
on the dependent resource being considered are also recursively deleted. This implies that
3. If a DR's reconcile pre-condition is not met, this DR is deleted. All the DRs that depend
on the dependent resource are also recursively deleted. This implies that
DRs are deleted in reverse order compared the one in which they are reconciled. The reason
for this behavior is (Will make a more detailed blog post about the design decision, much deeper
than the reference documentation)
Expand All @@ -202,11 +209,15 @@ demonstrated using examples:
idempotency (i.e. with the same input state, we should have the same output state), from this
follows that if the condition doesn't hold `true` anymore, the associated resource needs to
be deleted because the resource shouldn't exist/have been created.
4. For a DR to be deleted by a workflow, it needs to implement the `Deleter` interface, in which
case its `delete` method will be called, unless it also implements the `GarbageCollected`
interface. If a DR doesn't implement `Deleter` it is considered as automatically deleted. If
a delete post-condition exists for this DR, it needs to become `true` for the workflow to
consider the DR as successfully deleted.
4. if a DR's activation condition is not met, it won't be reconciled or deleted. If
other DR's depend on it, those will be recursively deleted in a same way as for reconcile pre-condition.
Event sources for a dependent resource with activation condition are registered/de-registered dynamically,
thus during the reconciliation.
5. For a DR to be deleted by a workflow, it needs to implement the `Deleter` interface, in which
case its `delete` method will be called, unless it also implements the `GarbageCollected`
interface. If a DR doesn't implement `Deleter` it is considered as automatically deleted. If
a delete post-condition exists for this DR, it needs to become `true` for the workflow to
consider the DR as successfully deleted.

### Samples

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,7 @@ private static List<DependentResourceSpec> dependentResources(
Utils.instantiate(dependent.readyPostcondition(), Condition.class, context),
Utils.instantiate(dependent.reconcilePrecondition(), Condition.class, context),
Utils.instantiate(dependent.deletePostcondition(), Condition.class, context),
Utils.instantiate(dependent.activationCondition(), Condition.class, context),
eventSourceName);
specsMap.put(dependentName, spec);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,18 +22,21 @@ public class DependentResourceSpec<R, P extends HasMetadata> {

private final Condition<?, ?> deletePostCondition;

private final Condition<?, ?> activationCondition;

private final String useEventSourceWithName;

public DependentResourceSpec(Class<? extends DependentResource<R, P>> dependentResourceClass,
String name, Set<String> dependsOn, Condition<?, ?> readyCondition,
Condition<?, ?> reconcileCondition, Condition<?, ?> deletePostCondition,
String useEventSourceWithName) {
Condition<?, ?> activationCondition, String useEventSourceWithName) {
this.dependentResourceClass = dependentResourceClass;
this.name = name;
this.dependsOn = dependsOn;
this.readyCondition = readyCondition;
this.reconcileCondition = reconcileCondition;
this.deletePostCondition = deletePostCondition;
this.activationCondition = activationCondition;
this.useEventSourceWithName = useEventSourceWithName;
}

Expand Down Expand Up @@ -87,6 +90,11 @@ public Condition getDeletePostCondition() {
return deletePostCondition;
}

@SuppressWarnings("rawtypes")
public Condition getActivationCondition() {
return activationCondition;
}

public Optional<String> getUseEventSourceWithName() {
return Optional.ofNullable(useEventSourceWithName);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,21 @@
*/
Class<? extends Condition> deletePostcondition() default Condition.class;

/**
* <p>
* If the condition is not met, the dependent resource won't be used. That is, no event sources
* will be registered for the dependent, and won't be reconciled or deleted. If other dependents
* are still "depend on" this resource, those still will be deleted when needed. Exactly the same
* way as for the reconcilePrecondition.
* </p>
* <p>
* This condition is evaluated dynamically, thus on every reconciliation as other conditions. Thus
* it's result can change dynamically, therefore the event source that the dependent resource
* provides are registered or de-registered dynamically during the reconciliation.
* </p>
*/
Class<? extends Condition> activationCondition() default Condition.class;

/**
* The list of named dependents that need to be reconciled before this one can be.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ public class Controller<P extends HasMetadata>
private final GroupVersionKind associatedGVK;
private final EventProcessor<P> eventProcessor;
private final ControllerHealthInfo controllerHealthInfo;
private final EventSourceContext<P> eventSourceContext;

public Controller(Reconciler<P> reconciler,
ControllerConfiguration<P> configuration,
Expand All @@ -98,9 +99,9 @@ public Controller(Reconciler<P> reconciler,
eventProcessor = new EventProcessor<>(eventSourceManager, configurationService);
eventSourceManager.postProcessDefaultEventSourcesAfterProcessorInitializer();
controllerHealthInfo = new ControllerHealthInfo(eventSourceManager);
final var context = new EventSourceContext<>(
eventSourceContext = new EventSourceContext<>(
eventSourceManager.getControllerResourceEventSource(), configuration, kubernetesClient);
initAndRegisterEventSources(context);
initAndRegisterEventSources(eventSourceContext);
configurationService.getMetrics().controllerRegistered(this);
}

Expand Down Expand Up @@ -236,7 +237,8 @@ public void initAndRegisterEventSources(EventSourceContext<P> context) {
}

// register created event sources
final var dependentResourcesByName = managedWorkflow.getDependentResourcesByName();
final var dependentResourcesByName =
managedWorkflow.getDependentResourcesByNameWithoutActivationCondition();
final var size = dependentResourcesByName.size();
if (size > 0) {
dependentResourcesByName.forEach((key, dependentResource) -> {
Expand Down Expand Up @@ -440,4 +442,8 @@ public EventProcessor<P> getEventProcessor() {
public ExecutorServiceManager getExecutorServiceManager() {
return getConfiguration().getConfigurationService().getExecutorServiceManager();
}

public EventSourceContext<P> eventSourceContext() {
return eventSourceContext;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,12 @@ public AbstractWorkflowExecutor(Workflow<P> workflow, P primary, Context<P> cont
protected synchronized void waitForScheduledExecutionsToRun() {
while (true) {
try {
this.wait();
if (noMoreExecutionsScheduled()) {
break;
} else {
logger().warn("Notified but still resources under execution. This should not happen.");
}
this.wait();
} catch (InterruptedException e) {
if (noMoreExecutionsScheduled()) {
logger().debug("interrupted, no more executions for: {}", primaryID);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ public Workflow<P> resolve(KubernetesClient client,
spec.getReconcileCondition(),
spec.getDeletePostCondition(),
spec.getReadyCondition(),
spec.getActivationCondition(),
resolve(spec, client, configuration));
alreadyResolved.put(node.getName(), node);
spec.getDependsOn()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -147,4 +147,15 @@ public Map<String, DependentResource> getDependentResourcesByName() {
.forEach((name, node) -> resources.put(name, node.getDependentResource()));
return resources;
}

public Map<String, DependentResource> getDependentResourcesByNameWithoutActivationCondition() {
final var resources = new HashMap<String, DependentResource>(dependentResourceNodes.size());
dependentResourceNodes
.forEach((name, node) -> {
if (node.getActivationCondition().isEmpty()) {
resources.put(name, node.getDependentResource());
}
});
return resources;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,19 +16,21 @@ public class DependentResourceNode<R, P extends HasMetadata> {
private Condition<R, P> reconcilePrecondition;
private Condition<R, P> deletePostcondition;
private Condition<R, P> readyPostcondition;
private Condition<R, P> activationCondition;
private final DependentResource<R, P> dependentResource;

DependentResourceNode(DependentResource<R, P> dependentResource) {
this(getNameFor(dependentResource), null, null, null, dependentResource);
this(getNameFor(dependentResource), null, null, null, null, dependentResource);
}

public DependentResourceNode(String name, Condition<R, P> reconcilePrecondition,
Condition<R, P> deletePostcondition, Condition<R, P> readyPostcondition,
DependentResource<R, P> dependentResource) {
Condition<R, P> activationCondition, DependentResource<R, P> dependentResource) {
this.name = name;
this.reconcilePrecondition = reconcilePrecondition;
this.deletePostcondition = deletePostcondition;
this.readyPostcondition = readyPostcondition;
this.activationCondition = activationCondition;
this.dependentResource = dependentResource;
}

Expand Down Expand Up @@ -63,6 +65,10 @@ public Optional<Condition<R, P>> getDeletePostcondition() {
return Optional.ofNullable(deletePostcondition);
}

public Optional<Condition<R, P>> getActivationCondition() {
return Optional.ofNullable(activationCondition);
}

void setReconcilePrecondition(Condition<R, P> reconcilePrecondition) {
this.reconcilePrecondition = reconcilePrecondition;
}
Expand All @@ -71,6 +77,10 @@ void setDeletePostcondition(Condition<R, P> cleanupCondition) {
this.deletePostcondition = cleanupCondition;
}

void setActivationCondition(Condition<R, P> activationCondition) {
this.activationCondition = activationCondition;
}

public Optional<Condition<R, P>> getReadyPostcondition() {
return Optional.ofNullable(readyPostcondition);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,8 @@ default boolean isEmpty() {
default Map<String, DependentResource> getDependentResourcesByName() {
return Collections.emptyMap();
}

default Map<String, DependentResource> getDependentResourcesByNameWithoutActivationCondition() {
return Collections.emptyMap();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,11 @@ public WorkflowBuilder<P> withDeletePostcondition(Condition deletePostcondition)
return this;
}

public WorkflowBuilder<P> withActivationCondition(Condition activationCondition) {
currentNode.setActivationCondition(activationCondition);
return this;
}

DependentResourceNode getNodeByDependentResource(DependentResource<?, ?> dependentResource) {
// first check by name
final var node =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,20 @@ protected void doRun(DependentResourceNode<R, P> dependentResourceNode,
DependentResource<R, P> dependentResource) {
var deletePostCondition = dependentResourceNode.getDeletePostcondition();

if (dependentResource.isDeletable()) {
var active =
isConditionMet(dependentResourceNode.getActivationCondition(), dependentResource);

if (dependentResource.isDeletable() && active) {
((Deleter<P>) dependentResource).delete(primary, context);
deleteCalled.add(dependentResourceNode);
}
boolean deletePostConditionMet = isConditionMet(deletePostCondition, dependentResource);

boolean deletePostConditionMet;
if (active) {
deletePostConditionMet = isConditionMet(deletePostCondition, dependentResource);
} else {
deletePostConditionMet = true;
}
if (deletePostConditionMet) {
markAsVisited(dependentResourceNode);
handleDependentCleaned(dependentResourceNode);
Expand Down
Loading