diff --git a/docs/proposals/20220414-runtime-hooks.md b/docs/proposals/20220414-runtime-hooks.md new file mode 100644 index 000000000000..110f22263057 --- /dev/null +++ b/docs/proposals/20220414-runtime-hooks.md @@ -0,0 +1,430 @@ +--- +title: Runtime Hooks for Add-on Management +authors: + - "@killianmuldoon" + - "@ykakarap" +reviewers: + - "@vincepri" + - "@CecileRobertMichon" + - "@enxebre" + - "@fabriziopandini" + - "@sbueringer" +creation-date: 2022-04-14 +last-updated: 2022-04-14 +status: implementable +replaces: +see-also: +superseded-by: +--- + +# Runtime hooks for Add-on Management + +## Table of Contents +* [Table of Contents](#table-of-contents) +* [Glossary](#glossary) +* [Summary](#summary) +* [Motivation](#motivation) + * [Goals](#goals) + * [Non-goals](#non-goals) +* [Proposal](#proposal) + * [User Stories](#user-stories) + * [Runtime hook definitions](#runtime-hook-definitions) + * [Before Cluster Create](#before-cluster-create) + * [After Control Plane Initialized](#after-control-plane-initialized) + * [Before Cluster Upgrade](#before-cluster-upgrade) + * [After Control Plane Upgrade](#after-control-plane-upgrade) + * [After Cluster Upgrade](#after-cluster-upgrade) + * [Before Cluster Delete](#before-cluster-delete) + * [Runtime Extensions developer guide](#runtime-extensions-developer-guide) + * [Security Model](#security-model) + * [Risks and Mitigations](#risks-and-mitigations) + * [Runtime Extension blocks Cluster lifecycle indefinitely](#runtime-extension-blocks-cluster-lifecycle-indefinitely) +* [Alternatives](#alternatives) + * [External components watching CAPI resources without hooks](#external-components-watching-capi-resources-without-hooks) + * [OpenAPI spec implementation alternatives](#openapi-spec-implementation-alternatives) + * [Adding Cluster info to request vs providing only the Cluster name](#adding-cluster-info-to-request-vs-providing-only-the-cluster-name) + * [Embedding CAPI types in request vs using runtime.RawExtension](#embedding-capi-types-in-request-vs-using-runtimerawextension) +* [Upgrade Strategy](#upgrade-strategy) + * [Cluster API version upgrade](#clusterapi-version-upgrade) + * [Kubernetes version upgrade](#kubernetes-version-upgrade) +* [Additional Details](#additional-details) + * [Test Plan](#test-plan) + * [Graduation Criteria](#graduation-criteria) + * [Version Skew Strategy](#version-skew-strategy) +* [Implementation History](#implementation-history) + +## Glossary + +Refer to the [Cluster API Book Glossary](https://cluster-api.sigs.k8s.io/reference/glossary.html). + +* Add-on: an application that extends the functionality of Kubernetes. + + +## Summary + +This proposal introduces a set of Runtime Hooks designed for providing the foundation for the implementation of add-on orchestration solutions on top of Cluster API. +However, given that the hooks defined in this proposal are going to model common events in the Cluster lifecycle they could be used for other use cases beyond add-on +management, but for the sake of having a focused and actionable scope, we are not exploring those option in this document. + +## Motivation + +Cluster Resource Set (CRS) is the current add-on management tool packaged with Cluster API, but many users rely on their own package management tool like helm, kapp, ArgoCD or flux, because those tools have a broad range of capabilities that are not currently available in CRS. + +The ability to orchestrate add-ons in line with events in the cluster lifecycle is becoming a requirement for many CAPI users, but in order to make this possible a mechanism to plug into the cluster lifecycle is required. This proposal introduces a set of Runtime Hooks designed to meet the need for add-on management orchestration including: + +* Operations for installing add-ons during the cluster provisioning workflow +* Operations for upgrading add-ons during the cluster upgrade workflow +* Operations for handling add-ons during the cluster deletion workflow + +Runtime Hooks enable the Cluster Lifecycle to trigger these processes based on the current state of the cluster, allowing them to start after some state has been reached or waiting for them to complete before moving on with Cluster-wide operations such as Cluster creation or deletion. + +Once these hooks are in place, it will be possible to build a comprehensive add-on orchestration solution on top of Cluster API that can leverage external tools such as helm, kapp, ArgoCD, flux or eventually CRS as an alternative. + +### Goals + +* Identify a set of Runtime Hooks that enable management of the entire add-on lifecycle +* Define the OpenAPI specification of these Runtime Hooks +* Document when the corresponding Runtime Extensions are called +* Provide guidelines for developers implementing a corresponding Runtime Extension + +### Non-Goals + +* Defining all possible Runtime Hooks in Cluster API; this proposal defines only a subset of hooks required for add-on orchestration. +* Define a full add-on management solution or define detailed steps for solving add-on related problems like CPI migration from in-tree to out-of-tree; this proposal is focused only in providing the foundational capabilities to do so. + +## Proposal + +This proposal adds a set of Runtime Hooks specifically designed for Cluster created from a ClusterClass and thus leveraging the idea of managed topology. + +The main reason for this choice is because a managed topology has a set of capabilities required for lifecycle hooks implementation: + +* A managed topology has the overarching view of the entire Cluster (vs other components in CAPI which are limited to controlling single resources or a subset of them). +* A managed topology already has the capability to control all resources in a Cluster, thus making it possible to orchestrate lifecycle workflows like e.g. upgrades. + +In practice, if we look at the six lifecycle hooks introduced by this proposal, we should recognize that four of them cannot be implemented outside of the topology controller +because there is no viable point to be used in an alternative (BeforeClusterCreate, BeforeClusterUpgrade, AfterControlPlaneUpgrade, and AfterClusterUpgrade). + +Also, by working in the topology controller it is possible to implement hooks allowing to block the cluster from transitioning from one state to another, which is a capability +required to properly orchestrate the addon lifecycle. + +### User Stories + +These user stories are based on a concrete example of an add-on - a metrics database - to illustrate the use of each Runtime Hook. + +As a developer of an add-ons orchestration solution: + +* **Before a Cluster is Created** I want to automatically check if enough disk space is available for allocation to the cluster for persistent storage of collected metrics values. +* **After the Control Plane** **is Initialized** I want to automatically install a metrics database and associated add-ons in the workload cluster. +* **Before the Cluster is Upgraded** I want to install a new version of the metrics database with a new version of the custom metrics apiservice to interact directly with the Kubernetes apiserver. +* **After the ControlPlane is Upgraded** I want to automatically check that the new version of the custom metrics apiservice is working and correctly fulfilled by my metrics database. +* **After the Cluster is Upgraded** I want to install new versions of metrics collectors to each upgraded node in the cluster. +* **Before the Cluster is Deleted** I want to automatically back up persistent volumes used by the metrics database. + +### Runtime hook definitions + +Below is a description for the Runtime Hooks introduced by this proposal. + +![runtime-hooks](images/runtime-hooks/runtime-hooks.png) + + +#### Before Cluster Create + +This hook is called after the Cluster object has been created by the user, immediately before all the objects which are part of a Cluster topology(*) are going to be created. Runtime Extension implementers can use this hook to determine/prepare add-ons for the Cluster and block the creation of those objects until everything is ready. + +##### Example Request: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: BeforeClusterCreateRequest +cluster: + apiVersion: cluster.x-k8s.io/v1beta1 + kind: Cluster + metadata: + name: test-cluster + namespace: test-ns + spec: + ... + status: + ... +``` + +##### Example Response: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: BeforeClusterCreateResponse +status: Success +message: "error message if status == Failure" +retryAfterSeconds: 10 +``` + +For additional details, refer to the [Draft OpenAPI spec](https://editor.swagger.io/?url=https://gist.githubusercontent.com/ykakarap/6a58d976df90928cdb31768125d73613/raw/8ce6175060039c2fb7b3ea9a091d712228f56852/lifecycle-hooks-openapi-2.yaml). + +(*) The objects which are part of a Cluster topology are the infrastructure Cluster, the control plane, the MachineDeployments and the templates derived from the ClusterClass. + + +#### After Control Plane Initialized + +This hook is called after the ControlPlane for the Cluster is marked as available for the first time. Runtime Extension implementers can use this hook to execute tasks, for example component installation on workload clusters, that are only possible once the Control Plane is available. This hook does not block any further changes to the Cluster. + +##### Example Request: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: AfterControlPlaneInitializedRequest +cluster: + apiVersion: cluster.x-k8s.io/v1beta1 + kind: Cluster + metadata: + name: test-cluster + namespace: test-ns + spec: + ... + status: + ... +``` + +##### Example Response: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: AfterControlPlaneInitializedResponse +status: Success +message: "error message if status == Failure" +``` + +For additional details, refer to the [Draft OpenAPI spec](https://editor.swagger.io/?url=https://gist.githubusercontent.com/ykakarap/6a58d976df90928cdb31768125d73613/raw/8ce6175060039c2fb7b3ea9a091d712228f56852/lifecycle-hooks-openapi-2.yaml). + + +#### Before Cluster Upgrade + +This hook is called after the Cluster object has been updated with a new spec.topology.version by the user, and immediately before the new version is going to be propagated to the control plane (*). Runtime Extension implementers can use this hook to execute pre-upgrade add-on tasks and block upgrades of the ControlPlane and Workers. + +##### Example Request: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: BeforeClusterUpgradeRequest +cluster: + apiVersion: cluster.x-k8s.io/v1beta1 + kind: Cluster + metadata: + name: test-cluster + namespace: test-ns + spec: + ... + status: + ... +fromKubernetesVersion: "v1.21.2" +toKubernetesVersion: "v1.22.0" +``` + +##### Example Response: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: BeforeClusterUpgradeResponse +status: Success +message: "error message if status == Failure" +retryAfterSeconds: 10 +``` + +For additional details, refer to the [Draft OpenAPI spec](https://editor.swagger.io/?url=https://gist.githubusercontent.com/ykakarap/6a58d976df90928cdb31768125d73613/raw/8ce6175060039c2fb7b3ea9a091d712228f56852/lifecycle-hooks-openapi-2.yaml). + +* Under normal circumstances spec.topology.version gets propagated to the control plane immediately; however if previous upgrades or worker machine rollouts are still in progress, the system waits for those operations to complete before starting the new upgrade. + +#### After Control Plane Upgrade + +This hook is called after the control plane has been upgraded to the version specified in spec.topology.version, and immediately before the new version is going to be propagated to the MachineDeployments existing in the Cluster. Runtime Extension implementers can use this hook to execute post-upgrade add-on tasks and block upgrades to workers until everything is ready. + +##### Example Request: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: AfterControlPlaneUpgradeRequest +cluster: + apiVersion: cluster.x-k8s.io/v1beta1 + kind: Cluster + metadata: + name: test-cluster + namespace: test-ns + spec: + ... + status: + ... +kubernetesVersion: "v1.22.0" +``` + +##### Example Response: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: AfterControlPlaneUpgradeResponse +status: Success +message: "error message if status == Failure" +retryAfterSeconds: 10 +``` + +For additional details, refer to the [Draft OpenAPI spec](https://editor.swagger.io/?url=https://gist.githubusercontent.com/ykakarap/6a58d976df90928cdb31768125d73613/raw/8ce6175060039c2fb7b3ea9a091d712228f56852/lifecycle-hooks-openapi-2.yaml). + + +#### After Cluster Upgrade + +This hook is called after the Cluster, control plane and workers have been upgraded to the version specified in spec.topology.version. Runtime Extensions implementers can use this hook to execute post-upgrade add-on tasks. This hook does not block any further changes or upgrades to the Cluster. + +##### Example Request: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: AfterClusterUpgradeRequest +cluster: + apiVersion: cluster.x-k8s.io/v1beta1 + kind: Cluster + metadata: + name: test-cluster + namespace: test-ns + spec: + ... + status: + ... +kubernetesVersion: "v1.22.0" +``` + +##### Example Response: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: AfterClusterUpgradeResponse +status: Success +message: "error message if status == Failure" +``` + +For additional details, refer to the [Draft OpenAPI spec](https://editor.swagger.io/?url=https://gist.githubusercontent.com/ykakarap/6a58d976df90928cdb31768125d73613/raw/8ce6175060039c2fb7b3ea9a091d712228f56852/lifecycle-hooks-openapi-2.yaml). + +#### Before Cluster Delete + +This hook is called after the Cluster has been deleted by the user, and immediately before objects existing in the Cluster are going to be deleted. Runtime Extension implementers can use this hook to execute cleanup tasks for the add-ons and block deletion of the Cluster and descendant objects until everything is ready. + +##### Example Request: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: BeforeClusterDeleteRequest +cluster: + apiVersion: cluster.x-k8s.io/v1beta1 + kind: Cluster + metadata: + name: test-cluster + namespace: test-ns + spec: + ... + status: + ... +``` + +##### Example Response: + +```yaml +apiVersion: hooks.runtime.cluster.x-k8s.io/v1alpha1 +kind: BeforeClusterDeleteResponse +status: Success +message: "error message if status == Failure" +retryAfterSeconds: 10 +``` + +For additional details, refer to the [Draft OpenAPI spec](https://editor.swagger.io/?url=https://gist.githubusercontent.com/ykakarap/6a58d976df90928cdb31768125d73613/raw/8ce6175060039c2fb7b3ea9a091d712228f56852/lifecycle-hooks-openapi-2.yaml). + + +### Runtime Extensions developer guide + +All guidelines defined in the [Runtime SDK](https://github.com/kubernetes-sigs/cluster-api/blob/b48a6ed07ac2bd353f99000270510369f4baa1a5/docs/proposals/20220221-runtime-SDK.md) apply to the implementation of Runtime Extensions of the hooks defined in this proposal. + +TL;DR; Runtime Extensions are components that should be designed, written and deployed with great caution given that they can affect the proper functioning of the Cluster API runtime. A poorly implemented Runtime Extension could potentially block lifecycle transitions from happening. + +Following recommendations are especially relevant: + +* [Blocking and non Blocking](https://github.com/kubernetes-sigs/cluster-api/blob/b48a6ed07ac2bd353f99000270510369f4baa1a5/docs/proposals/20220221-runtime-SDK.md#blocking-hooks) +* [Error management](https://github.com/kubernetes-sigs/cluster-api/blob/b48a6ed07ac2bd353f99000270510369f4baa1a5/docs/proposals/20220221-runtime-SDK.md#error-management) +* [Avoid dependencies](https://github.com/kubernetes-sigs/cluster-api/blob/b48a6ed07ac2bd353f99000270510369f4baa1a5/docs/proposals/20220221-runtime-SDK.md#avoid-dependencies) + + +### Security Model + +For the general Runtime Extension security model please refer to the [developer guide in the Runtime SDK proposal](https://github.com/kubernetes-sigs/cluster-api/blob/75b39db545ae439f4f6203b5e07496d3b0a6aa75/docs/proposals/20220221-runtime-SDK.md#security-model). + +### Risks and Mitigations + +#### Runtime Extension blocks Cluster lifecycle indefinitely + +Cluster lifecycle can be blocked indefinitely when a Runtime Extension either blocks or fails indefinitely. Mitigation: + +* Surface errors from the Runtime Extension that is blocking reconciliation in Conditions to drive necessary action needed by the user. +* A Runtime Extension should be extensively unit and e2e tested to ensure it behaves as expected. +* Users should be able to manually intervene and unblock the reconciliation. + +As future work, we will explore more options like circuit breaker and timeout to unblock reconciliation. + + +## Alternatives + +Alternatives to Runtime Hooks for comprehensive add-on management in Cluster API include: + +### External components watching CAPI resources without hooks + +This is the current pattern used by Cluster Resource Set. The implementers can only react to changes on CAPI resources, e.g. a Cluster being created, but they have no control over the cluster lifecycle. + +This and similar solutions based on scripting or git ops approaches are considered inadequate for comprehensive, integrated add-on management as they have limited insight into the state of the Cluster and can not easily influence reconciliation based on add-on state. + +More details about why watching Cluster API resources without hooks is not considered a valid alternative can be found in the [Cluster Addon Proposal](https://docs.google.com/document/d/1TdbfXC2_Hhg0mH7-7hXcT1Gg8h6oXKrKbnJqbpFFvjw/edit). + +### OpenAPI spec implementation alternatives + +For the implementation detail of Open API spec, we considered following alternative approaches: +#### Adding Cluster info to request vs providing only the Cluster name + +In the proposed Open API specification for request type we have a full Cluster object. We considered an alternative of only including the Cluster name and namespace to reduce the size of the message. It was rejected based on the assumption that most extensions will require at least some additional information from the Cluster. Sending the full object reduces calls to the API server. + +#### Embedding CAPI types in request vs using runtime.RawExtension + +In the proposed Open API specification we are including the Cluster API object in requests. We considered using runtime.RawExtension in order to avoid having to bump the version of lifecycle hooks when bumping the version of the CAPI types. It was rejected as sending another version of the CAPI type via runtime.RawExtension would always be a breaking change. Embedding the type directly makes the version of the API used explicit. + +## Upgrade strategy + +#### Cluster API version upgrade + +This proposal does not affect the Cluster API upgrade strategy. + +If a new ClusterAPI version introduces a new Lifecycle Hook version, Runtime Extensions should be adapted, to avoid issues when older Lifecycle Hook versions are eventually removed. For details about the deprecation rules please refer to the [Runtime SDK](https://github.com/kubernetes-sigs/cluster-api/blob/75b39db545ae439f4f6203b5e07496d3b0a6aa75/docs/proposals/20220221-runtime-SDK.md#runtime-sdk-rules-1). + + +#### Kubernetes version upgrade + +This proposal does not affect the Cluster API cluster upgrade strategy. + +However Runtime Extension will be able to tap into the upgrade process at defined stages. + +## Additional Details + +### Test Plan + +While in alpha phase it is expected that the Runtime Hooks will have unit and integration tests covering the topology reconciliation with calls to Runtime Extensions. + +With the increasing adoption of this feature we expect E2E test coverage for topology reconciliation with a Runtime Extension generating Runtime Hook Responses. + +### Graduation Criteria + +Main criteria for graduating this feature is adoption; further detail about graduation criteria will be added in future iterations of this document. + +### Version Skew Strategy + +See [upgrade strategy](#upgrade-strategy). + +## Implementation History + +* [x] 2022-03-29: Compiled a [CAEP Google Doc](https://docs.google.com/document/d/1vMwzGBi6XbIwKzP5aA7Mj9UdhAWKYqI-QmdDtcxWnA4) +* [x] 2022-04-04: Opened corresponding [issue](https://github.com/kubernetes-sigs/cluster-api/issues/6374) +* [x] 2022-04-06: Presented proposal at a [community meeting] +* [x] 2022-04-14: Opened proposal PR + + +[community meeting]: https://docs.google.com/document/d/1ushaVqAKYnZ2VN_aa3GyKlS4kEd6bSug13xaXOakAQI/edit#heading=h.pxsq37pzkbdq \ No newline at end of file diff --git a/docs/proposals/images/runtime-hooks/runtime-hooks-openapi.yaml b/docs/proposals/images/runtime-hooks/runtime-hooks-openapi.yaml new file mode 100644 index 000000000000..3b55d0f89c03 --- /dev/null +++ b/docs/proposals/images/runtime-hooks/runtime-hooks-openapi.yaml @@ -0,0 +1,1190 @@ +components: + schemas: + k8s.io.api.core.v1.ObjectReference: + description: ObjectReference contains enough information to let you inspect + or modify the referred object. + properties: + apiVersion: + description: API version of the referent. + type: string + fieldPath: + description: 'If referring to a piece of an object instead of an entire + object, this string should contain a valid JSON/Go field access statement, + such as desiredState.manifest.containers[2]. For example, if the object + reference is to a container within a pod, this would take on a value like: + "spec.containers{name}" (where "name" refers to the name of the container + that triggered the event) or if no container name is specified "spec.containers[2]" + (container with index 2 in this pod). This syntax is chosen only to have + some well-defined way of referencing a part of an object.' + type: string + kind: + description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names' + type: string + namespace: + description: 'Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/' + type: string + resourceVersion: + description: 'Specific resourceVersion to which this reference is made, + if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency' + type: string + uid: + description: 'UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids' + type: string + type: object + x-kubernetes-map-type: atomic + k8s.io.apiextensions-apiserver.pkg.apis.apiextensions.v1.JSON: + description: 'JSON represents any valid JSON value. These types are supported: + bool, int64, float64, string, []interface{}, map[string]interface{} and nil.' + k8s.io.apimachinery.pkg.apis.meta.v1.FieldsV1: + description: |- + FieldsV1 stores a set of fields in a data structure like a Trie, in JSON format. + + Each key is either a '.' representing the field itself, and will always map to an empty set, or a string representing a sub-field or item. The string will follow one of these four formats: 'f:', where is the name of a field in a struct, or key in a map 'v:', where is the exact json formatted value of a list item 'i:', where is position of a item in a list 'k:', where is a map of a list item's key fields to their unique values If a key maps to an empty Fields value, the field that key represents is part of the set. + + The exact format is defined in sigs.k8s.io/structured-merge-diff + type: object + k8s.io.apimachinery.pkg.apis.meta.v1.ManagedFieldsEntry: + description: ManagedFieldsEntry is a workflow-id, a FieldSet and the group version + of the resource that the fieldset applies to. + properties: + apiVersion: + description: APIVersion defines the version of this resource that this field + set applies to. The format is "group/version" just like the top-level + APIVersion field. It is necessary to track the version of a field set + because it cannot be automatically converted. + type: string + fieldsType: + description: 'FieldsType is the discriminator for the different fields format + and version. There is currently only one possible value: "FieldsV1"' + type: string + fieldsV1: + $ref: '#/components/schemas/k8s.io.apimachinery.pkg.apis.meta.v1.FieldsV1' + description: FieldsV1 holds the first JSON version format as described in + the "FieldsV1" type. + manager: + description: Manager is an identifier of the workflow managing these fields. + type: string + operation: + description: Operation is the type of operation which lead to this ManagedFieldsEntry + being created. The only valid values for this field are 'Apply' and 'Update'. + type: string + subresource: + description: Subresource is the name of the subresource used to update that + object, or empty string if the object was updated through the main resource. + The value of this field is used to distinguish between managers, even + if they share the same name. For example, a status update will be distinct + from a regular update using the same manager name. Note that the APIVersion + field is not related to the Subresource field and it always corresponds + to the version of the main resource. + type: string + time: + $ref: '#/components/schemas/k8s.io.apimachinery.pkg.apis.meta.v1.Time' + description: Time is timestamp of when these fields were set. It should + always be empty if Operation is 'Apply' + type: object + k8s.io.apimachinery.pkg.apis.meta.v1.ObjectMeta: + description: ObjectMeta is metadata that all persisted resources must have, + which includes all objects users must create. + properties: + annotations: + additionalProperties: + default: "" + type: string + description: 'Annotations is an unstructured key value map stored with a + resource that may be set by external tools to store and retrieve arbitrary + metadata. They are not queryable and should be preserved when modifying + objects. More info: http://kubernetes.io/docs/user-guide/annotations' + type: object + clusterName: + description: The name of the cluster which the object belongs to. This is + used to distinguish resources with same name and namespace in different + clusters. This field is not set anywhere right now and apiserver is going + to ignore it if set in create or update request. + type: string + creationTimestamp: + $ref: '#/components/schemas/k8s.io.apimachinery.pkg.apis.meta.v1.Time' + default: {} + description: |- + CreationTimestamp is a timestamp representing the server time when this object was created. It is not guaranteed to be set in happens-before order across separate operations. Clients may not set this value. It is represented in RFC3339 form and is in UTC. + + Populated by the system. Read-only. Null for lists. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata + deletionGracePeriodSeconds: + description: Number of seconds allowed for this object to gracefully terminate + before it will be removed from the system. Only set when deletionTimestamp + is also set. May only be shortened. Read-only. + format: int64 + type: integer + deletionTimestamp: + $ref: '#/components/schemas/k8s.io.apimachinery.pkg.apis.meta.v1.Time' + description: |- + DeletionTimestamp is RFC 3339 date and time at which this resource will be deleted. This field is set by the server when a graceful deletion is requested by the user, and is not directly settable by a client. The resource is expected to be deleted (no longer visible from resource lists, and not reachable by name) after the time in this field, once the finalizers list is empty. As long as the finalizers list contains items, deletion is blocked. Once the deletionTimestamp is set, this value may not be unset or be set further into the future, although it may be shortened or the resource may be deleted prior to this time. For example, a user may request that a pod is deleted in 30 seconds. The Kubelet will react by sending a graceful termination signal to the containers in the pod. After that 30 seconds, the Kubelet will send a hard termination signal (SIGKILL) to the container and after cleanup, remove the pod from the API. In the presence of network partitions, this object may still exist after this timestamp, until an administrator or automated process can determine the resource is fully terminated. If not set, graceful deletion of the object has not been requested. + + Populated by the system when a graceful deletion is requested. Read-only. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata + finalizers: + description: Must be empty before the object is deleted from the registry. + Each entry is an identifier for the responsible component that will remove + the entry from the list. If the deletionTimestamp of the object is non-nil, + entries in this list can only be removed. Finalizers may be processed + and removed in any order. Order is NOT enforced because it introduces + significant risk of stuck finalizers. finalizers is a shared field, any + actor with permission can reorder it. If the finalizer list is processed + in order, then this can lead to a situation in which the component responsible + for the first finalizer in the list is waiting for a signal (field value, + external system, or other) produced by a component responsible for a finalizer + later in the list, resulting in a deadlock. Without enforced ordering + finalizers are free to order amongst themselves and are not vulnerable + to ordering changes in the list. + items: + default: "" + type: string + type: array + x-kubernetes-patch-strategy: merge + generateName: + description: |- + GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server. + + If this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header). + + Applied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#idempotency + type: string + generation: + description: A sequence number representing a specific generation of the + desired state. Populated by the system. Read-only. + format: int64 + type: integer + labels: + additionalProperties: + default: "" + type: string + description: 'Map of string keys and values that can be used to organize + and categorize (scope and select) objects. May match selectors of replication + controllers and services. More info: http://kubernetes.io/docs/user-guide/labels' + type: object + managedFields: + description: ManagedFields maps workflow-id and version to the set of fields + that are managed by that workflow. This is mostly for internal housekeeping, + and users typically shouldn't need to set or understand this field. A + workflow can be the user's name, a controller's name, or the name of a + specific apply path like "ci-cd". The set of fields is always in the version + that the workflow used when modifying the object. + items: + $ref: '#/components/schemas/k8s.io.apimachinery.pkg.apis.meta.v1.ManagedFieldsEntry' + default: {} + type: array + name: + description: 'Name must be unique within a namespace. Is required when creating + resources, although some resources may allow a client to request the generation + of an appropriate name automatically. Name is primarily intended for creation + idempotence and configuration definition. Cannot be updated. More info: + http://kubernetes.io/docs/user-guide/identifiers#names' + type: string + namespace: + description: |- + Namespace defines the space within which each name must be unique. An empty namespace is equivalent to the "default" namespace, but "default" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty. + + Must be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces + type: string + ownerReferences: + description: List of objects depended by this object. If ALL objects in + the list have been deleted, this object will be garbage collected. If + this object is managed by a controller, then an entry in this list will + point to this controller, with the controller field set to true. There + cannot be more than one managing controller. + items: + $ref: '#/components/schemas/k8s.io.apimachinery.pkg.apis.meta.v1.OwnerReference' + default: {} + type: array + x-kubernetes-patch-merge-key: uid + x-kubernetes-patch-strategy: merge + resourceVersion: + description: |- + An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources. + + Populated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency + type: string + selfLink: + description: |- + SelfLink is a URL representing this object. Populated by the system. Read-only. + + DEPRECATED Kubernetes will stop propagating this field in 1.20 release and the field is planned to be removed in 1.21 release. + type: string + uid: + description: |- + UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations. + + Populated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids + type: string + type: object + k8s.io.apimachinery.pkg.apis.meta.v1.OwnerReference: + description: OwnerReference contains enough information to let you identify + an owning object. An owning object must be in the same namespace as the dependent, + or be cluster-scoped, so there is no namespace field. + properties: + apiVersion: + default: "" + description: API version of the referent. + type: string + blockOwnerDeletion: + description: If true, AND if the owner has the "foregroundDeletion" finalizer, + then the owner cannot be deleted from the key-value store until this reference + is removed. Defaults to false. To set this field, a user needs "delete" + permission of the owner, otherwise 422 (Unprocessable Entity) will be + returned. + type: boolean + controller: + description: If true, this reference points to the managing controller. + type: boolean + kind: + default: "" + description: 'Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + name: + default: "" + description: 'Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names' + type: string + uid: + default: "" + description: 'UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids' + type: string + required: + - apiVersion + - kind + - name + - uid + type: object + x-kubernetes-map-type: atomic + k8s.io.apimachinery.pkg.apis.meta.v1.Time: + description: Time is a wrapper around time.Time which supports correct marshaling + to YAML and JSON. Wrappers are provided for many of the factory methods that + the time package offers. + format: date-time + type: string + sigs.k8s.io.cluster-api.api.v1beta1.APIEndpoint: + description: APIEndpoint represents a reachable Kubernetes API endpoint. + properties: + host: + default: "" + description: The hostname on which the API server is serving. + type: string + port: + default: 0 + description: The port on which the API server is serving. + format: int32 + type: integer + required: + - host + - port + type: object + sigs.k8s.io.cluster-api.api.v1beta1.Cluster: + description: Cluster is the Schema for the clusters API. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + $ref: '#/components/schemas/k8s.io.apimachinery.pkg.apis.meta.v1.ObjectMeta' + default: {} + spec: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.ClusterSpec' + default: {} + status: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.ClusterStatus' + default: {} + type: object + sigs.k8s.io.cluster-api.api.v1beta1.ClusterNetwork: + description: ClusterNetwork specifies the different networking parameters for + a cluster. + properties: + apiServerPort: + description: APIServerPort specifies the port the API Server should bind + to. Defaults to 6443. + format: int32 + type: integer + pods: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.NetworkRanges' + description: The network ranges from which Pod networks are allocated. + serviceDomain: + description: Domain name for services. + type: string + services: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.NetworkRanges' + description: The network ranges from which service VIPs are allocated. + type: object + sigs.k8s.io.cluster-api.api.v1beta1.ClusterSpec: + description: ClusterSpec defines the desired state of Cluster. + properties: + clusterNetwork: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.ClusterNetwork' + description: Cluster network configuration. + controlPlaneEndpoint: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.APIEndpoint' + default: {} + description: ControlPlaneEndpoint represents the endpoint used to communicate + with the control plane. + controlPlaneRef: + $ref: '#/components/schemas/k8s.io.api.core.v1.ObjectReference' + description: ControlPlaneRef is an optional reference to a provider-specific + resource that holds the details for provisioning the Control Plane for + a Cluster. + infrastructureRef: + $ref: '#/components/schemas/k8s.io.api.core.v1.ObjectReference' + description: InfrastructureRef is a reference to a provider-specific resource + that holds the details for provisioning infrastructure for a cluster in + said provider. + paused: + description: Paused can be used to prevent controllers from processing the + Cluster and all its associated objects. + type: boolean + topology: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.Topology' + description: 'This encapsulates the topology for the cluster. NOTE: It is + required to enable the ClusterTopology feature gate flag to activate managed + topologies support; this feature is highly experimental, and parts of + it might still be not implemented.' + type: object + sigs.k8s.io.cluster-api.api.v1beta1.ClusterStatus: + description: ClusterStatus defines the observed state of Cluster. + properties: + conditions: + description: Conditions defines current service state of the cluster. + items: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.Condition' + default: {} + type: array + controlPlaneReady: + default: false + description: ControlPlaneReady defines if the control plane is ready. + type: boolean + failureDomains: + additionalProperties: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.FailureDomainSpec' + default: {} + description: FailureDomains is a slice of failure domain objects synced + from the infrastructure provider. + type: object + failureMessage: + description: FailureMessage indicates that there is a fatal problem reconciling + the state, and will be set to a descriptive error message. + type: string + failureReason: + description: FailureReason indicates that there is a fatal problem reconciling + the state, and will be set to a token value suitable for programmatic + interpretation. + type: string + infrastructureReady: + default: false + description: InfrastructureReady is the state of the infrastructure provider. + type: boolean + observedGeneration: + description: ObservedGeneration is the latest generation observed by the + controller. + format: int64 + type: integer + phase: + description: Phase represents the current phase of cluster actuation. E.g. + Pending, Running, Terminating, Failed etc. + type: string + type: object + sigs.k8s.io.cluster-api.api.v1beta1.ClusterVariable: + description: ClusterVariable can be used to customize the Cluster through patches. + It must comply to the corresponding ClusterClassVariable defined in the ClusterClass. + properties: + name: + default: "" + description: Name of the variable. + type: string + value: + $ref: '#/components/schemas/k8s.io.apiextensions-apiserver.pkg.apis.apiextensions.v1.JSON' + default: {} + description: 'Value of the variable. Note: the value will be validated against + the schema of the corresponding ClusterClassVariable from the ClusterClass. + Note: We have to use apiextensionsv1.JSON instead of a custom JSON type, + because controller-tools has a hard-coded schema for apiextensionsv1.JSON + which cannot be produced by another type via controller-tools, i.e. it + is not possible to have no type field. Ref: https://github.com/kubernetes-sigs/controller-tools/blob/d0e03a142d0ecdd5491593e941ee1d6b5d91dba6/pkg/crd/known_types.go#L106-L111' + required: + - name + - value + type: object + sigs.k8s.io.cluster-api.api.v1beta1.Condition: + description: Condition defines an observation of a Cluster API resource operational + state. + properties: + lastTransitionTime: + $ref: '#/components/schemas/k8s.io.apimachinery.pkg.apis.meta.v1.Time' + default: {} + description: Last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, + then using the time when the API field changed is acceptable. + message: + description: A human readable message indicating details about the transition. + This field may be empty. + type: string + reason: + description: The reason for the condition's last transition in CamelCase. + The specific API may choose whether or not this field is considered a + guaranteed API. This field may not be empty. + type: string + severity: + description: Severity provides an explicit classification of Reason code, + so the users or machines can immediately understand the current situation + and act accordingly. The Severity field MUST be set only when Status=False. + type: string + status: + default: "" + description: Status of the condition, one of True, False, Unknown. + type: string + type: + default: "" + description: Type of condition in CamelCase or in foo.example.com/CamelCase. + Many .condition.type values are consistent across resources like Available, + but because arbitrary conditions can be useful (see .node.status.conditions), + the ability to deconflict is important. + type: string + required: + - type + - status + - lastTransitionTime + type: object + sigs.k8s.io.cluster-api.api.v1beta1.ControlPlaneTopology: + description: ControlPlaneTopology specifies the parameters for the control plane + nodes in the cluster. + properties: + metadata: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.ObjectMeta' + default: {} + description: |- + Metadata is the metadata applied to the machines of the ControlPlane. At runtime this metadata is merged with the corresponding metadata from the ClusterClass. + + This field is supported if and only if the control plane provider template referenced in the ClusterClass is Machine based. + replicas: + description: Replicas is the number of control plane nodes. If the value + is nil, the ControlPlane object is created without the number of Replicas + and it's assumed that the control plane controller does not implement + support for this field. When specified against a control plane provider + that lacks support for this field, this value will be ignored. + format: int32 + type: integer + type: object + sigs.k8s.io.cluster-api.api.v1beta1.FailureDomainSpec: + description: FailureDomainSpec is the Schema for Cluster API failure domains. + It allows controllers to understand how many failure domains a cluster can + optionally span across. + properties: + attributes: + additionalProperties: + default: "" + type: string + description: Attributes is a free form map of attributes an infrastructure + provider might use or require. + type: object + controlPlane: + description: ControlPlane determines if this failure domain is suitable + for use by control plane machines. + type: boolean + type: object + sigs.k8s.io.cluster-api.api.v1beta1.MachineDeploymentTopology: + description: MachineDeploymentTopology specifies the different parameters for + a set of worker nodes in the topology. This set of nodes is managed by a MachineDeployment + object whose lifecycle is managed by the Cluster controller. + properties: + class: + default: "" + description: Class is the name of the MachineDeploymentClass used to create + the set of worker nodes. This should match one of the deployment classes + defined in the ClusterClass object mentioned in the `Cluster.Spec.Class` + field. + type: string + failureDomain: + description: FailureDomain is the failure domain the machines will be created + in. Must match a key in the FailureDomains map stored on the cluster object. + type: string + metadata: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.ObjectMeta' + default: {} + description: Metadata is the metadata applied to the machines of the MachineDeployment. + At runtime this metadata is merged with the corresponding metadata from + the ClusterClass. + name: + default: "" + description: Name is the unique identifier for this MachineDeploymentTopology. + The value is used with other unique identifiers to create a MachineDeployment's + Name (e.g. cluster's name, etc). In case the name is greater than the + allowed maximum length, the values are hashed together. + type: string + replicas: + description: Replicas is the number of worker nodes belonging to this set. + If the value is nil, the MachineDeployment is created without the number + of Replicas (defaulting to zero) and it's assumed that an external entity + (like cluster autoscaler) is responsible for the management of this value. + format: int32 + type: integer + variables: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.MachineDeploymentVariables' + description: Variables can be used to customize the MachineDeployment through + patches. + required: + - class + - name + type: object + sigs.k8s.io.cluster-api.api.v1beta1.MachineDeploymentVariables: + description: MachineDeploymentVariables can be used to provide variables for + a specific MachineDeployment. + properties: + overrides: + description: Overrides can be used to override Cluster level variables. + items: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.ClusterVariable' + default: {} + type: array + type: object + sigs.k8s.io.cluster-api.api.v1beta1.NetworkRanges: + description: NetworkRanges represents ranges of network addresses. + properties: + cidrBlocks: + items: + default: "" + type: string + type: array + required: + - cidrBlocks + type: object + sigs.k8s.io.cluster-api.api.v1beta1.ObjectMeta: + description: |- + ObjectMeta is metadata that all persisted resources must have, which includes all objects users must create. This is a copy of customizable fields from metav1.ObjectMeta. + + ObjectMeta is embedded in `Machine.Spec`, `MachineDeployment.Template` and `MachineSet.Template`, which are not top-level Kubernetes objects. Given that metav1.ObjectMeta has lots of special cases and read-only fields which end up in the generated CRD validation, having it as a subset simplifies the API and some issues that can impact user experience. + + During the [upgrade to controller-tools@v2](https://github.com/kubernetes-sigs/cluster-api/pull/1054) for v1alpha2, we noticed a failure would occur running Cluster API test suite against the new CRDs, specifically `spec.metadata.creationTimestamp in body must be of type string: "null"`. The investigation showed that `controller-tools@v2` behaves differently than its previous version when handling types from [metav1](k8s.io/apimachinery/pkg/apis/meta/v1) package. + + In more details, we found that embedded (non-top level) types that embedded `metav1.ObjectMeta` had validation properties, including for `creationTimestamp` (metav1.Time). The `metav1.Time` type specifies a custom json marshaller that, when IsZero() is true, returns `null` which breaks validation because the field isn't marked as nullable. + + In future versions, controller-tools@v2 might allow overriding the type and validation for embedded types. When that happens, this hack should be revisited. + properties: + annotations: + additionalProperties: + default: "" + type: string + description: 'Annotations is an unstructured key value map stored with a + resource that may be set by external tools to store and retrieve arbitrary + metadata. They are not queryable and should be preserved when modifying + objects. More info: http://kubernetes.io/docs/user-guide/annotations' + type: object + labels: + additionalProperties: + default: "" + type: string + description: 'Map of string keys and values that can be used to organize + and categorize (scope and select) objects. May match selectors of replication + controllers and services. More info: http://kubernetes.io/docs/user-guide/labels' + type: object + type: object + sigs.k8s.io.cluster-api.api.v1beta1.Topology: + description: Topology encapsulates the information of the managed resources. + properties: + class: + default: "" + description: The name of the ClusterClass object to create the topology. + type: string + controlPlane: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.ControlPlaneTopology' + default: {} + description: ControlPlane describes the cluster control plane. + rolloutAfter: + $ref: '#/components/schemas/k8s.io.apimachinery.pkg.apis.meta.v1.Time' + description: RolloutAfter performs a rollout of the entire cluster one component + at a time, control plane first and then machine deployments. + variables: + description: Variables can be used to customize the Cluster through patches. + They must comply to the corresponding VariableClasses defined in the ClusterClass. + items: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.ClusterVariable' + default: {} + type: array + version: + default: "" + description: The Kubernetes version of the cluster. + type: string + workers: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.WorkersTopology' + description: Workers encapsulates the different constructs that form the + worker nodes for the cluster. + required: + - class + - version + type: object + sigs.k8s.io.cluster-api.api.v1beta1.WorkersTopology: + description: WorkersTopology represents the different sets of worker nodes in + the cluster. + properties: + machineDeployments: + description: MachineDeployments is a list of machine deployments in the + cluster. + items: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.MachineDeploymentTopology' + default: {} + type: array + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterClusterUpgradeRequest: + description: AfterClusterUpgradeRequest is the request of the hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + cluster: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.Cluster' + default: {} + description: The cluster object the lifecycle hook corresponds to. + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + kubernetesVersion: + default: "" + description: The version after upgrade. + type: string + required: + - cluster + - kubernetesVersion + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterClusterUpgradeResponse: + description: AfterClusterUpgradeResponse is the response of AfterClusterUpgrade + hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + message: + default: "" + description: A human-readable description of the status of the call. + type: string + status: + default: "" + description: |- + Status of the call. One of "Success" or "Failure". + + Possible enum values: + - `"Failure"` + - `"Success"` + enum: + - Failure + - Success + type: string + required: + - status + - message + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterControlPlaneInitializedRequest: + description: AfterControlPlaneInitializedRequest is the request of the hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + cluster: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.Cluster' + default: {} + description: The cluster object the lifecycle hook corresponds to. + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + required: + - cluster + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterControlPlaneInitializedResponse: + description: AfterControlPlaneInitializedResponse is the response of AfterControlPlaneInitialized + hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + message: + default: "" + description: A human-readable description of the status of the call. + type: string + status: + default: "" + description: |- + Status of the call. One of "Success" or "Failure". + + Possible enum values: + - `"Failure"` + - `"Success"` + enum: + - Failure + - Success + type: string + required: + - status + - message + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterControlPlaneUpgradeRequest: + description: AfterControlPlaneUpgradeRequest is the request of the hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + cluster: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.Cluster' + default: {} + description: The cluster object the lifecycle hook corresponds to. + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + kubernetesVersion: + default: "" + description: The version after upgrade. + type: string + required: + - cluster + - kubernetesVersion + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterControlPlaneUpgradeResponse: + description: AfterControlPlaneUpgradeResponse is the response of AfterControlPlaneUpgrade + hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + message: + default: "" + description: A human-readable description of the status of the call. + type: string + retryAfterSeconds: + default: 0 + description: RetryAfterSeconds when set to a non-zero signifies that the + hook needs to be retried at a future time. + format: int32 + type: integer + status: + default: "" + description: |- + Status of the call. One of "Success" or "Failure". + + Possible enum values: + - `"Failure"` + - `"Success"` + enum: + - Failure + - Success + type: string + required: + - status + - retryAfterSeconds + - message + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterCreateRequest: + description: BeforeClusterCreateRequest is the request of the hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + cluster: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.Cluster' + default: {} + description: The cluster object the lifecycle hook corresponds to. + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + required: + - cluster + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterCreateResponse: + description: BeforeClusterCreateResponse is the response of BeforeClusterCreate + hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + message: + default: "" + description: A human-readable description of the status of the call. + type: string + retryAfterSeconds: + default: 0 + description: RetryAfterSeconds when set to a non-zero signifies that the + hook needs to be retried at a future time. + format: int32 + type: integer + status: + default: "" + description: |- + Status of the call. One of "Success" or "Failure". + + Possible enum values: + - `"Failure"` + - `"Success"` + enum: + - Failure + - Success + type: string + required: + - status + - retryAfterSeconds + - message + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterDeleteRequest: + description: BeforeClusterDeleteRequest is the request of the hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + cluster: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.Cluster' + default: {} + description: The cluster object the lifecycle hook corresponds to. + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + required: + - cluster + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterDeleteResponse: + description: BeforeClusterDeleteResponse is the response of BeforeClusterDelete + hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + message: + default: "" + description: A human-readable description of the status of the call. + type: string + retryAfterSeconds: + default: 0 + description: RetryAfterSeconds when set to a non-zero signifies that the + hook needs to be retried at a future time. + format: int32 + type: integer + status: + default: "" + description: |- + Status of the call. One of "Success" or "Failure". + + Possible enum values: + - `"Failure"` + - `"Success"` + enum: + - Failure + - Success + type: string + required: + - status + - retryAfterSeconds + - message + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterUpgradeRequest: + description: BeforeClusterUpgradeRequest is the request of the hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + cluster: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.api.v1beta1.Cluster' + default: {} + description: The cluster object the lifecycle hook corresponds to. + fromKubernetesVersion: + default: "" + description: The current version of the cluster. + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + toKubernetesVersion: + default: "" + description: The target version of upgrade. + type: string + required: + - cluster + - fromKubernetesVersion + - toKubernetesVersion + type: object + sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterUpgradeResponse: + description: BeforeClusterUpgradeResponse is the response of BeforeClusterUpgrade + hook. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + message: + default: "" + description: A human-readable description of the status of the call. + type: string + retryAfterSeconds: + default: 0 + description: RetryAfterSeconds when set to a non-zero signifies that the + hook needs to be retried at a future time. + format: int32 + type: integer + status: + default: "" + description: |- + Status of the call. One of "Success" or "Failure". + + Possible enum values: + - `"Failure"` + - `"Success"` + enum: + - Failure + - Success + type: string + required: + - status + - retryAfterSeconds + - message + type: object +info: + description: Open API specification for Cluster API Runtime SDK + license: + name: Apache 2.0 + url: http://www.apache.org/licenses/LICENSE-2.0.html + title: Cluster API - Runtime SDK + version: v1.2.0 +openapi: 3.0.0 +paths: + /hooks.runtime.cluster.x-k8s.io/v1alpha1/afterclusterupgrade/{name}: + post: + description: TThis hook is called after the Cluster, control plane and workers + have been upgraded to the version specified in spec.topology.version. Runtime + Extensions implementers can use this hook to execute post-upgrade tasks. This + hook does not block any further changes or upgrades to the Cluster. + operationId: hooksRuntimeClusterV1alpha1AfterClusterUpgrade + parameters: + - in: path + name: name + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterClusterUpgradeRequest' + responses: + "200": + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterClusterUpgradeResponse' + description: OK + summary: Called after the Cluster finished upgrade + tags: + - Lifecycle Hooks + /hooks.runtime.cluster.x-k8s.io/v1alpha1/aftercontrolplaneinitialized/{name}: + post: + description: This hook is called after the ControlPlane for the Cluster is marked + as available for the first time. Runtime Extension implementers can use this + hook to execute tasks that are only possible once the Control Plane is available. + This hook does not block any further changes to the Cluster. + operationId: hooksRuntimeClusterV1alpha1AfterControlPlaneInitialized + parameters: + - in: path + name: name + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterControlPlaneInitializedRequest' + responses: + "200": + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterControlPlaneInitializedResponse' + description: OK + summary: Called after the Control Plane is available for the first time + tags: + - Lifecycle Hooks + /hooks.runtime.cluster.x-k8s.io/v1alpha1/aftercontrolplaneupgrade/{name}: + post: + description: This hook is called after the control plane has been upgraded to + the version specified in spec.topology.version, and immediately before the + new version is going to be propagated to the MachineDeployments existing in + the Cluster. Runtime Extension implementers can use this hook to execute post-upgrade + tasks and eventually block upgrades to workers. + operationId: hooksRuntimeClusterV1alpha1AfterControlPlaneUpgrade + parameters: + - in: path + name: name + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterControlPlaneUpgradeRequest' + responses: + "200": + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.AfterControlPlaneUpgradeResponse' + description: OK + summary: Called after the Control Plane finished upgrade + tags: + - Lifecycle Hooks + /hooks.runtime.cluster.x-k8s.io/v1alpha1/beforeclustercreate/{name}: + post: + description: This hook is called after the Cluster object has been created by + the user, immediately before all the objects which are part of a Cluster topology + are going to be created. Runtime Extension implementers can use this hook + to determine/prepare add-ons for the Cluster and block the creation of those + objects. + operationId: hooksRuntimeClusterV1alpha1BeforeClusterCreate + parameters: + - in: path + name: name + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterCreateRequest' + responses: + "200": + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterCreateResponse' + description: OK + summary: Called before Cluster topology is created + tags: + - Lifecycle Hooks + /hooks.runtime.cluster.x-k8s.io/v1alpha1/beforeclusterdelete/{name}: + post: + description: This hook is called after the Cluster has been deleted by the user, + and immediately before objects existing in the Cluster are going to be deleted. + Runtime Extension implementers can use this hook to execute cleanup tasks + and eventually block deletion of the Cluster and descendant objects until + everything is ready. + operationId: hooksRuntimeClusterV1alpha1BeforeClusterDelete + parameters: + - in: path + name: name + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterDeleteRequest' + responses: + "200": + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterDeleteResponse' + description: OK + summary: Called before the Cluster is deleted + tags: + - Lifecycle Hooks + /hooks.runtime.cluster.x-k8s.io/v1alpha1/beforeclusterupgrade/{name}: + post: + description: This hook is called after the Cluster object has been updated with + a new spec.topology.version by the user, and immediately before the new version + is going to be propagated to the control plane. Runtime Extension implementers + can use this hook to execute pre-upgrade tasks and block upgrades of the ControlPlane + and Workers. + operationId: hooksRuntimeClusterV1alpha1BeforeClusterUpgrade + parameters: + - in: path + name: name + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterUpgradeRequest' + responses: + "200": + content: + application/json: + schema: + $ref: '#/components/schemas/sigs.k8s.io.cluster-api.exp.runtime.hooks.api.v1alpha1.BeforeClusterUpgradeResponse' + description: OK + summary: Called before the Cluster begins upgrade + tags: + - Lifecycle Hooks diff --git a/docs/proposals/images/runtime-hooks/runtime-hooks.png b/docs/proposals/images/runtime-hooks/runtime-hooks.png new file mode 100644 index 000000000000..7153ee288aef Binary files /dev/null and b/docs/proposals/images/runtime-hooks/runtime-hooks.png differ