diff --git a/docs/README.md b/docs/README.md index a46dc2a..2120864 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,6 +1,6 @@ # Documentation -kcp-api-syncagent is a Kubernetes agent responsible for integrating external Kubernetes clusters. +The api-syncagent is a Kubernetes agent responsible for integrating external Kubernetes clusters. It runs on a Kubernetes cluster, is configured with credentials to a kcp instance and will then synchronize data out of kcp (i.e. out of kcp workspaces) onto the local cluster, and vice versa. diff --git a/docs/consuming-services.md b/docs/consuming-services.md index 372c4e1..1dc34f9 100644 --- a/docs/consuming-services.md +++ b/docs/consuming-services.md @@ -19,7 +19,7 @@ A Service provided by a Sync Agent should not be confused with a Kubernetes Serv ## Consuming a Service To consume a service (or to make use of an `APIExport`) you have to create an `APIBinding` object -in the kcp workspace where the servie should be used. This section assumes that you are familiar +in the kcp workspace where the service should be used. This section assumes that you are familiar with kcp on the command line and have the kcp kubectl plugin installed. First you need to get the kubeconfig for accessing your kcp workspaces. Once you have set your diff --git a/docs/faq.md b/docs/faq.md new file mode 100644 index 0000000..63e2a61 --- /dev/null +++ b/docs/faq.md @@ -0,0 +1,38 @@ +# Frequently Asked Questions + +## Can I run multiple Sync Agents on the same service cluster? + +Yes, absolutely, however you must configure them properly: + +A given `PublishedResource` must only ever be processed by a single Sync Agent Pod. The Helm chart +configures leader-election by default, so you can scale up to have Pods on stand-by if needed. + +By default the Sync Agent will discover and process all `PublishedResources` in your cluster. Use +the `--published-resource-selector` (`publishedResourceSelector` in the Helm values.yaml) to +restrict an Agent to a subset of published resources. + +## Can I synchronize multiple kcp setups onto the same service cluster? + +Only if you have distinct API groups (and therefore also distinct `PublishedResources`) for them. +You cannot currently publish the same API group onto multiple kcp setups. See issue #13 for more +information. + +## What happens when CRDs are updated? + +At the moment, nothing. `APIResourceSchemas` in kcp are immutable and the Sync Agent currently does +not attempt to update existing schemas in an `APIExport`. If you add a _new_ CRD that you want to +publish, that's fine, it will be added to the `APIExport`. But changes to existing CRDs require +manual work. + +To trigger an update: + +* remove the `APIResourceSchema` from the `latestResourceSchemas`, +* delete the `APIResourceSchema` object in kcp, +* restart the api-syncagent + +## Does the Sync Agent handle permission claims? + +Only those required for its own operation. If you configure a namespaced resource to sync, it will +automatically add a claim for `namespaces` in kcp, plus it will add either `configmaps` or `secrets` +if related resources are configured in a `PublishedResource`. But you cannot specify additional +permissions claims. diff --git a/docs/getting-started.md b/docs/getting-started.md index 835a55f..128a338 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1,7 +1,7 @@ # Getting Started with the Sync Agent All that is necessary to run the Sync Agent is a running Kubernetes cluster (for testing you can use -[kind][kind]) [kcp][kcp] installation. +[kind][kind]) and a [kcp][kcp] installation. ## Prerequisites @@ -31,7 +31,7 @@ of your choice: # use the kcp kubeconfig $ export KUBECONFIG=/path/to/kcp.kubeconfig -# nativagate to the workspace wher the APIExport should exist +# nativagate to the workspace where the APIExport should exist $ kubectl ws :workspace:you:want:to:create:it # create it @@ -59,74 +59,153 @@ $ kubectl create secret generic kcp-kubeconfig \ --from-file "kubeconfig=admin.kubeconfig" ``` +### Helm Chart Setup + The Sync Agent is shipped as a Helm chart and to install it, the next step is preparing a `values.yaml` file for the Sync Agent Helm chart. We need to pass the target `APIExport`, a name for the Sync Agent itself and a reference to the kubeconfig secret we just created. ```yaml -syncAgent: - # Required: the name of the APIExport in kcp that this Sync Agent is supposed to serve. - apiExportName: test.example.com - - # Required: this Sync Agent's public name, will be shown in kcp, purely for informational purposes. - agentName: unique-test - - # Required: Name of the Kubernetes Secret that contains a "kubeconfig" key, with the kubeconfig - # provided by kcp to access it. - kcpKubeconfig: kcp-kubeconfig - - # Create additional RBAC on the service cluster. These rules depend somewhat on the Sync Agent - # configuration, but the following two rules are very common. If you configure the Sync Agent to - # only work with cluster-scoped objects, you do not need to grant it permissions to create - # namespaces, for example. - rbac: - createClusterRole: true - rules: - # in order to create APIResourceSchemas - - apiGroups: - - apiextensions.k8s.io - resources: - - customresourcedefinitions - verbs: - - get - - list - - watch - # so copies of remote objects can be placed in their target namespaces - - apiGroups: - - "" - resources: - - namespaces - verbs: - - get - - list - - watch - - create -``` +# Required: the name of the APIExport in kcp that this Sync Agent is supposed to serve. +apiExportName: test.example.com -In addition, it is important to create RBAC rules for the resources you want to publish. If you want -to publish the `Certificate` resource as created by cert-manager, you will need to append the -following ruleset: +# Required: This Agent's public name, purely for informational purposes. +# If not set, defaults to the Helm release name. +agentName: unique-test -```yaml - # so we can manage certificates - - apiGroups: - - cert-manager.io - resources: - - certificates - verbs: - - '*' +# Required: Name of the Kubernetes Secret that contains a "kubeconfig" key, +# with the kubeconfig provided by kcp to access it. +kcpKubeconfig: kcp-kubeconfig ``` Once this `values.yaml` file is prepared, install a recent development build of the Sync Agent: ```sh -helm install kcp-api-syncagent oci://github.com/kcp-dev/helm-charts/api-syncagent --version 9.9.9-9fc9a430d95f95f4b2210f91ef67b3ec153b5cab -f values.yaml -n kcp-system +helm repo add kcp https://kcp-dev.github.io/helm-charts +helm repo update + +helm install kcp-api-syncagent kcp/api-syncagent \ + --values values.yaml \ + --namespace kcp-system ``` Two `kcp-api-syncagent` Pods should start in the `kcp-system` namespace. If they crash you will need to identify the reason from container logs. A possible issue is that the provided kubeconfig does not have permissions against the target kcp workspace. +### Service Cluster RBAC + +The Sync Agent usually requires additional RBAC on the service cluster to function properly. The +Helm chart will automatically allow it to read CRDs, namespaces and Secrets, but depending on how +you configure your PublishedResources, additional permissions need to be created. + +For example, if the Sync Agent is meant to create `Certificate` objects (defined by cert-manager), +you would need to grant it permissions on those: + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: 'api-syncagent:unique-test' +rules: + - apiGroups: + - cert-manager.io + resources: + - certificates + verbs: + - get + - list + - watch + - create + - update + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: 'api-syncagent:unique-test' +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: 'api-syncagent:unique-test' +subjects: + - kind: ServiceAccount + name: 'kcp-api-syncagent' + namespace: kcp-system +``` + +**NB:** Even though the PublishedResources might only create/update Certificates in a single namespace, +due to the inner workings of the Agent they will still be watched (cached) cluster-wide. So you can +tighten permissions on `create`/`update` operations to certain namespaces, but `watch` permissions +need to be granted cluster-wide. + +### kcp RBAC + +The Helm chart is installed on the service cluster and so cannot provision the necessary RBAC for +the Sync Agent within kcp. Usually whoever creates the `APIExport` is also responsible for creating +the RBAC rules that grant the Agent access. + +The Sync Agent needs to + +* manage its `APIExport`, +* manage `APIResourceSchemas` and +* access the virtual workspace for its `APIExport`. + +This can be achieved by applying RBAC like this _in the workspace where the `APIExport` resides_: + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: api-syncagent-mango +rules: + # manage its APIExport + - apiGroups: + - apis.kcp.io + resources: + - apiexports + resourceNames: + - test.example.com + verbs: + - get + - list + - watch + - patch + - update + # manage APIResourceSchemas + - apiGroups: + - apis.kcp.io + resources: + - apiresourceschemas + verbs: + - get + - list + - watch + - create + # access the virtual workspace + - apiGroups: + - apis.kcp.io + resources: + - apiexports/content + resourceNames: + - test.example.com + verbs: + - '*' + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: api-syncagent-columbo:mango-system +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: api-syncagent-mango +subjects: + - kind: User + name: api-syncagent-mango +``` + ## Publish Resources Once the Sync Agent Pods are up and running, you should be able to follow the