Skip to content

📖 update documentation #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Documentation

kcp-api-syncagent is a Kubernetes agent responsible for integrating external Kubernetes clusters.
The api-syncagent is a Kubernetes agent responsible for integrating external Kubernetes clusters.
It runs on a Kubernetes cluster, is configured with credentials to a kcp instance and will then
synchronize data out of kcp (i.e. out of kcp workspaces) onto the local cluster, and vice versa.

Expand Down
2 changes: 1 addition & 1 deletion docs/consuming-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ A Service provided by a Sync Agent should not be confused with a Kubernetes Serv
## Consuming a Service

To consume a service (or to make use of an `APIExport`) you have to create an `APIBinding` object
in the kcp workspace where the servie should be used. This section assumes that you are familiar
in the kcp workspace where the service should be used. This section assumes that you are familiar
with kcp on the command line and have the kcp kubectl plugin installed.

First you need to get the kubeconfig for accessing your kcp workspaces. Once you have set your
Expand Down
38 changes: 38 additions & 0 deletions docs/faq.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Frequently Asked Questions

## Can I run multiple Sync Agents on the same service cluster?

Yes, absolutely, however you must configure them properly:

A given `PublishedResource` must only ever be processed by a single Sync Agent Pod. The Helm chart
configures leader-election by default, so you can scale up to have Pods on stand-by if needed.

By default the Sync Agent will discover and process all `PublishedResources` in your cluster. Use
the `--published-resource-selector` (`publishedResourceSelector` in the Helm values.yaml) to
restrict an Agent to a subset of published resources.

## Can I synchronize multiple kcp setups onto the same service cluster?

Only if you have distinct API groups (and therefore also distinct `PublishedResources`) for them.
You cannot currently publish the same API group onto multiple kcp setups. See issue #13 for more
information.

## What happens when CRDs are updated?

At the moment, nothing. `APIResourceSchemas` in kcp are immutable and the Sync Agent currently does
not attempt to update existing schemas in an `APIExport`. If you add a _new_ CRD that you want to
publish, that's fine, it will be added to the `APIExport`. But changes to existing CRDs require
manual work.

To trigger an update:

* remove the `APIResourceSchema` from the `latestResourceSchemas`,
* delete the `APIResourceSchema` object in kcp,
* restart the api-syncagent

## Does the Sync Agent handle permission claims?

Only those required for its own operation. If you configure a namespaced resource to sync, it will
automatically add a claim for `namespaces` in kcp, plus it will add either `configmaps` or `secrets`
if related resources are configured in a `PublishedResource`. But you cannot specify additional
permissions claims.
183 changes: 131 additions & 52 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Getting Started with the Sync Agent

All that is necessary to run the Sync Agent is a running Kubernetes cluster (for testing you can use
[kind][kind]) [kcp][kcp] installation.
[kind][kind]) and a [kcp][kcp] installation.

## Prerequisites

Expand Down Expand Up @@ -31,7 +31,7 @@ of your choice:
# use the kcp kubeconfig
$ export KUBECONFIG=/path/to/kcp.kubeconfig

# nativagate to the workspace wher the APIExport should exist
# nativagate to the workspace where the APIExport should exist
$ kubectl ws :workspace:you:want:to:create:it

# create it
Expand Down Expand Up @@ -59,74 +59,153 @@ $ kubectl create secret generic kcp-kubeconfig \
--from-file "kubeconfig=admin.kubeconfig"
```

### Helm Chart Setup

The Sync Agent is shipped as a Helm chart and to install it, the next step is preparing a `values.yaml`
file for the Sync Agent Helm chart. We need to pass the target `APIExport`, a name for the Sync Agent
itself and a reference to the kubeconfig secret we just created.

```yaml
syncAgent:
# Required: the name of the APIExport in kcp that this Sync Agent is supposed to serve.
apiExportName: test.example.com

# Required: this Sync Agent's public name, will be shown in kcp, purely for informational purposes.
agentName: unique-test

# Required: Name of the Kubernetes Secret that contains a "kubeconfig" key, with the kubeconfig
# provided by kcp to access it.
kcpKubeconfig: kcp-kubeconfig

# Create additional RBAC on the service cluster. These rules depend somewhat on the Sync Agent
# configuration, but the following two rules are very common. If you configure the Sync Agent to
# only work with cluster-scoped objects, you do not need to grant it permissions to create
# namespaces, for example.
rbac:
createClusterRole: true
rules:
# in order to create APIResourceSchemas
- apiGroups:
- apiextensions.k8s.io
resources:
- customresourcedefinitions
verbs:
- get
- list
- watch
# so copies of remote objects can be placed in their target namespaces
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- list
- watch
- create
```
# Required: the name of the APIExport in kcp that this Sync Agent is supposed to serve.
apiExportName: test.example.com

In addition, it is important to create RBAC rules for the resources you want to publish. If you want
to publish the `Certificate` resource as created by cert-manager, you will need to append the
following ruleset:
# Required: This Agent's public name, purely for informational purposes.
# If not set, defaults to the Helm release name.
agentName: unique-test

```yaml
# so we can manage certificates
- apiGroups:
- cert-manager.io
resources:
- certificates
verbs:
- '*'
# Required: Name of the Kubernetes Secret that contains a "kubeconfig" key,
# with the kubeconfig provided by kcp to access it.
kcpKubeconfig: kcp-kubeconfig
```

Once this `values.yaml` file is prepared, install a recent development build of the Sync Agent:

```sh
helm install kcp-api-syncagent oci://github.com/kcp-dev/helm-charts/api-syncagent --version 9.9.9-9fc9a430d95f95f4b2210f91ef67b3ec153b5cab -f values.yaml -n kcp-system
helm repo add kcp https://kcp-dev.github.io/helm-charts
helm repo update

helm install kcp-api-syncagent kcp/api-syncagent \
--values values.yaml \
--namespace kcp-system
```

Two `kcp-api-syncagent` Pods should start in the `kcp-system` namespace. If they crash you will need to
identify the reason from container logs. A possible issue is that the provided kubeconfig does not
have permissions against the target kcp workspace.

### Service Cluster RBAC

The Sync Agent usually requires additional RBAC on the service cluster to function properly. The
Helm chart will automatically allow it to read CRDs, namespaces and Secrets, but depending on how
you configure your PublishedResources, additional permissions need to be created.

For example, if the Sync Agent is meant to create `Certificate` objects (defined by cert-manager),
you would need to grant it permissions on those:

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: 'api-syncagent:unique-test'
rules:
- apiGroups:
- cert-manager.io
resources:
- certificates
verbs:
- get
- list
- watch
- create
- update

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: 'api-syncagent:unique-test'
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: 'api-syncagent:unique-test'
subjects:
- kind: ServiceAccount
name: 'kcp-api-syncagent'
namespace: kcp-system
```

**NB:** Even though the PublishedResources might only create/update Certificates in a single namespace,
due to the inner workings of the Agent they will still be watched (cached) cluster-wide. So you can
tighten permissions on `create`/`update` operations to certain namespaces, but `watch` permissions
need to be granted cluster-wide.

### kcp RBAC

The Helm chart is installed on the service cluster and so cannot provision the necessary RBAC for
the Sync Agent within kcp. Usually whoever creates the `APIExport` is also responsible for creating
the RBAC rules that grant the Agent access.

The Sync Agent needs to

* manage its `APIExport`,
* manage `APIResourceSchemas` and
* access the virtual workspace for its `APIExport`.

This can be achieved by applying RBAC like this _in the workspace where the `APIExport` resides_:

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: api-syncagent-mango
rules:
# manage its APIExport
- apiGroups:
- apis.kcp.io
resources:
- apiexports
resourceNames:
- test.example.com
verbs:
- get
- list
- watch
- patch
- update
# manage APIResourceSchemas
- apiGroups:
- apis.kcp.io
resources:
- apiresourceschemas
verbs:
- get
- list
- watch
- create
# access the virtual workspace
- apiGroups:
- apis.kcp.io
resources:
- apiexports/content
resourceNames:
- test.example.com
verbs:
- '*'

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: api-syncagent-columbo:mango-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: api-syncagent-mango
subjects:
- kind: User
name: api-syncagent-mango
```

## Publish Resources

Once the Sync Agent Pods are up and running, you should be able to follow the
Expand Down