Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ auth: use synthetic user/group when service account is not defined #1816

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/operator-controller/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -302,7 +302,7 @@ func run() error {
return err
}
tokenGetter := authentication.NewTokenGetter(coreClient, authentication.WithExpirationDuration(1*time.Hour))
clientRestConfigMapper := action.ServiceAccountRestConfigMapper(tokenGetter)
clientRestConfigMapper := action.ClusterExtensionUserRestConfigMapper(tokenGetter)

cfgGetter, err := helmclient.NewActionConfigGetter(mgr.GetConfig(), mgr.GetRESTMapper(),
helmclient.StorageDriverMapper(action.ChunkedStorageDriverMapper(coreClient, mgr.GetAPIReader(), cfg.systemNamespace)),
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# kustomization file for secure OLMv1
# DO NOT ADD A NAMESPACE HERE
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../../base/operator-controller
- ../../../base/common
components:
- ../../../components/tls/operator-controller

patches:
- target:
kind: Deployment
name: operator-controller-controller-manager
path: patches/enable-featuregate.yaml
- target:
kind: ClusterRole
name: operator-controller-manager-role
path: patches/impersonate-perms.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# enable synthetic-user feature gate
- op: add
path: /spec/template/spec/containers/0/args/-
value: "--feature-gates=SyntheticPermissions=true"
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# enable synthetic-user feature gate
- op: add
path: /rules/-
value:
apiGroups:
- ""
resources:
- groups
- users
verbs:
- impersonate
133 changes: 133 additions & 0 deletions docs/draft/howto/use-synthetic-permissions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
## Synthetic User Permissions

!!! note
This feature is still in *alpha* the `SyntheticPermissions` feature-gate must be enabled to make use of it.
See the instructions below on how to enable it.

Synthetic user permissions enables fine-grained configuration of ClusterExtension management client RBAC permissions.
User can not only configure RBAC permissions governing the management across all ClusterExtensions, but also on a
case-by-case basis.

### Update OLM to enable Feature

```terminal title=Enable SyntheticPermissions feature
kubectl kustomize config/overlays/featuregate/synthetic-user-permissions | kubectl apply -f -
```

```terminal title=Wait for rollout to complete
kubectl rollout status -n olmv1-system deployment/operator-controller-controller-manager
```

### How does it work?

When managing a ClusterExtension, OLM will assume the identity of user "olm:clusterextensions:<clusterextension-name>"
and group "olm:clusterextensions" limiting Kubernetes API access scope to those defined for this user and group. These
users and group do not exist beyond being defined in Cluster/RoleBinding(s) and can only be impersonated by clients with
`impersonate` verb permissions on the `users` and `groups` resources.

### Demo

[![asciicast](https://asciinema.org/a/Jbtt8nkV8Dm7vriHxq7sxiVvi.svg)](https://asciinema.org/a/Jbtt8nkV8Dm7vriHxq7sxiVvi)

#### Examples:

##### ClusterExtension management as cluster-admin

To enable ClusterExtensions management as cluster-admin, bind the `cluster-admin` cluster role to the `olm:clusterextensions`
group:

```
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: clusterextensions-group-admin-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: Group
name: "olm:clusterextensions"
```

##### Scoped olm:clusterextension group + Added perms on specific extensions

Give ClusterExtension management group broad permissions to manage ClusterExtensions denying potentially dangerous
permissions such as being able to read cluster wide secrets:

```
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: clusterextension-installer
rules:
- apiGroups: [ olm.operatorframework.io ]
resources: [ clusterextensions/finalizers ]
verbs: [ update ]
- apiGroups: [ apiextensions.k8s.io ]
resources: [ customresourcedefinitions ]
verbs: [ create, list, watch, get, update, patch, delete ]
- apiGroups: [ rbac.authorization.k8s.io ]
resources: [ clusterroles, roles, clusterrolebindings, rolebindings ]
verbs: [ create, list, watch, get, update, patch, delete ]
- apiGroups: [""]
resources: [configmaps, endpoints, events, pods, pod/logs, serviceaccounts, services, services/finalizers, namespaces, persistentvolumeclaims]
verbs: ['*']
- apiGroups: [apps]
resources: [ '*' ]
verbs: ['*']
- apiGroups: [ batch ]
resources: [ '*' ]
verbs: [ '*' ]
- apiGroups: [ networking.k8s.io ]
resources: [ '*' ]
verbs: [ '*' ]
- apiGroups: [authentication.k8s.io]
resources: [tokenreviews, subjectaccessreviews]
verbs: [create]
```

```
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: clusterextension-installer-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: clusterextension-installer
subjects:
- kind: Group
name: "olm:clusterextensions"
```

Give a specific ClusterExtension secrets access, maybe even on specific namespaces:

```
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: clusterextension-privileged
rules:
- apiGroups: [""]
resources: [secrets]
verbs: ['*']
```

```
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: clusterextension-privileged-binding
namespace: <some namespace>
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: clusterextension-privileged
subjects:
- kind: User
name: "olm:clusterextensions:argocd-operator"
```

Note: In this example the ClusterExtension user (or group) will still need to be updated to be able to manage
the CRs coming from the argocd operator. Some look ahead and RBAC permission wrangling will still be required.
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: olm.operatorframework.io/v1
kind: ClusterExtension
metadata:
name: argocd-operator
spec:
namespace: argocd-system
serviceAccount:
name: "olm.synthetic-user"
source:
sourceType: Catalog
catalog:
packageName: argocd-operator
version: 0.6.0
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: clusterextensions-group-admin-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: Group
name: "olm:clusterextensions"
30 changes: 30 additions & 0 deletions hack/demo/synthetic-user-cluster-admin-demo.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/usr/bin/env bash

#
# Welcome to the SingleNamespace install mode demo
#
trap "trap - SIGTERM && kill -- -$$" SIGINT SIGTERM EXIT

# enable 'SyntheticPermissions' feature
kubectl kustomize config/overlays/featuregate/synthetic-user-permissions | kubectl apply -f -

# wait for operator-controller to become available
kubectl rollout status -n olmv1-system deployment/operator-controller-controller-manager

# create install namespace
kubectl create ns argocd-system

# give cluster extension group cluster admin privileges - all cluster extensions installer users will be cluster admin
bat --style=plain ${DEMO_RESOURCE_DIR}/synthetic-user-perms/cegroup-admin-binding.yaml

# apply cluster role binding
kubectl apply -f ${DEMO_RESOURCE_DIR}/synthetic-user-perms/cegroup-admin-binding.yaml

# install cluster extension - for now .spec.serviceAccount = "olm.synthetic-user"
bat --style=plain ${DEMO_RESOURCE_DIR}/synthetic-user-perms/argocd-clusterextension.yaml

# apply cluster extension
kubectl apply -f ${DEMO_RESOURCE_DIR}/synthetic-user-perms/argocd-clusterextension.yaml

# wait for cluster extension installation to succeed
kubectl wait --for=condition=Installed clusterextension/argocd-operator --timeout="60s"
41 changes: 40 additions & 1 deletion internal/operator-controller/action/restconfig.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,52 @@ import (

"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/rest"
"k8s.io/client-go/transport"
"sigs.k8s.io/controller-runtime/pkg/client"

ocv1 "github.com/operator-framework/operator-controller/api/v1"
"github.com/operator-framework/operator-controller/internal/operator-controller/authentication"
"github.com/operator-framework/operator-controller/internal/operator-controller/features"
)

func ServiceAccountRestConfigMapper(tokenGetter *authentication.TokenGetter) func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error) {
const syntheticServiceAccountName = "olm.synthetic-user"

type clusterExtensionRestConfigMapper struct {
saRestConfigMapper func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error)
synthUserRestConfigMapper func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error)
}

func (m *clusterExtensionRestConfigMapper) mapper() func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error) {
synthAuthFeatureEnabled := features.OperatorControllerFeatureGate.Enabled(features.SyntheticPermissions)
return func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error) {
cExt := o.(*ocv1.ClusterExtension)
if synthAuthFeatureEnabled && cExt.Spec.ServiceAccount.Name == syntheticServiceAccountName {
return m.synthUserRestConfigMapper(ctx, o, c)
}
return m.saRestConfigMapper(ctx, o, c)
}
}

func ClusterExtensionUserRestConfigMapper(tokenGetter *authentication.TokenGetter) func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we pass a enableSyntheticUserAuthentication function parameter so that we can reference the feature gate only in main.go?

Copy link
Contributor

@perdasilva perdasilva Mar 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is that a general goal that we have that FGs should only be referenced in main? I might need to update the Single-OwnNamespace FG if that's the case. My only worry about it is that if it's only checked somewhere down the stack, we end up having to thread it all the way down, which could be painful. What is the value of having it in main? In the end I end up searching the code for usages of the FG anyway. Is it helpful in other contexts as well?

m := &clusterExtensionRestConfigMapper{
saRestConfigMapper: serviceAccountRestConfigMapper(tokenGetter),
synthUserRestConfigMapper: syntheticUserRestConfigMapper(),
}
return m.mapper()
}

func syntheticUserRestConfigMapper() func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error) {
return func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error) {
cExt := o.(*ocv1.ClusterExtension)
cc := rest.CopyConfig(c)
cc.Wrap(func(rt http.RoundTripper) http.RoundTripper {
return transport.NewImpersonatingRoundTripper(authentication.SyntheticImpersonationConfig(*cExt), rt)
})
return cc, nil
}
}

func serviceAccountRestConfigMapper(tokenGetter *authentication.TokenGetter) func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error) {
return func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error) {
cExt := o.(*ocv1.ClusterExtension)
saKey := types.NamespacedName{
Expand Down
92 changes: 92 additions & 0 deletions internal/operator-controller/action/restconfig_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
package action

import (
"context"
"testing"

"github.com/stretchr/testify/require"
"k8s.io/client-go/rest"
featuregatetesting "k8s.io/component-base/featuregate/testing"
"sigs.k8s.io/controller-runtime/pkg/client"

ocv1 "github.com/operator-framework/operator-controller/api/v1"
"github.com/operator-framework/operator-controller/internal/operator-controller/features"
)

const (
saAccountWrapper = "service account wrapper"
synthUserWrapper = "synthetic user wrapper"
)

func fakeRestConfigWrapper() clusterExtensionRestConfigMapper {
// The rest config's host field is artificially used to differentiate between the wrappers
return clusterExtensionRestConfigMapper{
saRestConfigMapper: func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error) {
return &rest.Config{
Host: saAccountWrapper,
}, nil
},
synthUserRestConfigMapper: func(ctx context.Context, o client.Object, c *rest.Config) (*rest.Config, error) {
return &rest.Config{
Host: synthUserWrapper,
}, nil
},
}
}

func TestMapper_SyntheticPermissionsEnabled(t *testing.T) {
featuregatetesting.SetFeatureGateDuringTest(t, features.OperatorControllerFeatureGate, features.SyntheticPermissions, true)

for _, tc := range []struct {
description string
serviceAccountName string
expectedMapper string
fgEnabled bool
}{
{
description: "user service account wrapper if extension service account is _not_ called olm.synthetic-user",
serviceAccountName: "not.olm.synthetic-user",
expectedMapper: saAccountWrapper,
fgEnabled: true,
}, {
description: "user synthetic user wrapper is extension service account is called olm.synthetic-user",
serviceAccountName: "olm.synthetic-user",
expectedMapper: synthUserWrapper,
fgEnabled: true,
},
} {
t.Run(tc.description, func(t *testing.T) {
m := fakeRestConfigWrapper()
mapper := m.mapper()
ext := &ocv1.ClusterExtension{
Spec: ocv1.ClusterExtensionSpec{
ServiceAccount: ocv1.ServiceAccountReference{
Name: tc.serviceAccountName,
},
},
}
cfg, err := mapper(context.Background(), ext, &rest.Config{})
require.NoError(t, err)

// The rest config's host field is artificially used to differentiate between the wrappers
require.Equal(t, tc.expectedMapper, cfg.Host)
})
}
}

func TestMapper_SyntheticPermissionsDisabled(t *testing.T) {
m := fakeRestConfigWrapper()
mapper := m.mapper()
ext := &ocv1.ClusterExtension{
Spec: ocv1.ClusterExtensionSpec{
ServiceAccount: ocv1.ServiceAccountReference{
Name: "olm.synthetic-user",
},
},
}
cfg, err := mapper(context.Background(), ext, &rest.Config{})
require.NoError(t, err)

// The rest config's host field is artificially used to differentiate between the wrappers
require.Equal(t, saAccountWrapper, cfg.Host)
}
Loading
Loading