Error installing v0.21.1 crds

[jbpin]

Bug Report

What did you do?
kubectl apply -f https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.21.1/crds.yaml

What did you expect to see?
no error

What did you see instead? Under which circumstances?

customresourcedefinition.apiextensions.k8s.io/catalogsources.operators.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/installplans.operators.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/olmconfigs.operators.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/operatorconditions.operators.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/operatorgroups.operators.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/operators.operators.coreos.com unchanged
customresourcedefinition.apiextensions.k8s.io/subscriptions.operators.coreos.com unchanged
The CustomResourceDefinition "clusterserviceversions.operators.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

Environment

• operator-lifecycle-manager version: v0.21.1

• Kubernetes version information:
  v1.22.2

• Kubernetes cluster kind:
  Cloud instance

[jbpin]

I was able to install OLM by editing the script and use create instead of apply in the CRD step.

[exdx]

I believe this is a result of the last-provided-configuration annotation that gets appended to the object during an apply call, which doesn't get automatically appended during a create.

[javajon]

Experiencing the same error:

=== Kubernetes Status ===
[+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/bootstrap-controller ok
[+]poststarthook/rbac/bootstrap-roles ok
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
healthz check passed
Client Version: v1.23.4
Server Version: v1.23.4
NAME           STATUS   ROLES                  AGE    VERSION
controlplane   Ready    control-plane,master   2m3s   v1.23.4
node01         Ready    <none>                 102s   v1.23.4

$ VERSION=v0.21.1

$ curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/$VERSION/install.sh | bash -s $VERSION
customresourcedefinition.apiextensions.k8s.io/catalogsources.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/installplans.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/olmconfigs.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/operatorconditions.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/operatorgroups.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/operators.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/subscriptions.operators.coreos.com created
The CustomResourceDefinition "clusterserviceversions.operators.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

Works fine on the same host with previous version 0.20.0:

$ VERSION=v0.20.0

$ curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/$VERSION/install.sh | bash -s $VERSION
customresourcedefinition.apiextensions.k8s.io/catalogsources.operators.coreos.com configured
customresourcedefinition.apiextensions.k8s.io/clusterserviceversions.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/installplans.operators.coreos.com configured
customresourcedefinition.apiextensions.k8s.io/olmconfigs.operators.coreos.com configured
customresourcedefinition.apiextensions.k8s.io/operatorconditions.operators.coreos.com configured
customresourcedefinition.apiextensions.k8s.io/operatorgroups.operators.coreos.com configured
customresourcedefinition.apiextensions.k8s.io/operators.operators.coreos.com configured
customresourcedefinition.apiextensions.k8s.io/subscriptions.operators.coreos.com configured
customresourcedefinition.apiextensions.k8s.io/catalogsources.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/clusterserviceversions.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/installplans.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/olmconfigs.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/operatorconditions.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/operatorgroups.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/operators.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/subscriptions.operators.coreos.com condition met
namespace/olm created
namespace/operators created
serviceaccount/olm-operator-serviceaccount created
clusterrole.rbac.authorization.k8s.io/system:controller:operator-lifecycle-manager created
clusterrolebinding.rbac.authorization.k8s.io/olm-operator-binding-olm created
olmconfig.operators.coreos.com/cluster created
deployment.apps/olm-operator created
deployment.apps/catalog-operator created
clusterrole.rbac.authorization.k8s.io/aggregate-olm-edit created
clusterrole.rbac.authorization.k8s.io/aggregate-olm-view created
operatorgroup.operators.coreos.com/global-operators created
operatorgroup.operators.coreos.com/olm-operators created
clusterserviceversion.operators.coreos.com/packageserver created
catalogsource.operators.coreos.com/operatorhubio-catalog created
Waiting for deployment "olm-operator" rollout to finish: 0 of 1 updated replicas are available...
deployment "olm-operator" successfully rolled out
deployment "catalog-operator" successfully rolled out
Package server phase: Installing
Package server phase: Succeeded
deployment "packageserver" successfully rolled out

$ kubectl get deployments -n olm
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
catalog-operator   1/1     1            1           20s
olm-operator       1/1     1            1           20s
packageserver      2/2     2            2           14s

[exdx]

We have updated the install.sh script to use kubectl create when applying objects, this prevents the last-applied-configuration annotation from being appended to the CRD object. This will enable those installing OLM on a fresh cluster to be able to install successfully.

Locally, another option is to simply use --server-side=true with apply when applying the OLM CRDs. Server-side apply is GA on Kubernetes 1.22.

The install.sh is configured to work only on a cluster where OLM is not previously installed. Unfortunately, there is not currently a supported way to update OLM itself on-cluster -- this is being tracked in #2695.

This is a short term fix -- the longer term solution is to reduce the depth of the OpenAPI specs that get generated when generating OLM CRDs so that the CRDs are slimmer and do not cause size issues.