[question] should I manually apply CRD after upgrade to 2.0.0?

[jichenjc]

previous in cloud-provider-openstack we used 1.2.2 version seems we don't need apply CRD from snapshoter

but when I upgrade to 2.0.1, I start to see logs
kubernetes/cloud-provider-openstack#974

root@jck8s4:~/go/src/github.com/cloud-provider-openstack# kubectl logs csi-cinder-nodeplugin-6w6gp -n kube-system -c csi-snapshotter | head
I0317 09:09:22.722966       1 main.go:89] Version: v2.0.1-0-g69da5b8d
I0317 09:09:22.724607       1 connection.go:153] Connecting to unix:///var/lib/csi/sockets/pluginproxy/csi.sock
W0317 09:09:24.613018       1 metrics.go:142] metrics endpoint will not be started because `metrics-address` was not specified.
I0317 09:09:24.613037       1 common.go:111] Probing CSI driver for readiness
I0317 09:09:24.925491       1 snapshot_controller_base.go:115] Starting CSI snapshotter
E0317 09:09:24.934291       1 reflector.go:156] github.com/kubernetes-csi/external-snapshotter/pkg/client/informers/externalversions/factory.go:117: Failed to list *v1beta1.VolumeSnapshotContent: volumesnapshotcontents.snapshot.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:csi-cinder-node-sa" cannot list resource "volumesnapshotcontents" in API group "snapshot.storage.k8s.io" at the cluster scope
E0317 09:09:24.934512       1 reflector.go:156] github.com/kubernetes-csi/external-snapshotter/pkg/client/informers/externalversions/factory.go:117: Failed to list *v1beta1.VolumeSnapshotClass: volumesnapshotclasses.snapshot.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:csi-cinder-node-sa" cannot list resource "volumesnapshotclasses" in API group "snapshot.storage.k8s.io" at the cluster scope
E0317 09:09:25.935746       1 reflector.go:156] github.com/kubernetes-csi/external-snapshotter/pkg/client/informers/externalversions/factory.go:117: Failed to list *v1beta1.VolumeSnapshotContent: volumesnapshotcontents.snapshot.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:csi-cinder-node-sa" cannot list resource "volumesnapshotcontents" in API group "snapshot.storage.k8s.io" at the cluster scope
E0317 09:09:25.936787       1 reflector.go:156] github.com/kubernetes-csi/external-snapshotter/pkg/client/informers/externalversions/factory.go:117: Failed to list *v1beta1.VolumeSnapshotClass: volumesnapshotclasses.snapshot.storage.k8s.io is forbidden: User "system:serviceaccount:kube-system:csi-cinder-node-sa" cannot list resource "volumesnapshotclasses" in API group "snapshot.storage.k8s.io" at the cluster scope

I applied config/crd/*.yaml in this repo then everything is fine
should I manually apply this or I have some configuration error when upgrade to 2.0.1? thanks

[zhucan]

@jichenjc You need to delete the old CRD and apply the "config/crd/*.yaml".

[jichenjc]

my k8s is fresh new env through hack command ..

ALLOW_PRIVILEGED=true RUNTIME_CONFIG="storage.k8s.io/v1=true" LOG_LEVEL=5 hack/local-up-cluster.sh

previously I don't need create CRD and now I need create CRD , correct? and is there any links so that I can apply on the fly?
otherwise I need download source code and apply? which mean it might out of sync... thanks

[zhucan]

Before the v2.0.0（it's alpha）， we needn't to create crd，Because when the snapshotter sidecar running， it will create the crd in default. After the v2.0.0（it's beta）, we should create the crd by the yaml, If you k8s cluster has old crd, you need to delete it.

[zhucan]

@jichenjc Take a look at this, #147

[jichenjc]

perfect, that's what I need , thanks a lot

the only question is where to apply those CRDs? usually you can apply on line ,but apply the master branch directly is not a good idea ...
https://github.com/kubernetes-csi/external-snapshotter/releases doesn't have this kind of CRD

[jichenjc]

Sorry, I mean follwing is what I need

Before the v2.0.0（it's alpha）， we needn't to create crd，Because when the snapshotter sidecar running， it will create the crd in default. After the v2.0.0（it's beta）, we should create the crd by the yaml, If you k8s cluster has old crd, you need to delete it.

#147 seems not related because https://github.com/kubernetes-csi/external-snapshotter/releases told us v2.0.0 is Jan this year while the question #147 is July last year

[zhucan]

What's version do you want to use?

[jichenjc]

latest one.. v2.0.1 ,thanks a lot

[zhucan]

You should use there yaml for apply "https://github.com/kubernetes-csi/external-snapshotter/tree/v2.0.1/config/crd"

[jichenjc]

ok, got it ,thanks~

[jichenjc]

could you please let me know whether we have plan to put CRD into release?
otherwise ,I will take https://github.com/kubernetes-csi/external-snapshotter/tree/v2.0.1/config/crd into our doc..

[jichenjc]

kubectl apply -f https://github.com/kubernetes-csi/external-snapshotter/tree/v2.0.1/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
error: error parsing https://github.com/kubernetes-csi/external-snapshotter/tree/v2.0.1/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml: error converting YAML to JSON: yaml: line 114: mapping values are not allowed in this context

download and it contains html ..

[yuxiangqian]

@jichenjc on top of what @zhucan has pointed out, in VolumeSnapshot BETA, there will be two controllers in your cluster (in Alpha it was only one). Details could be found in here. In short:

1. snapshot controller, which is per-cluster. Its responsible for managing VolumeSnapshot and VSC binding, status update etc. CRDs COULD be installed along with this controller.
2. sidecar controller, which is per-storage-vendor with CSI driver. It's responsible for snapshot creation/deletion via CSI calls.

looks to me you were only trying to install the sidecar controller. is the snapshot controller available (see addon)?

[jichenjc]

@yuxiangqian I want to install both
kubernetes/cloud-provider-openstack#974
is the PR that do the upgrade to beta version, it use both
https://github.com/kubernetes/cloud-provider-openstack/pull/974/files#diff-b50205da73c11460aff7f62130649d99R61

https://github.com/kubernetes/cloud-provider-openstack/pull/974/files#diff-471358b5b38c3faa372e98659bbb1c4bR45

the CRD need to be created (as @zhucan pointed out, alpha doesn't need so) ,I am looking for the crds ,seems the addon (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/volumesnapshots) is the answer? thanks

[xing-yang]

@jichenjc To use snapshot v2.0.1, you can install CRDs from the following location:
https://github.com/kubernetes-csi/external-snapshotter/tree/release-2.0/config/crd

The addon is for in-tree testing purpose.

[jichenjc]

@xing-yang thanks , as previous mentioned, I understand I can download file from given places
but it's html format now.. not convenient for end user (e.g I need document user how to apply thoes CRD)

# kubectl apply -f https://github.com/kubernetes-csi/external-snapshotter/tree/release-2.0/config/crd
error: error parsing https://github.com/kubernetes-csi/external-snapshotter/tree/release-2.0/config/crd: error converting YAML to JSON: yaml: line 114: mapping values are not allowed in this context

[xing-yang]

@jichenjc Please check the usage section of README: https://github.com/kubernetes-csi/external-snapshotter/blob/master/README.md#usage

Snapshot controller should not install the CRDs. It should be the k8s distro that deploys the CRDs and the snapshot controller.

[jichenjc]

@xing-yang thanks, I will read this and get back here later on

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.