Skip to content

Commit 7b7cbed

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #206 from xing-yang/readme
Update README
2 parents ed82822 + 710319e commit 7b7cbed

File tree

1 file changed

+111
-33
lines changed

1 file changed

+111
-33
lines changed

‎README.md

+111-33
Original file line numberDiff line numberDiff line change
@@ -1,72 +1,150 @@
1+
[![Build Status](https://travis-ci.org/kubernetes-csi/external-snapshotter.svg?branch=master)](https://travis-ci.org/kubernetes-csi/external-snapshotter)
2+
13
# CSI Snapshotter
24

3-
The CSI external-snapshotter is part of Kubernetes implementation of [Container Storage Interface (CSI)](https://github.com/container-storage-interface/spec).
5+
The CSI snapshotter is part of Kubernetes implementation of [Container Storage Interface (CSI)](https://github.com/container-storage-interface/spec).
6+
7+
The volume snapshot feature supports CSI v1.0 and higher. It was introduced as an Alpha feature in Kubernetes v1.12 and has been promoted to an Beta feature in Kubernetes 1.17.
48

5-
The volume snapshot feature supports CSI v1.0 and it has been an Alpha feature in Kubernetes since v1.12.
69

710
## Overview
811

9-
CSI Snapshotter is an external controller that watches Kubernetes Snapshot CRD objects and triggers CreateSnapshot/DeleteSnapshot against a CSI endpoint. Full design can be found at Kubernetes proposal at [here](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/csi-snapshot.md)
12+
With the promotion of Volume Snapshot to beta, the feature is now enabled by default on standard Kubernetes deployments instead of being opt-in.
1013

11-
## Design
14+
The move of the Kubernetes Volume Snapshot feature to beta also means:
15+
* A revamp of volume snapshot APIs.
16+
* The CSI external-snapshotter sidecar is split into two controllers, a snapshot controller and a CSI external-snapshotter sidecar.
17+
18+
The snapshot controller is deployed by the Kubernetes distributions and is responsible for watching the VolumeSnapshot CRD objects and manges the creation and deletion lifecycle of snapshots.
19+
20+
The CSI external-snapshotter sidecar watches Kubernetes VolumeSnapshotContent CRD objects and triggers CreateSnapshot/DeleteSnapshot against a CSI endpoint.
21+
22+
Blog post for the beta feature can be found [here](https://kubernetes.io/blog/2019/12/09/kubernetes-1-17-feature-cis-volume-snapshot-beta)
1223

13-
External snapshotter follows [controller](https://github.com/kubernetes/community/blob/master/contributors/devel/controllers.md) pattern and uses informers to watch for `VolumeSnapshot` and `VolumeSnapshotContent` create/update/delete events. It filters out these objects with `Snapshotter==<CSI driver name>` specified in the associated VolumeSnapshotClass object and then processes these events in workqueues with exponential backoff.
1424

15-
### Snapshotter
25+
## Compatibility
1626

17-
Snapshotter talks to CSI over socket (/run/csi/socket by default, configurable by -csi-address). The snapshotter then:
27+
This information reflects the head of this branch.
1828

19-
* Discovers the supported snapshotter name by `GetDriverName` call.
29+
| Compatible with CSI Version | Container Image | Min K8s Version | Snapshot CRD version |
30+
| ------------------------------------------------------------------------------------------ | ----------------------------| --------------- | -------------------- |
31+
| [CSI Spec v1.0.0](https://github.com/container-storage-interface/spec/releases/tag/v1.0.0) | quay.io/k8scsi/csi-snapshotter | 1.17 | v1beta1 |
32+
| [CSI Spec v1.0.0](https://github.com/container-storage-interface/spec/releases/tag/v1.0.0) | quay.io/k8scsi/snapshot-controller | 1.17 | v1beta1 |
33+
34+
35+
## Feature Status
36+
37+
The `VolumeSnapshotDataSource` feature gate was introduced in Kubernetes 1.12 and it is enabled by default in Kubernetes 1.17 when the volume snapshot feature is promoted to beta.
38+
39+
40+
## Design
2041

21-
* Uses ControllerGetCapabilities for find out if CSI driver supports `ControllerServiceCapability_RPC_CREATE_DELETE_SNAPSHOT` and `ControllerServiceCapability_RPC_LIST_SNAPSHOTS` calls. Otherwise, the controller will not start.
42+
Both the snapshot controller and CSI external-snapshotter sidecar follow [controller](https://github.com/kubernetes/community/blob/master/contributors/devel/controllers.md) pattern and uses informers to watch for events. The snapshot controller watches for `VolumeSnapshot` and `VolumeSnapshotContent` create/update/delete events.
2243

23-
* Processes new/updated/deleted `VolumeSnapshots`: The snapshotter only processes `VolumeSnapshot` that has `snapshotter` specified in its `VolumeSnapshotClass` matches its driver name. The process workflow is as follows
24-
* If the snapshot status is `Ready`, the controller checks whether the snapshot and its content still binds correctly. If there is any problem with the binding (e.g., snapshot points to a non-exist snapshot content), update the snapshot status and emit event.
25-
* If the snapshot status is not ready, there are two cases.
26-
* `SnapshotContentName` is not empty: the controller verifies whether the snapshot content exists and also binds to the snapshot. If verification passes, the controller binds the snapshot and its content objects and marks it is ready. Otherwise, it updates the error status of the snapshot.
27-
* `SnapshotContentName` is set empty: the controller will first check whether there is already a content object which binds the snapshot correctly with snapshot uid (`VolumeSnapshotRef.UID`) specified. If so, the controller binds these two objects. Otherwise, the controller issues a create snapshot operation. Please note that if the error status shows that snapshot creation already failed before, it will not try to create snapshot again.
44+
The CSI external-snapshotter sidecar only watches for `VolumeSnapshotContent` create/update/delete events. It filters out these objects with `Driver==<CSI driver name>` specified in the associated VolumeSnapshotClass object and then processes these events in workqueues with exponential backoff.
45+
46+
The CSI external-snapshotter sidecar talks to CSI over socket (/run/csi/socket by default, configurable by -csi-address).
47+
48+
### Hightlights in the snapshot v1beta1 APIs
49+
50+
* DeletionPolicy is a required field in both VolumeSnapshotClass and VolumeSnapshotContent. This way the user has to explicitly specify it, leaving no room for confusion.
51+
* VolumeSnapshotSpec has a required Source field. Source may be either a PersistentVolumeClaimName (if dynamically provisioning a snapshot) or VolumeSnapshotContentName (if pre-provisioning a snapshot).
52+
* VolumeSnapshotContentSpec has a required Source field. This Source may be either a VolumeHandle (if dynamically provisioning a snapshot) or a SnapshotHandle (if pre-provisioning volume snapshots).
53+
* VolumeSnapshot contains a Status to indicate the current state of the volume snapshot. It has a field BoundVolumeSnapshotContentName to indicate the VolumeSnapshot object is bound to a VolumeSnapshotContent.
54+
* VolumeSnapshotContent contains a Status to indicate the current state of the volume snapshot content. It has a field SnapshotHandle to indicate that the VolumeSnapshotContent represents a snapshot on the storage system.
2855

29-
* Processes new/updated/deleted `VolumeSnapshotContents`: The snapshotter only processes `VolumeSnapshotContent` in which the CSI driver specified in the spec matches the controller's driver name.
30-
* If the `VolumeSnapshotRef` is set to nil, skip this content since it is not bound to any snapshot object.
31-
* Otherwise, the controller verifies whether the content object is correctly bound to a snapshot object. In case the `VolumeSnapshotRef.UID` is set but it does not match its snapshot object or snapshot no long exists, the content object and its associated snapshot will be deleted.
3256

3357
## Usage
3458

35-
### Running on command line
59+
The Volume Snapshot feature now depends on a new, volume snapshot controller in addition to the volume snapshot CRDs. Both the volume snapshot controller and the CRDs are independent of any CSI driver. Regardless of the number CSI drivers deployed on the cluster, there must be only one instance of the volume snapshot controller running and one set of volume snapshot CRDs installed per cluster.
3660

37-
For debugging, it is possible to run snapshotter on command line. For example,
61+
Therefore, it is strongly recommended that Kubernetes distributors bundle and deploy the controller and CRDs as part of their Kubernetes cluster management process (independent of any CSI Driver).
3862

39-
```bash
40-
csi-snapshotter -kubeconfig ~/.kube/config -v 5 -csi-address /run/csi/socket
41-
```
63+
If your cluster does not come pre-installed with the correct components, you may manually install these components by executing the following steps.
4264

43-
### Running in a statefulset
65+
Install Snapshot Beta CRDs:
66+
* kubectl create -f config/crd
67+
* https://github.com/kubernetes-csi/external-snapshotter/tree/master/config/crd
68+
* Do this once per cluster
4469

45-
It is necessary to create a new service account and give it enough privileges to run the snapshotter. We provide .yaml files that deploy for use together with the hostpath example driver. A real production deployment must customize them:
70+
Install Common Snapshot Controller:
71+
* kubectl create -f deploy/kubernetes/snapshot-controller
72+
* https://github.com/kubernetes-csi/external-snapshotter/tree/master/deploy/kubernetes/snapshot-controller
73+
* Do this once per cluster
4674

47-
```bash
48-
for i in $(find deploy/kubernetes -name '*.yaml'); do kubectl create -f $i; done
49-
```
75+
Install CSI Driver:
76+
* Follow instructions provided by your CSI Driver vendor.
77+
* Here is an example to install the sample hostpath CSI driver
78+
* kubectl create -f deploy/kubernetes/csi-snapshotter
79+
* https://github.com/kubernetes-csi/external-snapshotter/tree/master/deploy/kubernetes/csi-snapshotter
5080

51-
### Running with Leader Election
81+
### Snapshot controller command line options
82+
83+
#### Important optional arguments that are highly recommended to be used
84+
* `--leader-election`: Enables leader election. This is useful when there are multiple replicas of the same snapshot controller running for the same Kubernetes deployment. Only one of them may be active (=leader). A new leader will be re-elected when current leader dies or becomes unresponsive for ~15 seconds.
85+
86+
* `--leader-election-namespace <namespace>`: The namespace where the leader election resource exists. Defaults to the pod namespace if not set.
87+
88+
#### Other recognized arguments
89+
* `--kubeconfig <path>`: Path to Kubernetes client configuration that the snapshot controller uses to connect to Kubernetes API server. When omitted, default token provided by Kubernetes will be used. This option is useful only when the snapshot controller does not run as a Kubernetes pod, e.g. for debugging.
90+
91+
* `--resync-period <duration>`: Internal resync interval when the snapshot controller re-evaluates all existing `VolumeSnapshot` instances and tries to fulfill them, i.e. create / delete corresponding snapshots. It does not affect re-tries of failed calls! It should be used only when there is a bug in Kubernetes watch logic. Default is 60 seconds.
92+
93+
* `--version`: Prints current snapshot controller version and quits.
94+
95+
* All glog / klog arguments are supported, such as `-v <log level>` or `-alsologtostderr`.
96+
97+
### CSI external snapshotter sidecar command line options
98+
99+
#### Important optional arguments that are highly recommended to be used
100+
* `--csi-address <path to CSI socket>`: This is the path to the CSI driver socket inside the pod that the external-snapshotter container will use to issue CSI operations (`/run/csi/socket` is used by default).
101+
102+
* `--leader-election`: Enables leader election. This is useful when there are multiple replicas of the same external-snapshotter running for one CSI driver. Only one of them may be active (=leader). A new leader will be re-elected when current leader dies or becomes unresponsive for ~15 seconds.
103+
104+
* `--leader-election-namespace <namespace>`: The namespace where the leader election resource exists. Defaults to the pod namespace if not set.
105+
106+
* `--timeout <duration>`: Timeout of all calls to CSI driver. It should be set to value that accommodates majority of `CreateSnapshot`, `DeleteSnapshot`, and `ListSnapshots` calls. 1 minute is used by default.
107+
108+
* `snapshot-name-prefix`: Prefix to apply to the name of a created snapshot. Default is `snapshot`.
109+
110+
* `snapshot-name-uuid-length`: Length in characters for the generated uuid of a created snapshot. Defaults behavior is to NOT truncate.
111+
112+
#### Other recognized arguments
113+
* `--kubeconfig <path>`: Path to Kubernetes client configuration that the CSI external-snapshotter uses to connect to Kubernetes API server. When omitted, default token provided by Kubernetes will be used. This option is useful only when the external-snapshotter does not run as a Kubernetes pod, e.g. for debugging.
114+
115+
* `--resync-period <duration>`: Internal resync interval when the CSI external-snapshotter re-evaluates all existing `VolumeSnapshotContent` instances and tries to fulfill them, i.e. update / delete corresponding snapshots. It does not affect re-tries of failed CSI calls! It should be used only when there is a bug in Kubernetes watch logic. Default is 60 seconds.
116+
117+
* `--version`: Prints current CSI external-snapshotter version and quits.
118+
119+
* All glog / klog arguments are supported, such as `-v <log level>` or `-alsologtostderr`.
120+
121+
122+
## Upgrade from v1alpha1 to v1beta1
123+
124+
The change from v1alpha1 to v1beta1 snapshot APIs is not backward compatible.
125+
126+
If you have already deployed v1alpha1 snapshot APIs and external-snapshotter sidecar controller and want to upgrade to v1beta1, you need to do the following:
127+
* Delete volume snapshots created using v1alpha1 snapshot CRDs and external-snapshotter sidecar controller.
128+
* Uninstall v1alpha1 snapshot CRDs, external-snapshotter sidecar controller, and CSI driver.
129+
* Install v1beta1 snapshot CRDs, snapshot controller, CSI external-snapshotter sidecar and CSI driver.
52130

53-
If you want to run external-snapshotter with higher availability, you can enable resource based leader election. To enable this, set the following flags:
54-
```bash
55-
--leader-election=true
56-
```
57131

58132
## Testing
59133

60134
Running Unit Tests:
61135

62136
```bash
63-
go test -timeout 30s github.com/kubernetes-csi/external-snapshotter/pkg/controller
137+
go test -timeout 30s github.com/kubernetes-csi/external-snapshotter/pkg/common-controller
138+
139+
go test -timeout 30s github.com/kubernetes-csi/external-snapshotter/pkg/sidecar-controller
64140
```
65141

142+
66143
## Dependency Management
67144

68145
external-snapshotter uses [go modules](https://blog.golang.org/using-go-modules).
69146

147+
70148
## Community, discussion, contribution, and support
71149

72150
Learn how to engage with the Kubernetes community on the [community page](http://kubernetes.io/community/).

0 commit comments

Comments
 (0)