Skip to content

Commit 7a41ef4

Browse files
carlbraganzacarlbraganza
committed
Documentation on the external-snaphot-metadata sidecar (alpha).
1 parent 1360f37 commit 7a41ef4

File tree

4 files changed

+155
-0
lines changed

4 files changed

+155
-0
lines changed

‎book/src/SUMMARY.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
- [cluster-driver-registrar](cluster-driver-registrar.md) (deprecated)
1919
- [external-health-monitor-controller](external-health-monitor-controller.md)
2020
- [external-health-monitor-agent](external-health-monitor-agent.md)
21+
- [external-snapshot-metadata](external-snapshot-metadata.md)
2122
- [CSI objects](csi-objects.md)
2223
- [CSIDriver Object](csi-driver-object.md)
2324
- [CSINode Object](csi-node-object.md)
@@ -43,6 +44,7 @@
4344
- [CSI Windows](csi-windows.md)
4445
- [Volume Mode Conversion](prevent-volume-mode-conversion.md)
4546
- [Cross-Namespace Data Sources](cross-namespace-data-sources.md)
47+
- [Changed Block Tracking](changed-block-tracking.md)
4648
- [Deploying a CSI Driver on Kubernetes](deploying.md)
4749
- [Example](example.md)
4850
- [Driver Testing](testing-drivers.md)

‎book/src/changed-block-tracking.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Changed Block Tracking
2+
3+
## Status
4+
5+
Status | Min K8s Version | Max K8s Version | Min CSI Version | Max CSI Version |
6+
-------|-----------------|-----------------|-----------------|-----------------|
7+
Alpha | 1.32 | - | [v1.10.0](https://github.com/container-storage-interface/spec/releases/tag/v1.10.0) | -
8+
9+
10+
## Overview
11+
12+
This optional feature provides a secure mechanism to obtain metadata
13+
on the allocated blocks of a CSI VolumeSnapshot, or the changed blocks between two arbitrary pairs of CSI VolumeSnapshot objects of the same PersistentVolume.
14+
15+
A CSI driver must advertise its support for this feature by specifying the
16+
`SNAPSHOT_METADATA_SERVICE` capability in the response of the
17+
[GetPluginCapabilities](https://github.com/container-storage-interface/spec/blob/master/spec.md#getplugincapabilities)
18+
RPC (part of the
19+
[CSI Identity Service](https://github.com/container-storage-interface/spec/blob/master/spec.md#identity-service-rpc)),
20+
and must implement the
21+
[CSI SnapshotMetadata Service](https://github.com/container-storage-interface/spec/blob/master/spec.md#snapshot-metadata-service-rpcs).
22+
23+
CSI drivers that implement this feature should deploy the
24+
[external-snapshot-metadata](./external-snapshot-metadata.md)
25+
sidecar and create the associated
26+
[Snapshot Metadata Service CR](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking#snapshot-metadata-service-custom-resource).
27+
The sidecar implements the server side of the
28+
[Kubernetes SnapshotMetadata Service API](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking#the-kubernetes-snapshotmetadata-service-api)
29+
that Kubernetes backup application clients must use to obtain VolumeSnapshot metadata.
30+
31+
> See [Kubernetes Enhancement Proposal 3314](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking)
32+
> for details of the Changed Block Tracking feature.

‎book/src/example.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,13 @@ The [deployment example](https://github.com/kubernetes-csi/csi-driver-host-path#
55

66
The example deployment uses the original RBAC rule files that are maintained together with sidecar apps and deploys into the default namespace. A real production should copy the RBAC files and customize them as explained in the comments of those files.
77

8+
The Hostpath CSI driver only supports the [Changed Block Tracking](./changed-block-tracking.md) feature for
9+
PersistentVolumes with `Block` VolumeMode.
10+
The support is not enabled by default, but requires additional configuration when deploying the driver.
11+
When enabled, the
12+
[external-snapshot-metadata](./external-snapshot-metadata.md)
13+
sidecar will also be deployed, and the additional RBAC policies needed will get created.
14+
See the [docs/example-snapshot-metadata.md](https://github.com/kubernetes-csi/csi-driver-host-path/blob/master/docs/example-snapshot-metadata.md)
15+
document for details.
16+
817
If you encounter any problems, please check the [Troubleshooting page](troubleshooting.html).

‎book/src/external-snapshot-metadata.md

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# CSI external-snapshot-metadata
2+
3+
## Status and Releases
4+
5+
**Git Repository:** [https://github.com/kubernetes-csi/external-snapshot-metadata](https://github.com/kubernetes-csi/external-snapshot-metadata)
6+
7+
### Supported Versions
8+
9+
Latest stable release | Branch | Min CSI Version | Max CSI Version | Container Image | [Min K8s Version](project-policies.md#minimum-version) | [Max K8s Version](project-policies.md#maximum-version) | [Recommended K8s Version](project-policies.md#recommended-version) |
10+
--|--|--|--|--|--|--|--
11+
v0.1.0 | [v0.1.0](https://github.com/kubernetes-csi/external-snapshot-metadata/releases/tag/v0.1.0) | [v1.10.0](https://github.com/container-storage-interface/spec/releases/tag/v1.10.0) | - | registry.k8s.io/sig-storage/csi-snapshot-metadata:v0.1.0 | v1.32 | - | v1.32
12+
13+
14+
## Alpha
15+
16+
### Description
17+
This sidecar securely serves snapshot metadata to Kubernetes clients through the
18+
[Kubernetes SnapshotMetadata Service API](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking#the-kubernetes-snapshotmetadata-service-api).
19+
This API is similar to the
20+
[CSI SnapshotMetadata Service](https://github.com/container-storage-interface/spec/blob/master/spec.md#snapshot-metadata-service-rpcs)
21+
but is designed to be used by Kuberetes authenticated and authorized backup applications.
22+
23+
The sidecar authenticates and authorizes each Kubernetes backup application request made through the
24+
Kubernetes SnapshotMetadata Service API.
25+
It then acts as a proxy as it fetches the desired metadata from the CSI driver and
26+
streams it directly to the requesting application with no load on the Kubernetes API server.
27+
28+
See ["The External Snapshot Metadata Sidecar"](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking#the-external-snapshot-metadata-sidecar)
29+
section in the CSI Changed Block Tracking KEP for additional details on the sidecar.
30+
31+
### Usage
32+
Backup applications, identified by their authorized ServiceAccount objects,
33+
directly communicate with the sidecar using the
34+
[Kubernetes SnapshotMetadata Service API](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking#the-kubernetes-snapshotmetadata-service-api).
35+
The authorization needed is described in the
36+
["Risks and Mitigations"](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking#risks-and-mitigations)
37+
section of the CSI Changed Block Tracking KEP.
38+
In particular, this requires the ability to use the Kubernetes
39+
[TokenRequest](https://kubernetes.io/docs/reference/kubernetes-api/authentication-resources/token-request-v1/)
40+
API and to access the objects required to use the API.
41+
42+
The availability of this optional feature is advertised to backup applications by the presence of a
43+
[Snapshot Metadata Service CR](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking#snapshot-metadata-service-custom-resource)
44+
named for the CSI driver that provisions the PersistentVolume and VolumeSnapshot objects involved.
45+
The CR contains the gRPC service endpoint and CA certificate, and an audience string for authentication.
46+
The backup application must use the Kubernetes
47+
[TokenRequest](https://kubernetes.io/docs/reference/kubernetes-api/authentication-resources/token-request-v1/)
48+
API with the audience string to obtain a Kubernetes authentication token for use in the
49+
Kubernetes SnapshotMetadata Service API call.
50+
The backup application should establish trust for the CA certificate before making the gRPC call
51+
to the service endpoint.
52+
53+
VolumeSnapshot metadata can be lengthy, and the Kubernetes SnapshotMetadata Service API supports
54+
restarting an interrupted metadata request from an intermediate point in case of failure.
55+
56+
### Deployment
57+
The CSI `external-snapshot-metadata` sidecar should be deployed by
58+
CSI drivers that support the
59+
[Changed Block Tracking](./changed-block-tracking.md) feature.
60+
The sidecar must be deployed in the same pod as the CSI driver and
61+
will communicate with its gRPC [CSI SnapshotMetadata Service](https://github.com/container-storage-interface/spec/blob/master/spec.md#snapshot-metadata-service-rpcs)
62+
and [CSI Identity Service](https://github.com/container-storage-interface/spec/blob/master/spec.md#identity-service-rpc)
63+
over a UNIX domain socket.
64+
65+
The sidecar should be configured to run under the authority of its
66+
CSI driver ServiceAccount, which must be authorized as described in the
67+
["Risks and Mitigations"](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking#risks-and-mitigations)
68+
section of the CSI Changed Block Tracking KEP.
69+
In particular, this requires the ability to use the Kubernetes
70+
[TokenReview](https://kubernetes.io/docs/reference/kubernetes-api/authentication-resources/token-review-v1/)
71+
and
72+
[SubjectAccessReview](https://kubernetes.io/docs/reference/kubernetes-api/authorization-resources/subject-access-review-v1/)
73+
APIs.
74+
75+
A Service object must be created to expose the endpoints of the
76+
[Kubernetes SnapshotMetadata Service API](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking#the-kubernetes-snapshotmetadata-service-api)
77+
gRPC server implemented by the sidecar.
78+
79+
A [SnapshotMetadataService CR](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking#snapshot-metadata-service-custom-resource),
80+
named for the CSI driver, must be created to advertise the
81+
availability of this optional feature to Kubernetes backup application clients.
82+
The CR contains the CA certificate and Service endpoint address
83+
of the sidecar and the audience string needed for the client
84+
authentication token.
85+
86+
### Resources
87+
88+
The [sidecar repository](https://github.com/kubernetes-csi/external-snapshot-metadata) contains:
89+
90+
- The [protobuf specification](https://github.com/kubernetes-csi/external-snapshot-metadata/tree/main/proto/schema.proto)
91+
of the
92+
[Kubernetes SnapshotMetadata Service API](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/3314-csi-changed-block-tracking#the-kubernetes-snapshotmetadata-service-api).
93+
94+
- The
95+
[Go client interface](https://github.com/kubernetes-csi/external-snapshot-metadata/tree/main/client)
96+
for the Kubernetes SnapshotMetadata Service API.
97+
98+
- The
99+
[pkg/iterator](https://github.com/kubernetes-csi/external-snapshot-metadata/tree/main/pkg/iterator)
100+
client helper package.
101+
This may be used by backup applications instead of working directly with the
102+
Kubernetes SnapshotMetadata Service gRPC client interfaces.
103+
The
104+
[snapshot-metadata-lister](https://github.com/kubernetes-csi/external-snapshot-metadata/tree/main/examples/snapshot-metadata-lister)
105+
example Go command illustrates the use of this helper package.
106+
107+
- Go language
108+
[CSI SnapshotMetadataService API client mocks](https://github.com/kubernetes-csi/external-snapshot-metadata/tree/main/pkg/csiclientmocks).
109+
110+
The sample [Hostpath CSI driver](example.md) has been extended to support the
111+
[Changed Block Tracking](./changed-block-tracking.md) feature
112+
and provides an illustration on how to deploy a CSI driver.

0 commit comments

Comments
 (0)