Discover existing snapshots on disk

[ashish-amarnath]

Signed-off-by: Ashish Amarnath [email protected]

What type of PR is this?
/kind feature

What this PR does / why we need it:
This change lets this driver discover existing snapshots on disk and therefore improves testability of the scenario of "Importing an existing volume snapshot with Kubernetes"

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Adds ability to discover and import existing on-disk tarballs as snapshots.

[k8s-ci-robot]

Welcome @ashish-amarnath!

It looks like this is your first PR to kubernetes-csi/csi-driver-host-path 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-csi/csi-driver-host-path has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @ashish-amarnath. Thanks for your PR.

I'm waiting for a kubernetes-csi member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[ashish-amarnath]

This was verified on a 1.17 cluster.
Logs from testing:

I0228 23:26:55.219236       1 controllerserver.go:554] Enabling controller service capability: CREATE_DELETE_SNAPSHOT
I0228 23:26:55.219248       1 controllerserver.go:554] Enabling controller service capability: LIST_SNAPSHOTS
I0228 23:26:55.219263       1 controllerserver.go:554] Enabling controller service capability: CLONE_VOLUME
I0228 23:26:55.219275       1 controllerserver.go:554] Enabling controller service capability: EXPAND_VOLUME
I0228 23:26:55.219316       1 hostpath.go:129] discovering existing snapshots in /csi-data-dir
I0228 23:26:55.219369       1 hostpath.go:147] snapshot: /csi-data-dir/deadfeed.snap, id: deadfeed
I0228 23:26:55.219582       1 server.go:100] Listening for connections on address: &net.UnixAddr{Name:"//csi/csi.sock", Net:"unix"}
I0228

Inspecting storage objects

$ kc get volumesnapshotcontent
NAME                                READYTOUSE   RESTORESIZE   DELETIONPOLICY   DRIVER                VOLUMESNAPSHOTCLASS      VOLUMESNAPSHOT         AGE
snapcontent-hand-crafted-deadfeed   true         0             Delete           hostpath.csi.k8s.io   csi-hostpath-snapclass   velero-csi-pvc-9jwhg   31m

$ kcn csi-app get volumesnapshot
NAME                   READYTOUSE   SOURCEPVC   SOURCESNAPSHOTCONTENT               RESTORESIZE   SNAPSHOTCLASS   SNAPSHOTCONTENT                     CREATIONTIME   AGE
velero-csi-pvc-9jwhg   true                     snapcontent-hand-crafted-deadfeed   0                             snapcontent-hand-crafted-deadfeed   50y            30m

$ kcn csi-app get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS      REASON   AGE
pvc-063555a0-dc11-48f7-be58-b6d12c9da482   1Gi        RWO            Delete           Bound    csi-app/csi-pvc   csi-hostpath-sc            30m

$ kcn csi-app get pvc
NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
csi-pvc   Bound    pvc-063555a0-dc11-48f7-be58-b6d12c9da482   1Gi        RWO            csi-hostpath-sc   30m

Inspecting data on the hostpathplugin driver's disk

kc exec -ti csi-hostpathplugin-0 -c hostpath sh
/ # cd /csi-data-dir/
/csi-data-dir # ls
74f924f9-5a82-11ea-8468-fe468a74e113  deadfeed.snap
/csi-data-dir # ls 74f924f9-5a82-11ea-8468-fe468a74e113/
deadfeed
/csi-data-dir # cksum 74f924f9-5a82-11ea-8468-fe468a74e113/deadfeed
242258054 1574432 74f924f9-5a82-11ea-8468-fe468a74e113/deadfeed
/csi-data-dir #

Inspecting the data on the pod using the csi-pvc

kcn csi-app exec -ti my-csi-app sh
/ # cksum /data/deadfeed
242258054 1574432 /data/deadfeed
/ #

[xing-yang]

/ok-to-test

[xing-yang]

This is very useful to test pre-provisioned snapshots.

Can you also add steps on how you manually create those snapshots?
This is "/kind feature". Need a release note.

[ashish-amarnath]

@xing-yang I've added release notes in the PR description.
Wasn't sure where I should add the steps to create the snapshots.
Adding the steps here.

1. Deploy the csi-hostpath-plugin dirver in a kubernetes 1.17 cluster (this test will use the v1beta1 snapshot apis to import snapshots into the cluster)
2. Exec into the hostpath plugin driver container in the hostpath-plugin pod

$ kubect exec -ti csi-hostpathplugin-0 -c hostpath sh

3. Now create a .snap file in the dataRoot directory to use as a volume snapshot. The hostpath plugin creates a tarballs of the volumes to represent snapshot.

/ # cd /csi-data-dir/
/csi-data-dir # mkdir foo
/csi-data-dir # while true; do echo -n DEADFEED >> ./foo/deadfeed; done
^C
/csi-data-dir #cksum foo/deadfeed
242258054 1574432 foo/deadfeed
/csi-data-dir # tar -cvzf ./deadfeed.snap -C ./foo .
./
./deadfeed
/csi-data-dir # ls
deadfeed.snap  foo
/csi-data-dir # rm -rf ./foo/

4. Now restart the hostpath-plugin dirver pod to allow it to discover the newly created snapshot (deadfeed.snap)
5. On restart of the driver, confirm from the logs that the snapshot created above have been recognized and imported by adding to the hostPathVolumeSnapshots map.
6. Now to load this snapshot into a PV usable by a pod, we will create a volumesnapshotcontent, volumesnapshot and then a pvc using the volumesnapshot as the dataSource.

• Create a namespace to house our test objects, called csi-app
• Create a volumesnapshotcontent object

{
  "apiVersion": "snapshot.storage.k8s.io/v1beta1",
  "kind": "VolumeSnapshotContent",
  "metadata": {
    "name": "snapcontent-hand-crafted-deadfeed"
  },
  "spec": {
    "deletionPolicy": "Delete",
    "driver": "hostpath.csi.k8s.io",
    "source": {
      "snapshotHandle": "deadfeed"
    },
    "volumeSnapshotClassName": "csi-hostpath-snapclass",
    "volumeSnapshotRef": {
      "apiVersion": "snapshot.storage.k8s.io/v1beta1",
      "kind": "VolumeSnapshot",
      "name": "velero-csi-pvc-9jwhg",
      "namespace": "csi-app"
    }
  }
}

• Create a volumesnapshot object. in the csi-app namespace, referring to the above created volumesnapshotcontent

{
  "apiVersion": "snapshot.storage.k8s.io/v1beta1",
  "kind": "VolumeSnapshot",
  "metadata": {
    "annotations": {
      "velero.io/backup-name": "csi-app-backup-1"
    },
    "generateName": "velero-csi-pvc-",
    "generation": 1,
    "name": "velero-csi-pvc-9jwhg",
    "namespace": "csi-app"
  },
  "spec": {
    "source": {
      "volumeSnapshotContentName": "snapcontent-hand-crafted-deadfeed"
    }
  }
}

• Create a pvc, in the csi-app namespace, that uses the above created volumesnapshot as its dataSource

{
  "apiVersion": "v1",
  "kind": "PersistentVolumeClaim",
  "metadata": {
    "annotations": {
      "velero.io/backup-name": "csi-app-backup-1",
      "velero.io/volume-snapshot-name": "velero-csi-pvc-9jwhg"
    },
    "name": "csi-pvc",
    "namespace": "csi-app"
  },
  "spec": {
    "storageClassName": "csi-hostpath-sc",
    "dataSource": {
      "name": "velero-csi-pvc-9jwhg",
      "kind": "VolumeSnapshot",
      "apiGroup": "snapshot.storage.k8s.io"
    },
    "accessModes": [
      "ReadWriteOnce"
    ],
    "resources": {
      "requests": {
        "storage": "1Gi"
      }
    }
  }
}

• On doing this, the provisioner, along with the hostpath plugin driver will provision a pv and load it with the data from the snapshot, datafeed.snap

$ kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS      REASON   AGE
pvc-063555a0-dc11-48f7-be58-b6d12c9da482   1Gi        RWO            Delete           Bound    csi-app/csi-pvc   csi-hostpath-sc            8h

• Create a pod that uses this pvc to inspect the data in the pvc from examples/csi-app.yaml

kind: Pod
apiVersion: v1
metadata:
  name: my-csi-app
spec:
  containers:
    - name: my-frontend
      image: busybox
      volumeMounts:
      - mountPath: "/data"
        name: my-csi-volume
      command: [ "sleep", "1000000" ]
  volumes:
    - name: my-csi-volume
      persistentVolumeClaim:
        claimName: csi-pvc # defined in csi-pvc.yaml

Create the above pod and exec into the pod and inspect contents of /data

$ kubectl csi-app exec -ti my-csi-app sh
/ # cksum /data/deadfeed
242258054 1574432 /data/deadfeed
/ #

The checksums of the deadfeed file volumemounted by the my-csi-app pod matches that of the the file that was created as part of setting up the snapshot.

Verify the state of the kubernetes storage objects

• volumesnapshot and pvc

$ kubectl -n csi-app get volumesnapshot,pvc
NAME                                                          READYTOUSE   SOURCEPVC   SOURCESNAPSHOTCONTENT               RESTORESIZE   SNAPSHOTCLASS   SNAPSHOTCONTENT                     CREATIONTIME   AGE
volumesnapshot.snapshot.storage.k8s.io/velero-csi-pvc-9jwhg   true                     snapcontent-hand-crafted-deadfeed   0                             snapcontent-hand-crafted-deadfeed   50y            8h

NAME                            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
persistentvolumeclaim/csi-pvc   Bound    pvc-063555a0-dc11-48f7-be58-b6d12c9da482   1Gi        RWO            csi-hostpath-sc   8h

• volumesnapshotcontents and pv

$ kubectl get volumesnapshotcontents,pv
NAME                                                                              READYTOUSE   RESTORESIZE   DELETIONPOLICY   DRIVER                VOLUMESNAPSHOTCLASS      VOLUMESNAPSHOT         AGE
volumesnapshotcontent.snapshot.storage.k8s.io/snapcontent-hand-crafted-deadfeed   true         0             Delete           hostpath.csi.k8s.io   csi-hostpath-snapclass   velero-csi-pvc-9jwhg   8h

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS      REASON   AGE
persistentvolume/pvc-063555a0-dc11-48f7-be58-b6d12c9da482   1Gi        RWO            Delete           Bound    csi-app/csi-pvc   csi-hostpath-sc            8h

This above test demonstrates the ability of this driver to import existing snapshots and provision volumes using the data in the volumesnapshot. Thus verifying this PR.

[xing-yang]

@ashish-amarnath For release note, you only need:
Adds ability to discover and import existing on-disk tarballs as snapshots.

The PR info will be taken care when this driver is released.
It's fine to add the steps here in this PR. We can decide where to document this later.

[xing-yang]

On doing this, the snapshot controller along with the hostpath plugin driver will provision a pv and load it with the data from the snapshot, datafeed.tgz

It should be the provisioner not the snapshot controller that provisions the PV.

[xing-yang]

6. Now to create PVs using this snapsthot

s/snapsthot/snapshot

You are only creating PV that uses the snapshot as data source, but not creating the source PV/PVC for this snapshot, right?

[ashish-amarnath]

@xigang

It should be the provisioner not the snapshot controller that provisions the PV.
Thanks for the correction 😄 I've updated the comment.

You are only creating PV that uses the snapshot as data source, but not creating the source PV/PVC for this snapshot, right?

I am creating volumesnapshotcontent, volumesnapshot and then the pvc using the volumesnapshot as the dataSource.

I've also updated the comment w/ this clarification.

[xing-yang]

Can you rework this based on this change? #162

[xing-yang]

/lgtm

[msau42]

Thank you for adding comprehensive unit tests! Just some minor comments

[xing-yang]

/lgtm

[msau42]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ashish-amarnath, msau42, xing-yang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [msau42]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment