fix: delete volume error in archive deletion mode

[andyzhangx]

What type of PR is this?
/kind bug

What this PR does / why we need it:
fix: delete volume error in archive deletion mode

when onDelete: archive, there could be delete volume error with error msg rename xxx: no such file or directory since there is a bug in csi-provisioner that DeleteVolume may be performed twice, this PR adds a 1 min cache and lock to workaround this issue.

[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.334699       1 utils.go:76] GRPC call: /csi.v1.Controller/DeleteVolume
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.334752       1 utils.go:77] GRPC request: {"secrets":"***stripped***","volume_id":"135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive"}
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.334843       1 controllerserver.go:154] DeleteVolume: found mountOptions(dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks) for volume(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.334864       1 controllerserver.go:169] begin to delete or archive subdirectory since secret is provided
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.334876       1 controllerserver.go:292] internally mounting //135.236.234.180/share at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.335104       1 nodeserver.go:209] NodeStageVolume: targetPath(/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2) volumeID(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive) context(map[source://135.236.234.180/share]) mountflags([dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks]) mountOptions([dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks])
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.335322       1 mount_linux.go:218] Mounting cmd (mount) with arguments (-t cifs -o dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks,<masked> //135.236.234.180/share /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.373268       1 nodeserver.go:241] volume(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive) mount "//135.236.234.180/share" on "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2" succeeded
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.374874       1 controllerserver.go:186] DeleteVolume: subdirectory(smb-4792/pvc-ckk2v) contains '/', make sure the parent directory(/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792) exists
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.381550       1 controllerserver.go:193] archiving subdirectory /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/smb-4792/pvc-ckk2v --> /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.381577       1 controllerserver.go:195] removing archived subdirectory at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.383991       1 controllerserver.go:199] removed archived subdirectory at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.391986       1 controllerserver.go:310] internally unmounting /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.392012       1 nodeserver.go:264] NodeUnstageVolume: CleanupMountPoint on /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2 with volume 135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.393671       1 mount_helper_common.go:93] unmounting "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2" (corruptedMount: false, mounterCanSkipMountPointChecks: true)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.393698       1 mount_linux.go:360] Unmounting /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.399036       1 mount_helper_common.go:150] Deleting path "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2"
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.399168       1 nodeserver.go:273] NodeUnstageVolume: unmount volume 135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive on /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2 successfully
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.399192       1 utils.go:83] GRPC response: {}
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.427703       1 utils.go:76] GRPC call: /csi.v1.Controller/DeleteVolume
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.427736       1 utils.go:77] GRPC request: {"secrets":"***stripped***","volume_id":"135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive"}
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.427840       1 controllerserver.go:154] DeleteVolume: found mountOptions(dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks) for volume(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.427855       1 controllerserver.go:169] begin to delete or archive subdirectory since secret is provided
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.427871       1 controllerserver.go:292] internally mounting //135.236.234.180/share at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.428098       1 nodeserver.go:209] NodeStageVolume: targetPath(/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2) volumeID(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive) context(map[source://135.236.234.180/share]) mountflags([dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks]) mountOptions([dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks])
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.428345       1 mount_linux.go:218] Mounting cmd (mount) with arguments (-t cifs -o dir_mode=0777,file_mode=0777,uid=0,gid=0,mfsymlinks,<masked> //135.236.234.180/share /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.454514       1 nodeserver.go:241] volume(135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive) mount "//135.236.234.180/share" on "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2" succeeded
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.454559       1 controllerserver.go:186] DeleteVolume: subdirectory(smb-4792/pvc-ckk2v) contains '/', make sure the parent directory(/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792) exists
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.456730       1 controllerserver.go:193] archiving subdirectory /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/smb-4792/pvc-ckk2v --> /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.456753       1 controllerserver.go:195] removing archived subdirectory at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.470770       1 controllerserver.go:199] removed archived subdirectory at /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.476121       1 controllerserver.go:310] internally unmounting /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.476146       1 nodeserver.go:264] NodeUnstageVolume: CleanupMountPoint on /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2 with volume 135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.476172       1 mount_helper_common.go:93] unmounting "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2" (corruptedMount: false, mounterCanSkipMountPointChecks: true)
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.476185       1 mount_linux.go:360] Unmounting /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.484264       1 mount_helper_common.go:150] Deleting path "/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2"
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] I0829 08:02:15.484631       1 nodeserver.go:273] NodeUnstageVolume: unmount volume 135.236.234.180/share#smb-4792/pvc-ckk2v#pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2#archive on /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2 successfully
[pod/csi-smb-controller-79db546bb6-rn2bd/smb] E0829 08:02:15.484657       1 utils.go:81] GRPC error: rpc error: code = Internal desc = archive subdirectory(/tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/smb-4792/pvc-ckk2v, /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v) failed with rename /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/smb-4792/pvc-ckk2v /tmp/pvc-b1130ed7-05eb-4a93-99ff-92b80fc82ac2/archived-smb-4792/pvc-ckk2v: no such file or directory

Which issue(s) this PR fixes:

Fixes #

Requirements:

• uses conventional commit messages
• includes documentation
• adds unit tests
• tested upgrade from previous version

Special notes for your reviewer:

Release note:

fix: delete volume error in archive deletion mode

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andyzhangx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [andyzhangx]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coveralls]

Pull Request Test Coverage Report for Build 10634580152

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 12 of 23 (52.17%) changed or added relevant lines in 2 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-0.5%) to 78.389%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/smb/smb.go	1	3	33.33%
pkg/smb/controllerserver.go	11	20	55.0%

Totals
Change from base Build 10609986454:	-0.5%
Covered Lines:	954
Relevant Lines:	1217

---

💛 - Coveralls

[andyzhangx]

/retest

[andyzhangx]

/retest