Skip to content

Commit b5c4404

Browse files
authored
Velero Built-in Data Mover design for OADP (#1107)
fixes #1052
1 parent d34c763 commit b5c4404

File tree

1 file changed

+110
-0
lines changed

1 file changed

+110
-0
lines changed

docs/design/upstream-datamover.md

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
# Design for Integrating Velero Built-in Data Mover (VBDM) support into OADP
2+
3+
4+
## Abstract
5+
One of the primary new features of Velero 1.12 is the new Velero Built-in Data Mover (VBDM).
6+
The OADP Operator needs to add configuration (and e2e tests) to support this new datamover and remove any configuration and testing of the legacy OADP datamover.
7+
8+
## Background
9+
Prior to the release of Velero 1.12, OADP 1.1 and 1.2 integrated an OADP-specific datamover based on Volsync into OADP and the OADP-installed Velero version.
10+
This took the form of some modifications to Velero core and the Velero CSI plugins, a new VSM plugin, and a separate VSM controller that used Volsync to store Volume data in the BackupStorageLocation.
11+
In addition to this, DPA and installation support for the OADP datamover was included in the OADP operator.
12+
13+
In order to integrate VBDM into OADP, all of the above legacy OADP datamover parts will be removed from the OADP operator, the OADP Velero build, and the OADP build of the Velero Plugin for CSI.
14+
15+
## Goals
16+
- Enable Velero Built-in Data Mover (VBDM) in OADP.
17+
- Remove support for the legacy Volsync-based OADP Data Mover.
18+
19+
## Non Goals
20+
- Enabling restore of OADP 1.1 or 1.2 Data Mover backups in OADP with Velero Built-in Datamover enabled
21+
22+
23+
## High-Level Design
24+
Built-in Data Mover functionality will be available in Velero 1.12.
25+
This proposal will allow for enabling this new datamover for backup and restore in OADP.
26+
As part of implementation, any OADP support for the previous Volsync-based datamover will be removed.
27+
All OADP end-to-end tests for data movement will be updated to use the new Built-in Data Mover instead of the Volsync-based data mover.
28+
29+
30+
## Detailed Design
31+
32+
### Volsync Datamover removal
33+
34+
All Volsync Datamover-related code and configuration will be removed from oadp-operator.
35+
36+
#### DPA CRD
37+
38+
For the `DataProtectionApplication` CRD, remove the `DataMover` (and related) structs, including `RetainPolicy` and `VolumeOptions`.
39+
Specifically, `DataMover` will be removed from the `Features` struct which is included in DPA spec.
40+
`Features` will be empty for now, but available for future tech preview features that need explicit configuration here.
41+
```
42+
// Features defines the configuration for the DPA to enable the tech preview features
43+
type Features struct {
44+
}
45+
```
46+
Other places where DataMover-related fields will be removed from the DPA:
47+
1. Remove `dataMoverImageFqin` from the list of valid keys for `UnsupportedOverrides`.
48+
1. Remove `vsmPluginImageFqin` from the list of valid keys for `UnsupportedOverrides`.
49+
1. Remove vsm from the list of default plugins.
50+
1. Remove related consts and labels.
51+
52+
#### OADP controllers
53+
54+
In addition, related controller actions, validation, and unit tests for Volsync data mover will be removed:
55+
1. controllers/validator.go: datamover-related validation
56+
1. controllers/datamover: the entire file
57+
1. controllers/datamover_test.go: the entire file
58+
1. controllers/velero.go: datamover-related velero configuration
59+
1. controllers/dpa_controller.go: datamover-related entries in `ReconcileBatch`
60+
1. pkg/common/common.go: datamover-related const definitions
61+
1. pkg/velero/server/config.go: datamover CRDs removed from restore priorities list
62+
63+
#### Other OADP CRDs
64+
On the CRD side, the `VolumeSnapshotBackup` and `VolumeSnapshotRestore` CRDs will be removed from `config/crd/bases` and from `make bundle` processing.
65+
The new velero `DataUpload` and `DataDownload` will be added.
66+
67+
#### OADP container images
68+
69+
References to `velero-plugin-for-vsm` and `volume-snapshot-mover` images need to be removed from `config/manager/manager.yaml`, golang code, and `deploy/disconnected-prep.sh`.
70+
71+
#### Velero code
72+
73+
In addition to pulling in Velero 1.12 upstream changes into the OADP Velero fork, we will need to remove all of the Velero customizations in our velero and velero-plugin-for-csi repos that were added to support the Volsync data mover.
74+
75+
### Velero Built-in Data Mover configuration
76+
77+
No new DPA fields will be needed for VBDM -- to enable it, the user just needs to include the CSI plugin and enable the node agent. For 1.3, users should use the new `NodeAgentConfig` spec field for this, although the existing (deprecated in 1.3) `Restic` spec field will also work here..
78+
79+
Note that unlike in the OADP Volsync data mover, with VBDM, to use the datamover for a specific backup, that backup must be created with the `--snapshot-move-data` flag to `velero backup create` or with `Spec.SnapshotMoveData` set to `true` on the Backup CR.
80+
81+
### End-to-end test modification
82+
83+
Existing end-to-end tests for datamover use cases will be modified to configure OADP (and Backup CRs) for VBDM.
84+
This will involve removing any Volsync datamover-specific configuration and test assertions, replacing it with VBDM configuration and test assertions.
85+
Existing DPA end-to-end tests will also be modified.
86+
For the DPA tests, this will mostly involve removal of code, since there is no new server/installation configuration needed to enable data movement.
87+
88+
89+
## Alternatives Considered
90+
1. Sticking with Volsync data mover in OADP. This was rejected because the upstream solution will allow us to leverage the existing community and maintainership of Velero to drive features, testing, and bug fixing. In addition, the future of Volsync is in question.
91+
1. Keeping Volsync data mover active in parallel with VBDM to ease user/customer transition. This was rejected due to the maintanence complexity of making the two solutions compatible with each other, and with Volsync data mover in tech preview, future support is not required.
92+
93+
## Security Considerations
94+
From a security point of view, volume data backed up via VBDM is equivalent to data backed up via the existing filesystem backup mechanism.
95+
Any future security enhancements made for the fs backup feature will apply to VBDM as well.
96+
97+
## Compatibility
98+
VBDM is not compatible with the previous OADP 1.2 Volsync-based data mover.
99+
Volsync backups from OADP 1.2 will not be restoreable with any OADP version incorporatig VBDM.
100+
101+
## Implementation
102+
Initial work for this will be done on the velero-datamover topic branch of oadp-operator.
103+
Once the design changes are fully approved and basic functionality for OADP is fully working again (possibly with the exception of the e2e test changes), it will be merged to master and work will continue there.
104+
105+
The velero and velero-plugin-for-csi forks have already been updated with the current VBDM changes but will continue to be updated periodically throughout the remainder of the Velero 1.12 development cycle.
106+
107+
The next steps for implementation will be the OADP DPA and other CRD changes and the controller work.
108+
The final implementation task will be updating the OADP e2e tests.
109+
110+
## Open Issues

0 commit comments

Comments
 (0)