Skip to content

✨clusterctl: add the clusterctl move command #2130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions cmd/clusterctl/cmd/move.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,11 @@ limitations under the License.
package cmd

import (
"fmt"

"github.com/pkg/errors"
"github.com/spf13/cobra"
"sigs.k8s.io/cluster-api/cmd/clusterctl/pkg/client"
)

type moveOptions struct {
Expand Down Expand Up @@ -59,5 +62,17 @@ func init() {
}

func runMove() error {
c, err := client.New(cfgFile)
if err != nil {
return err
}
fmt.Println("Performing move...")
if err := c.Move(client.MoveOptions{
FromKubeconfig: mo.fromKubeconfig,
ToKubeconfig: mo.toKubeconfig,
Namespace: mo.namespace,
}); err != nil {
return err
}
return nil
}
24 changes: 17 additions & 7 deletions cmd/clusterctl/pkg/client/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,14 @@ type DeleteOptions struct {
Providers []string
}

// Client is exposes the clusterctl high-level client library
// MoveOptions carries the options supported by move.
type MoveOptions struct {
FromKubeconfig string
ToKubeconfig string
Namespace string
}

// Client is exposes the clusterctl high-level client library.
type Client interface {
// GetProvidersConfig returns the list of providers configured for this instance of clusterctl.
GetProvidersConfig() ([]Provider, error)
Expand All @@ -70,6 +77,9 @@ type Client interface {

// Delete deletes providers from a management cluster.
Delete(options DeleteOptions) error

// Move moves all the Cluster API objects existing in a namespace (or from all the namespaces if empty) to a target management cluster.
Move(options MoveOptions) error
}

// clusterctlClient implements Client.
Expand All @@ -85,7 +95,7 @@ type ClusterClientFactory func(string) (cluster.Client, error)
// Ensure clusterctlClient implements Client.
var _ Client = &clusterctlClient{}

// NewOptions carries the options supported by New
// NewOptions carries the options supported by New.
type NewOptions struct {
injectConfig config.Client
injectRepositoryFactory RepositoryClientFactory
Expand Down Expand Up @@ -130,7 +140,7 @@ func newClusterctlClient(path string, options ...Option) (*clusterctlClient, err
}

// if there is an injected config, use it, otherwise use the default one
// provided by the config low level library
// provided by the config low level library.
configClient := cfg.injectConfig
if configClient == nil {
c, err := config.New(path)
Expand All @@ -140,13 +150,13 @@ func newClusterctlClient(path string, options ...Option) (*clusterctlClient, err
configClient = c
}

// if there is an injected RepositoryFactory, use it, otherwise use a default one
// if there is an injected RepositoryFactory, use it, otherwise use a default one.
repositoryClientFactory := cfg.injectRepositoryFactory
if repositoryClientFactory == nil {
repositoryClientFactory = defaultRepositoryFactory(configClient)
}

// if there is an injected ClusterFactory, use it, otherwise use a default one
// if there is an injected ClusterFactory, use it, otherwise use a default one.
clusterClientFactory := cfg.injectClusterFactory
if clusterClientFactory == nil {
clusterClientFactory = defaultClusterFactory()
Expand All @@ -159,14 +169,14 @@ func newClusterctlClient(path string, options ...Option) (*clusterctlClient, err
}, nil
}

// defaultClusterFactory is a ClusterClientFactory func the uses the default client provided by the cluster low level library
// defaultClusterFactory is a ClusterClientFactory func the uses the default client provided by the cluster low level library.
func defaultClusterFactory() func(kubeconfig string) (cluster.Client, error) {
return func(kubeconfig string) (cluster.Client, error) {
return cluster.New(kubeconfig, cluster.Options{}), nil
}
}

// defaultRepositoryFactory is a RepositoryClientFactory func the uses the default client provided by the repository low level library
// defaultRepositoryFactory is a RepositoryClientFactory func the uses the default client provided by the repository low level library.
func defaultRepositoryFactory(configClient config.Client) func(providerConfig config.Provider) (repository.Client, error) {
return func(providerConfig config.Provider) (repository.Client, error) {
return repository.New(providerConfig, configClient.Variables(), repository.Options{})
Expand Down
4 changes: 4 additions & 0 deletions cmd/clusterctl/pkg/client/client_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,10 @@ func (f fakeClient) Delete(options DeleteOptions) error {
return f.internalClient.Delete(options)
}

func (f fakeClient) Move(options MoveOptions) error {
return f.internalClient.Move(options)
}

// newFakeClient returns a clusterctl client that allows to execute tests on a set of fake config, fake repositories and fake clusters.
// you can use WithCluster and WithRepository to prepare for the test case.
func newFakeClient(configClient config.Client) *fakeClient {
Expand Down
47 changes: 47 additions & 0 deletions cmd/clusterctl/pkg/client/move.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/*
Copyright 2020 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package client

func (c *clusterctlClient) Move(options MoveOptions) error {
// Get the client for interacting with the source management cluster.
fromCluster, err := c.clusterClientFactory(options.FromKubeconfig)
if err != nil {
return err
}

// Ensures the custom resource definitions required by clusterctl are in place.
if err := fromCluster.ProviderInventory().EnsureCustomResourceDefinitions(); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to install the CRDs in the source cluster during move?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually we need the CRDs and all the required providers to be in place.
#2102 contains a TODO for implementing a preflight check for this in the Mover.Move method, and hopefully, this will be addressed next week.

return err
}

// Get the client for interacting with the target management cluster.
toCluster, err := c.clusterClientFactory(options.ToKubeconfig)
if err != nil {
return err
}

// Ensures the custom resource definitions required by clusterctl are in place
if err := toCluster.ProviderInventory().EnsureCustomResourceDefinitions(); err != nil {
return err
}

if err := fromCluster.ObjectMover().Move(options.Namespace, toCluster); err != nil {
return err
}

return nil
}
35 changes: 35 additions & 0 deletions docs/book/src/clusterctl/commands/move.md
Original file line number Diff line number Diff line change
@@ -1 +1,36 @@
# clusterctl move

The `clusterctl move` command allows to move the Cluster API objects defining workload clusters, like e.g. Cluster, Machines,
MachineDeployments, etc. from one management cluster to another management cluster.

<aside class="note warning">

<h1> Warning </h1>

Before running `clusterctl move`, the user should take care of preparing the target management cluster, including also installing
all the required provider using `clusterctl init`.

The version of the providers installed in the target management cluster should be at least the same version of the
corresponding provider in the source cluster.

</aside>

You can use:

```shell
clusterctl move --to-kubeconfig="path-to-target-kubeconfig.yaml"
```

To move all the Cluster API objects objects in the source management cluster; in case if you want to move only the
Cluster API objects defined in a specific namespace, you can use the `--namespace` flag.

<aside class="note">

<h1> Pause Reconciliation </h1>

Before moving a `Cluster`, clusterctl sets the `Cluster.Spec.Paused` field to `true` stopping
the controllers to reconcile the workload cluster _in the source management cluster_.

The `Cluster` object created in the target management cluster instead will be actively reconciled.

</aside>
38 changes: 36 additions & 2 deletions docs/book/src/clusterctl/provider-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,25 @@ Templates writers should use the common variables to ensure consistency across p
Additionally, value of the command argument to `clusterctl config cluster <cluster-name>` (`<cluster-name>` in this case), will
be applied to every occurrence of the `${ CLUSTER_NAME }` variable.

## OwnerReferences chain

Each provider is responsible to ensure that all the providers resources (like e.g. `VSphereCluster`, `VSphereMachine`, `VSphereVM` etc.
for the `vsphere` provider) MUST have a `Metadata.OwnerReferences` entry that links directly or indirectly to a `Cluster` object.

Please note that all the provider specific resources that are referenced by the Cluster API core objects will get the `OwnerReference`
sets by the Cluster API core controllers, e.g.:

- The Cluster controller ensures that all the objects referenced in `Cluster.Spec.InfrastructureRef` get an `OwnerReference`
that links directly to the corresponding `Cluster`.
- The Machine controller ensures that all the objects referenced in `Machine.Spec.InfrastructureRef` get an `OwnerReference`
that links to the corresponding `Machine`, and the `Machine` is linked to the `Cluster` through its own `OwnerReference` chain.

That means that, practically speaking, provider implementers are responsible for ensuring that the `OwnerReference`s
are set only for objects that are not directly referenced by Cluster API core objects, e.g.:

- All the `VSphereVM` instances should get an `OwnerReference` that links to the corresponding `VSphereMachine`, and the `VSphereMachine`
is linked to the `Cluster` through its own `OwnerReference` chain.

## Additional notes

### Components YAML transformations
Expand Down Expand Up @@ -199,10 +218,25 @@ If, for any reason, the provider authors/YAML designers decide not to comply wit
The provider authors/YAML designers should be aware that it is their responsibility to ensure the proper
functioning of all the `clusterctl` features both in single tenancy or multi-tenancy scenarios and/or document known limitations.

### Move constraints
### Move

WIP
Provider authors should be aware that `clusterctl move` command implements a discovery mechanism that considers:

* All the objects of Kind defined in one of the CRDs installed by clusterctl using `clusterctl init`.
* `Secret` and `ConfigMap` objects.
* the `OwnerReference` chain of the above objects.

`clusterctl move` does NOT consider any objects:

* Not included in the set of objects defined above.
* Included in the set of objects defined above, but not directly or indirectly to a `Cluster` object through the `OwnerReference` chain.

If moving some of excluded object is required, the provider authors should create documentation describing the
the exact move sequence to be executed by the user.

Additionally, provider authors should be aware that `clusterctl move` assumes all the provider's Controllers respect the
`Cluster.Spec.Paused` field introduced in the v1alpha3 Cluster API specification.

### Adopt

WIP