Skip to content

DOCSP-40077 ABC Migrations + Migration Name Reference Page #332

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jun 26, 2024
4 changes: 4 additions & 0 deletions source/includes/abc-migration-intro.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Starting in {+c2c-beta-program-short+} 1.8, you can perform A->B->C migrations.
A->B->C migrations allow you to perform two consecutive migrations, where the
destination cluster of the first migration acts as the source cluster for the
second migration.
4 changes: 2 additions & 2 deletions source/includes/beta-feature.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.. important:: Cluster-to-Cluster Sync Beta Program

This feature is only available in ``mongosync-beta``. To learn more, see
:ref:`c2c-beta-program`.
This feature is only available in {+c2c-beta-program-short+}. To learn more,
see :ref:`c2c-beta-program`.
4 changes: 4 additions & 0 deletions source/includes/many-to-one-cluster.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Starting in {+c2c-beta-program-short+} 1.8, you can perform Many-to-One
migrations. Many-to-One migrations allow you to sync multiple source
clusters simultaneously with a destination cluster. For example, you can
consolidate data from many small clusters into a central cluster.
3 changes: 0 additions & 3 deletions source/includes/many-with-one-cluster.rst

This file was deleted.

4 changes: 0 additions & 4 deletions source/includes/migrationName-description.rst

This file was deleted.

5 changes: 5 additions & 0 deletions source/includes/migrationName.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Starting in {+c2c-beta-program-short+} 1.8, you can specify a ``migrationName``
to describe a sync operation. For example, you can set ``migrationName`` to
distinguish and identify different sync operations when working with
:ref:`Many-to-One <c2c-beta-many-to-one>` or :ref:`A->B->C
<c2c-beta-abc-migration>` migrations.
18 changes: 0 additions & 18 deletions source/includes/opts/migrationName.rst

This file was deleted.

32 changes: 0 additions & 32 deletions source/quickstart.txt
Original file line number Diff line number Diff line change
Expand Up @@ -307,35 +307,3 @@ Synchronization Notes

- To estimate the size of ``oplog`` needed for initial synchronization,
see :ref:`c2c-oplog-sizing`.

.. _c2c-quickstart-many-with-one:

Sync Multiple Source Clusters Simultaneously with a Destination Cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: /includes/many-with-one-cluster.rst

For limitations, see :ref:`c2c-limitations_multiple_sync`.

The following examples connect source clusters running on port ``27000``
and ``27001`` with a destination cluster running on port ``35000``. The
commands also set the optional :option:`--migrationName` string to
describe the operations.

.. code-block:: shell

./bin/mongosync \
--cluster0 "mongodb://localhost:27000" \
--cluster1 "mongodb://localhost:35000" \
--migrationName "cluster_27000_to_cluster_35000_sync"

./bin/mongosync \
--cluster0 "mongodb://localhost:27001" \
--cluster1 "mongodb://localhost:35000" \
--migrationName "cluster_27001_to_cluster_35000_sync"

.. include:: /includes/many-with-one-cluster.rst

To start the sync operation between the source clusters and the
destination cluster, see the earlier section
:ref:`c2c-quickstart-synchronize`.
16 changes: 13 additions & 3 deletions source/reference/beta-program.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
Each ``mongosync`` release has a corresponding {+c2c-beta-program-short+}
build that includes its own set of experimental features.

.. _c2c-beta-program-disclaimer:

Get Started
-----------

Expand Down Expand Up @@ -67,8 +69,8 @@ Beta Features
* - Feature
- Description

* - :ref:`c2c-beta-namespace-remapping`
- # DESCRIPTION TBD
* - :ref:`c2c-beta-abc-migration`
- .. include:: /includes/abc-migration-intro.rst

* - :ref:`c2c-beta-document-filtering`
- .. include:: /includes/document-filtering-intro.rst
Expand All @@ -79,14 +81,22 @@ Beta Features
* - :ref:`c2c-beta-destination-data-handling`
- .. include:: /includes/destinationDataHandling-introduction.rst

* - :ref:`c2c-beta-migration-name`
- .. include:: /includes/migrationName.rst

* - :ref:`c2c-beta-namespace-remapping`
- # DESCRIPTION TBD

* - :ref:`c2c-beta-many-to-one`
- .. include:: /includes/many-with-one-cluster.rst
- .. include:: /includes/many-to-one-cluster.rst

.. toctree::
:titlesonly:

/reference/beta-program/abc-migration
/reference/beta-program/namespace-remapping
/reference/beta-program/document-filtering
/reference/beta-program/orr
/reference/beta-program/destinationDataHandling
/reference/beta-program/migration-name
/reference/beta-program/many-to-one
89 changes: 89 additions & 0 deletions source/reference/beta-program/abc-migration.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
.. _c2c-beta-abc-migration:

==================
A->B->C Migrations
==================

.. default-domain:: mongodb

.. contents:: On this page
:local:
:backlinks: none
:depth: 2
:class: singlecol

.. include:: /includes/beta-feature.rst

.. include:: /includes/abc-migration-intro.rst

Syntax
------

To perform an A->B->C migration, run the following commands when starting
:ref:`mongosync <c2c-mongosync>`:

.. code-block:: shell

./bin/mongosync \
--cluster0 <cluster-A-connection-string> \
--cluster1 <cluster-B-connection-string> \
--migrationName <string>

./bin/mongosync \
--cluster0 <cluster-B-connection-string> \
--cluster1 <cluster-C-connection-string> \
--migrationName <string>

To start the sync operation between the source and destination clusters, see
:ref:`c2c-quickstart-synchronize`.

Behavior
--------

The first migration (A->B) must finish :ref:`committing <c2c-api-commit>`
before the second migration (B->C) starts committing.

.. warning::

If the second migration starts committing before the first migration
finishes, you may experience data loss.

Before you use any experimental {+c2c-beta-program-short+} features,
review the :ref:`{+c2c-full-beta-program+} Disclaimer
<c2c-beta-program-disclaimer>`.

Example
-------

The following example performs two consecutive migrations that:

#. Connect a source cluster running on port ``27000`` with a destination
cluster running on port ``27001``.

#. Use the destination cluster running on port ``27001`` as a source cluster
for the second migration.

#. Connect the source cluster on port ``27001`` with a destination cluster
running on port ``27002``.

The command also sets :ref:`--migrationName <c2c-beta-migration-name>`
to describe the operations and store migration metadata for each
sync.

.. code-block:: shell

./bin/mongosync \
--cluster0 "mongodb://localhost:27000" \
--cluster1 "mongodb://localhost:27001" \
--migrationName "cluster_27000_to_cluster_27001_sync"

./bin/mongosync \
--cluster0 "mongodb://localhost:27001" \
--cluster1 "mongodb://localhost:27002" \
--migrationName "cluster_27001_to_cluster_27002_sync"

Learn More
----------

- :ref:`c2c-beta-program`
- :ref:`c2c-beta-migration-name`
6 changes: 3 additions & 3 deletions source/reference/beta-program/document-filtering.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
.. _c2c-beta-document-filtering:

================
Filter Documents
================
==================
Document Filtering
==================

.. default-domain:: mongodb

Expand Down
83 changes: 80 additions & 3 deletions source/reference/beta-program/many-to-one.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
.. _c2c-beta-many-to-one:

================================================================
Synchronize Data Between Multiple Source Clusters Simultaneously
================================================================
======================
Many-to-One Migrations
======================

.. default-domain:: mongodb

Expand All @@ -13,3 +13,80 @@ Synchronize Data Between Multiple Source Clusters Simultaneously
:class: singlecol

.. include:: /includes/beta-feature.rst

.. include:: /includes/many-to-one-cluster.rst

Syntax
------

To sync multiple source clusters with one destination cluster, run the following
commands when starting :ref:`mongosync <c2c-mongosync>`:

.. code-block:: shell

./bin/mongosync \
--cluster0 <source-A-connection-string> \
--cluster1 <destination-connection-string> \
--migrationName <string>

./bin/mongosync \
--cluster0 <source-B-connection-string> \
--cluster1 <destination-connection-string> \
--migrationName <string>

To start the sync operation between the source clusters and the
destination cluster, see :ref:`c2c-quickstart-synchronize`.

Behavior
--------

A namespace is a ``database_name.collection_name`` combination. You can only
sync namespaces that don't conflict.

For example, consider this scenario:

- Two source clusters S1 and S2.
- A destination cluster D.
- Databases named ``inventory`` and ``sales`` on both S1 and S2.
- Collections named ``products``, ``orderLines``, ``orderStatus``, and
``orders`` on both S1 and S2.
- You can sync both of these combinations:

- ``inventory.products`` and ``sales.orderStatus`` on S1 with D.
- ``inventory.orderLines`` and ``sales.orders`` on S2 with D.

- You cannot sync both of these combinations because they conflict:

- ``inventory.products`` and ``inventory.orderLines`` on S1 with D. If D is
initially empty, you can sync S1 with D. ``inventory.products`` and
``inventory.orderLines`` are copied from S1 to D.
- ``inventory.products`` and ``inventory.orderLines`` on S2 with D. You cannot
sync S2 with D because ``inventory.products`` and ``inventory.orderLines``
conflict with the namespaces already on D from the scenario in the previous
point.

Example
-------

The following example connects source clusters running on port ``27000``
and ``27001`` with a destination cluster running on port ``35000``. The command
also sets the :ref:`--migrationName <c2c-beta-migration-name>` option to
describe the operations and store migration metadata for each sync.

.. code-block:: shell

./bin/mongosync \
--cluster0 "mongodb://localhost:27000" \
--cluster1 "mongodb://localhost:35000" \
--migrationName "cluster_27000_to_cluster_35000_sync"

./bin/mongosync \
--cluster0 "mongodb://localhost:27001" \
--cluster1 "mongodb://localhost:35000" \
--migrationName "cluster_27001_to_cluster_35000_sync"

Learn More
----------

- :ref:`c2c-beta-program`
- :ref:`c2c-beta-migration-name`
65 changes: 65 additions & 0 deletions source/reference/beta-program/migration-name.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
.. _c2c-beta-migration-name:

=============
migrationName
=============

.. default-domain:: mongodb

.. contents:: On this page
:local:
:backlinks: none
:depth: 2
:class: singlecol

.. include:: /includes/beta-feature.rst

.. include:: /includes/migrationName.rst

Syntax
------

To set ``migrationName`` from the command line, use the ``--migrationName``
setting:

.. code-block:: shell

./bin/mongosync \
--cluster0 <connection-string> \
--cluster1 <connection-string> \
--migrationName <string>

To set ``migrationName`` in a :ref:`configuration file <c2c-config>`,
include the ``migrationName`` option:

.. code-block:: yaml

cluster0: <connection-string>
cluster1: <connection-string>
migrationName: <string>

Behavior
--------

The ``migrationName`` string can contain up to 44 alphanumeric and underscore
characters. ``migrationName`` is appended to the string
``"mongosync_internal_"`` to set the migration metadata database name.

For example, if you set ``migrationName`` to
``"cluster_27000_to_cluster_35000_sync"``, the resulting ``mongosync`` metadata
database name is ``"mongosync_internal_cluster_27000_to_cluster_35000_sync"``.

.. _c2c-beta-multiple-migration-examples:

Examples
--------

To view examples that use ``migrationName``, see:

- :ref:`c2c-beta-abc-migration`
- :ref:`c2c-beta-many-to-one`

Learn More
----------

- :ref:`c2c-beta-program`
Loading