Skip to content

DOCSP-39059 namespace remap #359

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
8a58611
DOCSP-39059 Namespace Remap
kennethdyer Jul 10, 2024
f333a36
Adds request/response example
kennethdyer Jul 10, 2024
f0747c1
Adds request/response example
kennethdyer Jul 10, 2024
f27ad5a
Adds RN
kennethdyer Jul 10, 2024
fe1b89c
Adds to/from to the start parameters
kennethdyer Jul 24, 2024
a123b83
Adds to/from to the start parameters
kennethdyer Jul 24, 2024
8c94565
Adds to/from to the start parameters
kennethdyer Jul 24, 2024
768784f
Adds namespace page todos
kennethdyer Jul 24, 2024
ce746da
Adds syntax
kennethdyer Jul 24, 2024
176f9bf
Fixes typo
kennethdyer Jul 24, 2024
3f0e1dd
Procedure
kennethdyer Jul 24, 2024
160fc80
Procedure
kennethdyer Jul 24, 2024
c655457
Request parameters
kennethdyer Jul 24, 2024
7902384
Request parameters
kennethdyer Jul 24, 2024
c60acba
Links
kennethdyer Jul 25, 2024
548d5cc
Minor edit
kennethdyer Jul 26, 2024
d54330d
Fixes per Jonathan Wilson
kennethdyer Jul 26, 2024
bdce6c9
Fixes per Jessica
kennethdyer Jul 26, 2024
b900817
Fixes per Jessica
kennethdyer Jul 26, 2024
3783756
Fixes per Jessica
kennethdyer Jul 26, 2024
902dfe8
Fixes per Jessica
kennethdyer Jul 26, 2024
c24b4fa
Fixes per Jessica
kennethdyer Jul 26, 2024
17ebf0a
Fixes per Jessica
kennethdyer Jul 26, 2024
f25ce3d
Fixes per Jessica
kennethdyer Jul 26, 2024
2a7a5e4
Fixes per Jessica
kennethdyer Jul 26, 2024
226ea50
Fixes per Jessica
kennethdyer Jul 26, 2024
76cedd2
Fixes per Jessica
kennethdyer Jul 30, 2024
ae625db
Fixes per Jessica
kennethdyer Jul 30, 2024
ffc4237
Fixes per Jessica
kennethdyer Jul 30, 2024
c50c3de
Fixes per Jessica
kennethdyer Jul 30, 2024
a6c3203
Fixes per Jessica
kennethdyer Jul 30, 2024
6f9680e
Fixes per Jessica
kennethdyer Jul 30, 2024
ea68186
Fixes per Jessica
kennethdyer Jul 30, 2024
e47ad6f
Fixes build issue
kennethdyer Jul 30, 2024
639fdd2
Fixes per Jessica
kennethdyer Jul 31, 2024
4c1967a
Fixes per Jessica
kennethdyer Jul 31, 2024
e710748
Fixes per Jessica
kennethdyer Jul 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions source/includes/api/requests/start-namespace-remap.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
curl -X POST "http://localhost:27182/api/v1/start" --data '
{
"source": "cluster0",
"destination": "cluster1",
"namespaceRemap": [
{
"from": {
"database": "accounts",
},
"to": {
"database": "sales",
}
}
]
} '
35 changes: 35 additions & 0 deletions source/includes/fact-valid-namespace-remap.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@

The following restrictions apply to namespace remapping:

- Namespace remapping doesn't support changes to collection names.

- Namespace remapping doesn't permit writing to the ``system``, ``config``, ``admin``,
or ``local`` databases, or writing to internal databases used by ``mongosync``.

- The database name on the destination cluster must be valid under Windows
restrictions.

For more information, see :limit:`Restrictions on Database Names for Windows`.

- Remapped database names on the destination cluster cannot differ only in case.

- The remap cannot produce namespace conflicts on the destination cluster.

For example:

.. code-block:: javascript
:copyable: false

"namespaceRemap": [
{
"from": { "database": "us-west" },
"to": {"database": "us-accounts" }
},
{
"from": { "database": "us-south" },
"to": { "database": "us-accounts" }
}
]

This remap would generate an namespace error if each database on the source cluster
contains a ``texas`` collection.
37 changes: 37 additions & 0 deletions source/reference/api/start.txt
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,19 @@ Request Body Parameters

.. versionadded:: 1.6

* - ``namespaceRemap``
- array
- Optional
- .. important::

This feature is only available in the :ref:`c2c-beta-program`.

Array of documents that specify namespace changes to make during sync.

For more information, see :ref:`c2c-beta-namespace-remapping`.

.. versionadded:: 1.8 Beta

* - ``reversible``
- boolean
- Optional
Expand Down Expand Up @@ -325,6 +338,27 @@ Response
:language: json
:copyable: false

Example: Start Sync to Remap Namespaces
---------------------------------------

The ``namespaceRemap`` option allows you to migrate collection data from one
namespace on the source cluster to a different namespace on the destination
cluster.

Request
~~~~~~~

.. literalinclude:: /includes/api/requests/start-namespace-remap.sh
:language: shell

Response
~~~~~~~~

.. literalinclude:: /includes/api/responses/success.json
:language: json
:copyable: false


Example: Start Sync from Replica Set to Sharded Cluster
-------------------------------------------------------

Expand Down Expand Up @@ -476,4 +510,7 @@ Endpoint Protection
.. |endpoint| replace:: ``start``
.. include:: /includes/fact-api-endpoint

Valid Namespace Remapping
~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: /includes/fact-valid-namespace-remap
126 changes: 122 additions & 4 deletions source/reference/beta-program/namespace-remapping.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
.. _c2c-beta-namespace-remapping:

============================
Change Collection Namespaces
============================
=====================
Change Database Names
=====================

.. default-domain:: mongodb

Expand All @@ -12,4 +12,122 @@ Change Collection Namespaces
:depth: 2
:class: singlecol

.. include:: /includes/beta-feature.rst
.. include:: /includes/beta-feature

Starting in ``mongosync`` beta 1.8, you can remap database names during sync.
This allows you to take data in one database on the source cluster and migrate
it into a different database on the destination cluster.

Syntax
------

The :ref:`/start <c2c-api-start>` API endpoint accepts an optional ``namespaceRemap``
parameter with the following syntax:

.. code-block:: sh

curl <host>:<port>/api/v1/start -XPOST \
--data '
{
"source": "cluster0",
"destination": "cluster1"
"namespaceRemap": [
{
"from": {
"database": "<source-database>"
},
"to": {
"database": "<destination-database>"
}
}
]
} '

Parameter Fields
----------------

.. list-table::
:header-rows: 1
:stub-columns: 1
:widths: 20 13 13 54

* - Field
- Type
- Required
- Description
* - ``namespaceRemap``
- array
- Optional
- Array of documents that specify namespace changes to make during sync.

.. versionadded:: 1.8 Beta

* - ``namespaceRemap[n].`` ``from``
- document
- Required
- Document that specifies the database on the source cluster
to migrate from in the remapping operation.

.. versionadded:: 1.8 Beta

* - ``namespaceRemap[n].`` ``from.database``
- string
- Required
- Database to read from on the source cluster.

.. versionadded:: 1.8 Beta

* - ``namespaceRemap[n].`` ``to``
- document
- Required
- Document that specifies the database on the destination cluster
to write to in the remapping operation.

.. versionadded:: 1.8 Beta

* - ``namespaceRemap[n].`` ``to.database``
- string
- Required
- Database to write to on the destination cluster.

.. versionadded:: 1.8 Beta


Behavior
--------

Valid Namespace Remapping
~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: /includes/fact-valid-namespace-remap

Steps
-----

.. procedure::
:style: normal

.. step:: Connect the source and destination clusters

The following example connects a source cluster (``cluster0``)
with a destination cluster (``cluster1``):

.. code-block:: shell

mongosync \
--cluster0 "mongodb://localhost:27000" \
--cluster1 "mongodb://localhost:35000"

.. step:: Sync the clusters with namespace remapping

The following :ref:`/start <c2c-api-start>` call starts sync and remaps the
``accounts`` database on the source cluster to
``sales`` database on the destination cluster:

.. literalinclude:: /includes/api/requests/start-namespace-remap.sh
:language: shell

Learn More
----------

- :ref:`/start <c2c-api-start>`
9 changes: 9 additions & 0 deletions source/release-notes/1.8.txt
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,15 @@ Set a Migration Name

To learn more, see :ref:`c2c-beta-migration-name`.

Namespace Remapping
~~~~~~~~~~~~~~~~~~~

Starting in 1.8 Beta, ``mongosync`` can migrate data from one database
on the source cluster into a new database on the destination. This allows you
to change database names during migration.

For more information, see :ref:`c2c-beta-namespace-remapping`.

Other Notes
~~~~~~~~~~~

Expand Down