start.txt

.. _c2c-api-start:

=========
``start``
=========


.. default-domain:: mongodb

.. contents:: On this page
   :local:
   :backlinks: none
   :depth: 1
   :class: singlecol

Description
-----------

Starts the synchronization between a source and destination cluster.

Requirements
------------

State
~~~~~

To use the ``start`` endpoint, ``mongosync`` must be in the ``IDLE``
state.

Permissions
~~~~~~~~~~~

The user specified in the ``mongosync`` connection string must have the
required permissions on the source and destination clusters. Refer to
:ref:`c2c-permissions-and-roles` to ensure that the user has the correct
permissions to start the synchronization.

Multiple ``mongosync`` Instances
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Ensure that you use the configured ``mongosync`` user in the connection 
strings for the :setting:`cluster0` or :setting:`cluster1` settings when
you start ``mongosync``. 

.. note::

   .. include:: /includes/api/facts/multiple-mongosync-endpoints

   For more information, see :ref:`Start Multiple Mongosyncs
   <c2c-sharded-start>`. 

Request
-------

.. code-block:: http
   :copyable: false

   POST /api/v1/start 

Request Body Parameters
~~~~~~~~~~~~~~~~~~~~~~~

.. list-table::
   :header-rows: 1
   :stub-columns: 1
   :widths: 20 13 13 54

   * - Parameter
     - Type
     - Necessity
     - Description

   * - ``source``
     - string
     - Required
     - Name of the source cluster.
    
   * - ``destination``
     - string
     - Required
     - Name of the destination cluster.

   * - ``buildIndexes``
     - string
     - Optional
     - Configures index builds during sync.

       Supported Options:

       - ``beforeDataCopy`` (the default) causes ``mongosync`` to build indexes
         on the destination cluster. These include both existing indexes and
         any indexes created during migration on the source cluster.

       - ``never`` causes ``mongosync`` to skip building unnecessary indexes 
         during sync.  This can improve migration performance, especially with 
         index heavy workloads.

         .. warning::

            Do **not** manually build indexes while ``mongosync`` is 
            performing a migration.  Wait until the migration is fully 
            committed.

         For more information on the indexes it does build, see 
         :ref:`c2c-required-indexes`.

       ``/start`` returns an error when ``buildIndexes`` is set to ``never``
       and ``reversible`` is set to ``true``.

       If you call ``/start`` without specifying the ``buildIndexes`` option,
       ``mongosync`` builds indexes on the destination cluster.

       .. versionadded:: 1.3.0

   * - ``documentFilter``
     - document
     - Optional
     - .. important::
          
          This feature is only available in the :ref:`c2c-beta-program`.

       .. include:: /includes/document-filtering-intro.rst
       
       To learn more, see :ref:`c2c-beta-document-filtering`.

       .. versionadded:: 1.8 Beta

   * - ``enableUserWriteBlocking``
     - boolean
     - Optional
     - If set to ``true``, blocks writes on the destination cluster
       while the synchronization is in progress. After the
       synchronization is committed to the destination cluster, the
       original source cluster blocks writes and the destination cluster
       accepts writes.

       To reverse sync, the ``enableUserWriteBlocking`` field must be set
       to ``true``. To allow the source cluster to accept writes again,
       for example after running migration tests, run the following
       command: 

       .. code-block:: shell 
          
          db.adminCommand( { setUserWriteBlockMode: 1, global: false } )

       Default value is ``false``.

   * - ``includeNamespaces``
     - array
     - Optional
     - Filters the databases or collections to include in sync. 

       .. include:: /includes/api/facts/namespace-explanation.rst

       .. versionadded:: 1.1

   * - ``excludeNamespaces``
     - array
     - Optional
     - Filters the databases or collections to exclude from sync.

       .. include:: /includes/api/facts/namespace-explanation.rst

       .. versionadded:: 1.6

   * - ``namespaceRemap``
     - array
     - Optional
     - .. important::
          
          This feature is only available in the :ref:`c2c-beta-program`.

       Array of documents that specify namespace changes to make during sync.

       For more information, see :ref:`c2c-beta-namespace-remapping`.

       .. versionadded:: 1.8 Beta

   * - ``reversible``
     - boolean
     - Optional
     - If set to ``true``, enables the sync operation to be
       reversed. 

       To reverse sync, the ``enableUserWriteBlocking`` field must be set 
       to ``true``.      

       This option is not supported for the following configurations:

       * Sync from a replica set to a sharded cluster

       * Sync sharded clusters that have different numbers of shards

       * Reversible sync when ``buildIndexes`` is set to ``never``.

       For more information, see the :ref:`reverse <c2c-api-reverse>` endpoint.
       
       Default value is ``false``.

   * - ``sharding``
     - document
     - Optional
     - Configures sync between a replica set and sharded cluster.
       Sync from a replica set to a sharded cluster requires this
       option.

       For more information, see :ref:`c2c-api-start-sharding`. 

       .. versionadded:: 1.1

.. _c2c-api-start-sharding:

Sharding Parameters
~~~~~~~~~~~~~~~~~~~

.. versionadded:: 1.1

To sync from a replica set to a sharded cluster, set the 
``sharding`` option to shard collections on the destination cluster.

The ``sharding`` option has the following parameters:

.. list-table::
   :header-rows: 1
   :stub-columns: 1
   :widths: 15 15 70

   * - Parameter
     - Type
     - Description

   * - ``createSupportingIndexes``
     - boolean
     - Optional. Sets whether sync creates a supporting index 
       for the shard key, if none exists.  Defaults to ``false``.

   * - ``shardingEntries``
     - array of documents
     - Required. Sets the namespace and key of collections to shard 
       during sync.

       Collections not included in this array sync to unsharded 
       collections on the destination cluster.  If set with an empty
       array, no collections are sharded.

   * - | ``shardingEntries``
       | ``.collection``
     - string
     - Sets the collection to shard. 

   * - | ``shardingEntries``
       | ``.database``
     - string
     - Sets the database of the collection to shard.
 
   * - | ``shardingEntries``
       | ``.shardCollection``
     - document 
     - Sets the shard key to generate on the destination cluster.

   * - | ``shardingEntries``
       | ``.shardCollection``
       | ``.key``
     - array 
     - Sets the fields to use for the shard key.

       For more information, see :ref:`shard-key`.

``mongosync`` throws an error if the ``sharding`` option is not set when
syncing from a replica set to a sharded cluster.  ``mongosync`` also
throws an error if the ``sharding`` option is set with any other
configuration.

Response
--------

.. include:: /includes/api/tables/basic-response.rst

Example: Start a Sync Job
-------------------------

.. include:: /includes/intro-start-api-example-intro.rst

Request
~~~~~~~

.. literalinclude:: /includes/api/requests/start.sh
   :language: shell

Response
~~~~~~~~

.. literalinclude:: /includes/api/responses/success.json
   :language: json
   :copyable: false

Example: Start a Reversible Sync Job
------------------------------------

.. include:: /includes/intro-start-api-example-intro.rst

The ``reversible`` and ``enableUserWriteBlocking`` fields allow the sync
to be reversed. To reverse the sync direction, see: :ref:`reverse
<c2c-api-reverse>`.

Request
~~~~~~~

.. literalinclude:: /includes/api/requests/start-reversible.sh
   :language: shell

Response
~~~~~~~~

.. literalinclude:: /includes/api/responses/success.json
   :language: json
   :copyable: false


Example: Start a Filtered Sync Job
----------------------------------

.. include:: /includes/example-filter-collection.rst

The ``includeNamespaces`` option creates a filter. To filter the sync,
see: :ref:`c2c-filtered-sync`

Request
~~~~~~~

.. literalinclude:: /includes/api/requests/start-filtered.sh
   :language: shell

Response
~~~~~~~~

.. literalinclude:: /includes/api/responses/success.json
   :language: json
   :copyable: false

Example: Start Sync from Replica Set to Sharded Cluster
-------------------------------------------------------

Request
~~~~~~~

.. literalinclude:: /includes/api/requests/start-rs-shard.sh

Response
~~~~~~~~

.. literalinclude:: /includes/api/responses/success.json
   :language: json
   :copyable: false

Behavior
--------

State
~~~~~

If the ``start`` request is successful, ``mongosync`` enters the
``RUNNING`` state.

Pre-Split Chunks
~~~~~~~~~~~~~~~~

.. versionadded:: 1.1

When ``mongosync`` syncs to a sharded cluster, it pre-splits chunks for
sharded collections on the destination cluster.  This is supported in the
following configurations:

* Sync from a replica set to a sharded cluster.

* Sync between sharded clusters that differ in the number of shards.


Shard Replica Sets 
~~~~~~~~~~~~~~~~~~

.. versionadded:: 1.1

Sync from a replica set to a sharded cluster requires the 
``sharding`` option. This option configures how ``mongosync`` shards
collections.

The ``sharding.shardEntries`` array specifies the collections to shard.
Collections that are not listed in this array replicate as unsharded.

Supporting Indexes
~~~~~~~~~~~~~~~~~~

.. versionadded:: 1.1

``mongosync`` syncs indexes from the source cluster to the destination
cluster.  But, when syncing from a replica set to a sharded cluster,
``mongosync`` may require an additional index to support the shard key,
which may not exist on the source cluster.

``mongosync`` can create supporting indexes for sharded collections during sync.
This is done by setting the ``sharding.createSupportingIndexes`` option.

When ``sharding.createSupportingIndexes`` is ``false`` (the default): 

* Each shard key you provide for the ``sharding.shardEntries`` option
  must have an existing index on the source cluster.

* One of the indexes used for the shard key must have simple collation if the 
  collection uses any other collation.

* To use a unique index in the shard key, you must specify its uniqueness when
  you create the index on the source cluster. 

* Unique indexes on the source cluster that are incompatible with the requested
  shard key on the destination cluster, such as a unique index on the source
  that does not contain the requested shard key as a prefix on the destination,
  can cause ``mongosync`` to fail.

When ``sharding.createSupportingIndexes`` is ``true``: 

* If the supporting indexes exist on the source cluster, ``mongosync`` 
  syncs the indexes to the destination cluster and uses them 
  as shard keys. 

* If the supporting indexes don't exist, ``mongosync`` creates them on the
  destination cluster.

The ``sharding.createSupportingIndexes`` option affects all sharded
collections.

.. _rename-during-sync:

Rename During Sync
~~~~~~~~~~~~~~~~~~

.. versionadded:: 1.1

Collections listed in the ``sharding.shardEntries`` array 
when synced from a replica set to a sharded cluster 
become sharded collections on the destination cluster.

Renaming a collection (such as with the :dbcommand:`renameCollection` command)
on the source cluster after calling ``start`` but before ``mongosync`` begins 
to copy the collection can block the collection from sharding on the destination.

.. note:: 

   Renaming collections to use a different database while syncing from a
   replica set to a sharded cluster is not supported.
    

To check whether it is safe to rename collections, call the 
:ref:`c2c-api-progress` endpoint and check the value of the 
``collectionCopy.estimatedCopiedBytes`` field in the return document.  

* A value of 0 indicates that ``mongosync`` has not started to copy the
  collection.

  Renaming a collection at this point may result in an unsharded collection 
  on the destination cluster, as the transition to copying can happen before
  the rename takes effect on the source.

* A value greater than 0 indicates that ``mongosync`` has started the copy.
  Renaming the collection from this point on does not block
  its sharding on the destination cluster, even in the event  of a crash.

.. _c2c-required-indexes:

Required Indexes
~~~~~~~~~~~~~~~~

When you call ``/start`` with the ``buildIndexes`` option set to ``never``,
``mongosync`` skips building unnecessary indexes.  

Indexes that are always built include:

* ``mongosync`` builds an index on the ``_id`` field of every
  collection it copies.

* ``mongosync`` builds dummy indexes that support the shard key for each
  sharded collection, which are removed after commit. When ``buildIndexes`` is
  set to ``never``, ``mongosync`` retains this index after commit.


Endpoint Protection
~~~~~~~~~~~~~~~~~~~

.. |endpoint| replace:: ``start``
.. include:: /includes/fact-api-endpoint