Skip to content
This repository was archived by the owner on May 1, 2025. It is now read-only.

Docsp 46301 clean verify data v1.12 #641

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 7 additions & 61 deletions source/reference/verification.txt
Original file line number Diff line number Diff line change
Expand Up @@ -92,73 +92,18 @@ source to the destination cluster.
and replica sets that use MongoDB 4.4 or earlier
releases, since the document field order can vary.

* - Document Comparison
- You can verify sync by comparing documents on the source
and destination clusters.

Write a script that queries collections on the source
cluster and then checks that the correct documents,
indexes, collections, metadata, and views exist with the
same values on the destination cluster.

Before you can verify data transfer with this method,
``mongosync`` must be in the ``COMMITTED`` state.

* - .. _c2c-index-comparison:

Index Comparison
- To verify the transfer of indexes, run the
:method:`db.collection.getIndexes` method on the source
and destination clusters and compare the results.

* - .. _c2c-metadata-comparison:

Metadata Comparison
- To verify the transfer of metadata, run the
:method:`db.getCollectionInfos` method on the source and
destination clusters and compare the results.

* - .. _c2c-shardkey-comparison:

Shard Key Comparison
- To verify the transfer of shard keys to a synced collection, run a query on the ``config.collections``
collection to find a document whose ``_id`` value is the namespace of the target collection.
Compare the ``key`` value of this document in the source and destination clusters.

For example, for a collection named ``pets`` in the ``records`` database, you can verify the shard key
using the following query in :binary:`mongosh`:

.. io-code-block::
:copyable: true

.. input::
:language: javascript

db.getSiblingDB("config").collections.find({ _id : "records.pets" })

.. output::
:language: javascript
:emphasize-lines: 5-7
:visible: false

{
"_id" : "records.pets",
"lastmod" : ISODate("2021-07-21T15:48:15.193Z"),
"timestamp": Timestamp(1626882495, 1),
"key" : {
"_id" : 1
},
"unique" : false,
"lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc"),
"uuid" : UUID("f8669e52-5c1b-4ea2-bbdc-a00189b341da")
}

* - :ref:`Migration Verifier <c2c-migration-verifier>`
- Migration Verifier connects to the source and destination
clusters and performs a series of verification checks,
comparing documents, views, and indexes to confirm the
sync was successful.

* - :ref:`Verifying Data with Custom Scripts <c2c-custom-manual-verification>`
- Different types of data require different methods of verification. You can
can use custom or manual verification to verify :ref:`documents <c2c-document-comparison>`,
:ref:`indexes <c2c-index-comparison>`, :ref:`metadata <c2c-metadata-comparison>`,
and :ref:`shard keys <c2c-shardkey-comparison>`.

The specific method you use to verify your data depends on your
application workload and the complexity of the data.

Expand All @@ -169,6 +114,7 @@ application workload and the complexity of the data.
Use Document Counts </reference/verification/count>
Use Hash Comparison </reference/verification/hash>
Use Migration Verifier </reference/verification/verifier>
Verifying Data with Custom Scripts </reference/verification/custom-manual>

Learn More
----------
Expand Down
84 changes: 84 additions & 0 deletions source/reference/verification/custom-manual.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
.. _c2c-custom-manual-verification:

==================================
Verifying Data with Custom Scripts
==================================

.. default-domain:: mongodb

.. facet::
:name: genre
:values: tutorial

.. contents:: On this page
:local:
:backlinks: none
:depth: 1
:class: singlecol

You can write a script to query your source cluster and compare the documents,
indexes, collections, metadata, and views with the destination cluster.

.. _c2c-document-comparison:

Comparing Documents
-------------------

You can verify sync by comparing documents on the source and destination clusters.

Before you can verify data transfer with this method, ``mongosync`` must be in
the ``COMMITTED`` state.

.. _c2c-index-comparison:

Comparing Indexes
-----------------

To verify the transfer of indexes, run :method:`db.collection.getIndexes`
on the source and destination clusters and compare the results.

.. _c2c-metadata-comparison:

Comparing Metadata
------------------

To verify the transfer of metadata, run :method:`db.getCollectionInfos`
on the source and destination clusters and compare the results.

.. _c2c-shardkey-comparison:

Comparing Shard Keys
--------------------

To verify the transfer of shard keys to a synced collection, run a query on the
``config.collections`` collection to find a document whose ``_id`` value is the
namespace of the target collection. Compare the ``key`` value of this document
in the source and destination clusters.

For example, for a collection named ``pets`` in the ``records`` database, you
can verify the shard key using the following query in :binary:`mongosh`:

.. io-code-block::
:copyable: true

.. input::
:language: javascript

db.getSiblingDB("config").collections.find({ _id : "records.pets" })

.. output::
:language: javascript
:emphasize-lines: 5-7
:visible: false

{
"_id" : "records.pets",
"lastmod" : ISODate("2021-07-21T15:48:15.193Z"),
"timestamp": Timestamp(1626882495, 1),
"key" : {
"_id" : 1
},
"unique" : false,
"lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc"),
"uuid" : UUID("f8669e52-5c1b-4ea2-bbdc-a00189b341da")
}