Skip to content
This repository was archived by the owner on May 1, 2025. It is now read-only.

Commit 9407332

Browse files
authored
DOCSP-46301-clean-verify-data (#617) (#641)
* DOCSP-46301-clean-verify-data * move code * fix indent errors * changing table * fix links * feedback * build errors
1 parent 076e929 commit 9407332

File tree

2 files changed

+91
-61
lines changed

2 files changed

+91
-61
lines changed

source/reference/verification.txt

Lines changed: 7 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -92,73 +92,18 @@ source to the destination cluster.
9292
and replica sets that use MongoDB 4.4 or earlier
9393
releases, since the document field order can vary.
9494

95-
* - Document Comparison
96-
- You can verify sync by comparing documents on the source
97-
and destination clusters.
98-
99-
Write a script that queries collections on the source
100-
cluster and then checks that the correct documents,
101-
indexes, collections, metadata, and views exist with the
102-
same values on the destination cluster.
103-
104-
Before you can verify data transfer with this method,
105-
``mongosync`` must be in the ``COMMITTED`` state.
106-
107-
* - .. _c2c-index-comparison:
108-
109-
Index Comparison
110-
- To verify the transfer of indexes, run the
111-
:method:`db.collection.getIndexes` method on the source
112-
and destination clusters and compare the results.
113-
114-
* - .. _c2c-metadata-comparison:
115-
116-
Metadata Comparison
117-
- To verify the transfer of metadata, run the
118-
:method:`db.getCollectionInfos` method on the source and
119-
destination clusters and compare the results.
120-
121-
* - .. _c2c-shardkey-comparison:
122-
123-
Shard Key Comparison
124-
- To verify the transfer of shard keys to a synced collection, run a query on the ``config.collections``
125-
collection to find a document whose ``_id`` value is the namespace of the target collection.
126-
Compare the ``key`` value of this document in the source and destination clusters.
127-
128-
For example, for a collection named ``pets`` in the ``records`` database, you can verify the shard key
129-
using the following query in :binary:`mongosh`:
130-
131-
.. io-code-block::
132-
:copyable: true
133-
134-
.. input::
135-
:language: javascript
136-
137-
db.getSiblingDB("config").collections.find({ _id : "records.pets" })
138-
139-
.. output::
140-
:language: javascript
141-
:emphasize-lines: 5-7
142-
:visible: false
143-
144-
{
145-
"_id" : "records.pets",
146-
"lastmod" : ISODate("2021-07-21T15:48:15.193Z"),
147-
"timestamp": Timestamp(1626882495, 1),
148-
"key" : {
149-
"_id" : 1
150-
},
151-
"unique" : false,
152-
"lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc"),
153-
"uuid" : UUID("f8669e52-5c1b-4ea2-bbdc-a00189b341da")
154-
}
155-
15695
* - :ref:`Migration Verifier <c2c-migration-verifier>`
15796
- Migration Verifier connects to the source and destination
15897
clusters and performs a series of verification checks,
15998
comparing documents, views, and indexes to confirm the
16099
sync was successful.
161100

101+
* - :ref:`Verifying Data with Custom Scripts <c2c-custom-manual-verification>`
102+
- Different types of data require different methods of verification. You can
103+
can use custom or manual verification to verify :ref:`documents <c2c-document-comparison>`,
104+
:ref:`indexes <c2c-index-comparison>`, :ref:`metadata <c2c-metadata-comparison>`,
105+
and :ref:`shard keys <c2c-shardkey-comparison>`.
106+
162107
The specific method you use to verify your data depends on your
163108
application workload and the complexity of the data.
164109

@@ -169,6 +114,7 @@ application workload and the complexity of the data.
169114
Use Document Counts </reference/verification/count>
170115
Use Hash Comparison </reference/verification/hash>
171116
Use Migration Verifier </reference/verification/verifier>
117+
Verifying Data with Custom Scripts </reference/verification/custom-manual>
172118

173119
Learn More
174120
----------
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
.. _c2c-custom-manual-verification:
2+
3+
==================================
4+
Verifying Data with Custom Scripts
5+
==================================
6+
7+
.. default-domain:: mongodb
8+
9+
.. facet::
10+
:name: genre
11+
:values: tutorial
12+
13+
.. contents:: On this page
14+
:local:
15+
:backlinks: none
16+
:depth: 1
17+
:class: singlecol
18+
19+
You can write a script to query your source cluster and compare the documents,
20+
indexes, collections, metadata, and views with the destination cluster.
21+
22+
.. _c2c-document-comparison:
23+
24+
Comparing Documents
25+
-------------------
26+
27+
You can verify sync by comparing documents on the source and destination clusters.
28+
29+
Before you can verify data transfer with this method, ``mongosync`` must be in
30+
the ``COMMITTED`` state.
31+
32+
.. _c2c-index-comparison:
33+
34+
Comparing Indexes
35+
-----------------
36+
37+
To verify the transfer of indexes, run :method:`db.collection.getIndexes`
38+
on the source and destination clusters and compare the results.
39+
40+
.. _c2c-metadata-comparison:
41+
42+
Comparing Metadata
43+
------------------
44+
45+
To verify the transfer of metadata, run :method:`db.getCollectionInfos`
46+
on the source and destination clusters and compare the results.
47+
48+
.. _c2c-shardkey-comparison:
49+
50+
Comparing Shard Keys
51+
--------------------
52+
53+
To verify the transfer of shard keys to a synced collection, run a query on the
54+
``config.collections`` collection to find a document whose ``_id`` value is the
55+
namespace of the target collection. Compare the ``key`` value of this document
56+
in the source and destination clusters.
57+
58+
For example, for a collection named ``pets`` in the ``records`` database, you
59+
can verify the shard key using the following query in :binary:`mongosh`:
60+
61+
.. io-code-block::
62+
:copyable: true
63+
64+
.. input::
65+
:language: javascript
66+
67+
db.getSiblingDB("config").collections.find({ _id : "records.pets" })
68+
69+
.. output::
70+
:language: javascript
71+
:emphasize-lines: 5-7
72+
:visible: false
73+
74+
{
75+
"_id" : "records.pets",
76+
"lastmod" : ISODate("2021-07-21T15:48:15.193Z"),
77+
"timestamp": Timestamp(1626882495, 1),
78+
"key" : {
79+
"_id" : 1
80+
},
81+
"unique" : false,
82+
"lastmodEpoch" : ObjectId("5078407bd58b175c5c225fdc"),
83+
"uuid" : UUID("f8669e52-5c1b-4ea2-bbdc-a00189b341da")
84+
}

0 commit comments

Comments
 (0)