chore(docs): add execution flow of CLI REPL eval (#1024)

Author: rose-m

docs/cli-code-evaluation.md

@@ -0,0 +1,44 @@
+# CLI REPL Code Evaluation
+
+This document describes the evaluation of user code in the [CLI REPL](../packages/cli-repl),
+i.e. what happens when the user types `db.coll.find({})` in the CLI REPL and hits enter.
+
+## Involved Components
+
+### Async REPL
+The [`async-repl`](../packages/cli-repl/src/async-repl.ts) is responsible for setting up the Node.js
+`REPLServer` that `mongosh` is using for the CLI. Most notabily we customize the `REPLServer.eval` function
+to disable certain default REPL behavior and set up customized interrupt and error handling.
+
+### Mongosh Node REPL
+The [`MongoshNodeRepl`](../packages/cli-repl/src/mongosh-repl.ts) is our main interface and orchestrator for the CLI. However it does not do any I/O
+on its own but relies on an external I/O provider. It initializes and manages the Shell API abstraction
+as well as handles things like access to the history or startup messages.
+
+### Shell Evaluator
+The [`ShellEvaluator`](../packages/shell-evaluator) is an intermediate package also used by non-CLI uses of the shell and is responsible
+for correctly transforming user input before evaluation. It relies on the `async-rewriter2` for input
+transformation.
+The Shell Evaluator also detects linux-commands like `show dbs` and triggers their execution without further code transformation or evaluation.
+
+### Async Rewriter
+The [`async-rewriter2`](../packages/async-rewriter2) transforms code written in a synchronous way into
+code with automatically inserted `await` statements to resolve promises returned by the Shell API.
+
+## Execution
+
+1. User enters `db.coll.find({})` and hits enter.
+2. The `REPLServer.eval` function is called which is customized in [`async-repl`](../packages/cli-repl/src/async-repl.ts).
+3. The customized `eval` function triggers [`MongoshNodeRepl.eval`](../packages/cli-repl/src/mongosh-repl.ts).
+4. `MongoshNodeRepl` forwards the user input to [`ShellEvaluator.customEval`](../packages/shell-evaluator/src/shell-evaluator.ts).
+5. `ShellEvaluator.customEval` calls `ShellEvaluator.innerEval`:
+   1. For linux-style commands, e.g. `show dbs`, processing is immediately triggered and the result is returned up the chain.
+   2. For JavaScript user input:
+      1. `ShellEvaluator` calls the `async-rewriter` to rewrite user input.
+      2. Rewritten user input is now evaluated in the current Shell API context using the _original, non-modified_ `REPLServer.eval`
+	 function that has been passed down.
+      3. The evaluation result is returned up the chain.
+
+See also the following diagram which additionally includes important events emitted during the execution.
+
+![](cli-code-evaluation.png)

docs/cli-code-evaluation.png

@@ -0,0 +1 @@
+<mxfile host="Electron" modified="2021-07-20T09:31:16.378Z" agent="5.0 (Macintosh; Intel Mac OS X 11_4_0) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/14.6.13 Chrome/89.0.4389.128 Electron/12.0.7 Safari/537.36" etag="m0ryZ81Uoev6-Ma22SGn" version="14.6.13" type="device"><diagram id="Ff37bpA9ezkQJEPIHSPT" name="CLI REPL Evaluation">7Vxbc6M2FP41nmkfkuFq8GMuTtOZbLoTz06zTx0CslEXEBVybO+vr2SEuUi2aYKRG5J9WOsgCXG+T+foSAdG5k28/g17afgFBSAaGVqwHpm3I8PQxxOX/sckm1ziToxcsMAw4JVKwQz+BFyocekSBiCrVSQIRQSmdaGPkgT4pCbzMEarerU5iup3Tb0Fv6NWCma+FwGh2p8wICF/CsMp5fcALsLizvSJ8yuxV1TmHWehF6BVRWROR+YNRojkv+L1DYiY8gq95O3u9lzdDQyDhLRp8Nf0HjyQ52+beBFPn7Hm/x6sLgo1Z2RTPDEIqAJ4EWESogVKvGhaSq8xWiYBYN1qtFTWeUAopUKdCv8GhGw4mt6SICoKSRzxq3OUkDsvhhEjxQ1aYggwHcQjoOq5BmtInlnfl4bNi995O/b7ds3vuy1sikJC8KbaipW/Vy+W7balomH++OyZ96qVizI6Th8c0iWnp4cXgByoN9mBT2cNQDGg46HtMIg8Al/r4/A4fRe7eiXC9AcH+b8Anvf76kVLfqen6deHGcCvAF8CKhb4UKLNIFiFkIBZ6m01saJzvoEsjKIbFCG8bWvObfaPyjOC0Q9QuTLe/nEuVOT5H2vxAxA/5N3yQQNMwPowTqJed6aIU50bIkPj5VU5rXeysDKlLe1EUFgqpl6HfDda8n0PLv3w3RD4Lmo9iqg3AcfJ7WVp7mLmcM0w6IKWhl2npa7LaCmy0jgVK21BYcHLpY+i6HIOk+CXkXM9cm5/FZRINUDq2qpP+QQloGEfuMiL4CKhRZ9qjLoB85rpE1IPfMUvxDAIon3w1CfEQcfSBVpWHS1XBMvqEyxdwuZP9/02czZpa87ea894068I0iGW1HLq1LLMBmnygfFWDd7shvEOQzlWQaWSLU6VLVpbtjg1tuhH2BJ4WbhbybSwFf3ySqWbnAhW38s2iX+BQRp1sCw8wfrNNFSv33QlM+ZjGl+9bfCkm0pXk/ankTyJkWyNv9rgWYyev6BkgbLwEQXg6WxtpWUpt5VKJs4HtZVmW1tpKbWVSvY3hmArW+OvdElZDLNiLGchbTdlAo8gfOkvM4Li6TlaTHus3GJOzs5idslhqy2HnXdyeNv0CmNvU6mQsig6q/TcCMb1wl9xOownjbOVY/VdrUGAfATdRuz62THk/+tTW/PRVepTzU+fqtgeKY0/i2Hu96kwSQA+S5c6lmyW9+tSdyfx52Mwa2Tvks9OWz6/N0Z4m3/VGtw45l8b9R2tB/9a6LAy2a7YpigVPYEVhuy06l2T7ASH5Vo3c5euPhsAOconr/HxXZ8aU+H2ZSre5/pcYTb+geECJszVaeeexNLRvGz6VFd5EoshGkl2ZjRivvaOUZReuIMJzEIBkyz0UvYzI4ABkAIM6ZhYCsBW9LUsHwdsDYrcQb2p9oOTqkg+iMCcpS9k9BYwWTxsS7f6FnTebUcIms7xY6wdyr3kEBji2d/LMtsBGOcb3CPzikF54aM4jSgqn2jK0JRutJt9olncrBWaNGQAFzBJl+Ss4cS53hp4ag045xFM74t75DWfuMK30HexLHIby1BTAnevCUCmcdz8zqhn/wT4MMCFNZbMX93uFVBx1/xRXPi+OWNxPzb3IHoFLO9uJEnJa6yIAg+4c1+6IvJd8DI/URAiy42UGddm5lR34IjbL5KoZBDgNLNxJPFhv9CIeauSvdJBQKM3l5iurRibsYCN5Gx4kNjYmmpsxPhNkqoxSGzGhmpsxC0PyaHwILFxLNXYiFGzJB10GNg0N/Mli+hesSnuX8HGGSg2hjyPQR00YrzqDhSaxqtEjiTjqF9oxMhT9nbwILDZ7cqei7uxxMhT9vrBIMExxqrBEWNPfagBThMc5cGnJQafsrcmBwmO8ujTEqPP4h2LwYOjPPy0xPBT9hrHIMCxzPpCWvbmd7/giPGnPtQNT6t5CivJiugVHFsMQGfgnyVIqPYNLVnGL5LUMfELCo3j7L3fRcAggz+9l21HDC6efUd7ta9H9i3raUlQVjmhk36aYR/IHSB0oQufuRCN2+54p5/vXIjv8gmQfJ6RtjqWs9tNt9NBKW4qTGNICDVkhgZeQSKefg9rskkAmvQKkGRrYaC+qrmRfcLglRbLr8vlSdHlN/rM6b8=</diagram></mxfile>

docs/diagrams.drawio

@@ -0,0 +1,10 @@
+# Development Documentation
+
+This folder contains development-specific documentation on the inner workings of `mongosh` and its packages.
+
+> If you are looking for end-user documentation, please refer to: https://docs.mongodb.com/mongodb-shell/.
+
+## Topics
+Go to any of the linked topics for details:
+
+* [CLI REPL Code Evaluation](./cli-code-evaluation.md)