Fold create-checkpoint.sh into build-crac-checkpoint-image.sh and add docs

Author: fapdash

README.md

@@ -4,6 +4,10 @@ An [Exercism test runner][test-runner-docs] automatically verifies if a submissi
 
 This repository contains the Java test runner, which implements the V3 spec of the [test runner interface][test-runner-interface-docs].
 
+This test runner uses [CRaC (Coordinated Restore at Checkpoint)](https://crac.org/) to speed up execution.
+The build logic is implemented and documented in [bin/build-crac-checkpoint-image.sh](bin/build-crac-checkpoint-image.sh).
+If you change this logic you might also have to adjust the GitHub [deploy action](.github/workflows/deploy.yml).
+
 ## Run the test runner
 
 To run the tests of an arbitrary exercise, do the following:
@@ -31,11 +35,11 @@ To run the tests to verify the behavior of the test runner, do the following:
 1. Open a terminal in the project's root
 2. Run `./bin/run-tests.sh`
 
-These are [golden tests][golden] that compare the `results.json` generated by running the current state of the code 
+These are [golden tests][golden] that compare the `results.json` generated by running the current state of the code
 against the "known good" `tests/<test-name>/expected_results.json`.
 All files created during the test run itself are discarded.
 
-When you've made modifications to the code that will result in a new "golden" state, 
+When you've made modifications to the code that will result in a new "golden" state,
 you'll need to generate and commit a new `tests/<test-name>/expected_results.json` file.
 
 ## Run the tests using Docker
@@ -47,11 +51,11 @@ To run the tests to verify the behavior of the test runner using the Docker imag
 1. Open a terminal in the project's root
 2. Run `./bin/run-tests-in-docker.sh`
 
-These are [golden tests][golden] that compare the `results.json` generated by running the current state of the code 
-against the "known good" `tests/<test-name>/expected_results.json`. 
+These are [golden tests][golden] that compare the `results.json` generated by running the current state of the code
+against the "known good" `tests/<test-name>/expected_results.json`.
 All files created during the test run itself are discarded.
 
-When you've made modifications to the code that will result in a new "golden" state, 
+When you've made modifications to the code that will result in a new "golden" state,
 you'll need to generate and commit a new `tests/<test-name>/expected_results.json` file.
 
 [test-runner-docs]: https://exercism.org/docs/building/tooling/test-runners

bin/build-crac-checkpoint-image.sh

@@ -3,15 +3,66 @@
 # Synopsis:
 # Build a Docker image containing a CraC checkpoint to restart from.
 # An initial image is built. A container is created from that image
-# and tests are run to warm up the JVM. Then a checkpoint is created.
-# The final image is created by committing the containiner
-# containing the checkpoint.
+# and tests are run to warm up the JVM. Once the runner has finished running
+# all the tests it creates a checkpoint of the VM process before exiting.
+# This checkpoint is written to the host file system, so it can be copied
+# into the final image.
+# Then the final image is created, which restores from the checkpoint
+# instead of starting a new JVM
+# This avoids slow JVM start up and JVM warm up costs.
+#
+# Example:
+# ./bin/build-crac-checkpoint-image.sh
 
-# build outside of Docker container, so we can copy jar into both images
+create_checkpoint() {
+    # Copy all tests into one merged project, so we can warm up the JVM
+    # TODO(FAP): this is missing some tests as most tests use the same filenames
+    mkdir -p tests/merged
+    for dir in tests/*; do
+        if [ -d "$dir" ] && [ "$dir" != "tests/merged/" ]; then
+            rsync -a "$dir"/ tests/merged/
+        fi
+    done
+
+    real_path() {
+        echo "$(cd "$(dirname -- "$1")" >/dev/null; pwd -P)/$(basename -- "$1")";
+    }
+
+    mkdir -p build/cr
+
+    image_tag="$1"
+    slug="merged"
+    solution_dir=$(realpath "tests/merged/")
+    output_dir=$(realpath "tests/merged/")
+
+    docker run --cap-add CHECKPOINT_RESTORE \
+           --cap-add SYS_PTRACE \
+           --name java-test-runner-crac \
+           --network none \
+           --mount type=bind,src="${solution_dir}",dst=/solution \
+           --mount type=bind,src="${output_dir}",dst=/output \
+           --mount type=bind,src="$(real_path build/cr)",dst=/opt/test-runner/crac-checkpoint \
+           --mount type=tmpfs,dst=/tmp \
+           "${image_tag}" "${slug}" /solution /output
+
+    docker rm -f java-test-runner-crac
+    rm -rf tests/merged/
+}
+
+# 1. Build jar outside of Docker, so we can copy the jar into both images
+echo "build-crac-checkpoint-image.sh: Building jar with Gradle"
 ./gradlew build
 
-docker build -t exercism/java-test-runner-crac-checkpoint -f Dockerfile.createCheckpoint .
+image_tag="exercism/java-test-runner-crac-checkpoint"
+
+# 2. Build first image with an entrypoint that will create a CraC checkpoint
+echo "build-crac-checkpoint-image.sh: Building image that creates CraC checkpoint"
+docker build -t "$image_tag" -f Dockerfile.createCheckpoint .
 
-bin/create-checkpoint.sh
+# 3. Run a container from image created in step 2 and create CraC checkpoint
+echo "build-crac-checkpoint-image.sh: Creating CraC checkpoint"
+create_checkpoint "$image_tag"
 
+# Build final test runner image, includes the jar and the checkpoint created in step 1 and 3
+echo "build-crac-checkpoint-image.sh: Building final test runner image"
 docker build -t exercism/java-test-runner -f Dockerfile .