docs: overhaul for scripts and their end-to-end tests (#1187)

## PR Checklist

- [x] Addresses an existing open issue: fixes #1141; fixes #1186
- [x] That issue was marked as [`status: accepting
prs`](https://github.com/JoshuaKGoldberg/create-typescript-app/issues?q=is%3Aopen+is%3Aissue+label%3A%22status%3A+accepting+prs%22)
- [x] Steps in
[CONTRIBUTING.md](https://github.com/JoshuaKGoldberg/create-typescript-app/blob/main/.github/CONTRIBUTING.md)
were taken

## Overview

Adds some inline comments to end-to-end tests, as well as a more full
description of what to do with migration snapshots in development docs.
Also simplifies the migration test error to suggest looking at the docs.

Touches a little bit on what #1045 requests around more clear and
standardized token docs.

Co-authored-by: John Reilly <[email protected]>
Co-authored-by: Mohammad Bagher Abiyat <[email protected]>

Author: 3 people

Diff for: .github/DEVELOPMENT.md

@@ -119,6 +119,8 @@ Each follows roughly the same general flow:
 
 ### The Creation Script
 
+> 📝 See [`docs/Creation.md`](../docs/Creation.md) for user documentation on the creation script.
+
 This template's "creation" script is located in `src/create/`.
 You can run it locally with `node bin/index.js --mode create`.
 Note that files need to be built with `pnpm run build` beforehand.
@@ -142,6 +144,8 @@ See `.github/workflows/test-create.yml`.
 
 ### The Initialization Script
 
+> 📝 See [`docs/Initialization.md`](../docs/Initialization.md) for user documentation on the initialization script.
+
 This template's "initialization" script is located in `src/initialize/`.
 You can run it locally with `pnpm run initialize`.
 It uses [`tsx`](https://github.com/esbuild-kit/tsx) so you don't need to build files before running.
@@ -172,6 +176,8 @@ See `.github/workflows/test-initialize.yml`.
 
 ### The Migration Script
 
+> 📝 See [`docs/Migration.md`](../docs/Migration.md) for user documentation on the migration script.
+
 This template's "migration" script is located in `src/migrate/`.
 Note that files need to be built with `pnpm run build` beforehand.
 
@@ -194,6 +200,9 @@ node ../create-typescript-app/bin/migrate.js
 
 #### Testing the Migration Script
 
+> 💡 Seeing `Oh no! Running the migrate script unexpectedly modified:` errors?
+> _[Unexpected File Modifications](#unexpected-file-modifications)_ covers that below.
+
 You can run the end-to-end test for migrating locally on the command-line:
 
 ```shell
@@ -210,5 +219,46 @@ The `pnpm run test:migrate` script is run in CI to ensure that templating change
 See `.github/workflows/test-migrate.yml`.
 
 > Tip: if the migration test is failing in CI and you don't see any errors, try [downloading the full logs](https://docs.github.com/en/actions/monitoring-and-troubleshooting-workflows/using-workflow-run-logs#downloading-logs).
-> There'll likely be a list of changed files under a message like _`Oh no! Running the migrate script modified some files:`_.
-> You can also try running the test script locally.
+
+##### Migration Snapshot Failures
+
+The migration test uses the [Vitest file snapshot](https://vitest.dev/guide/snapshot#file-snapshots) in `script/__snapshots__/migrate-test-e2e.js.snap` to store expected differences to this repository after running the migration script.
+The end-to-end migration test will fail any changes that don't keep the same differences in that snapshot.
+
+You can update the snapshot file by:
+
+1. Committing any changes to your local repository
+2. Running `pnpm i` and `pnpm build` if any updates have been made to the `package.json` or `src/` files, respectively
+3. Running `pnpm run test:migrate -u` to update the snapshot
+
+At this point there will be some files changed:
+
+- `script/__snapshots__/migrate-test-e2e.js.snap` will have updates if any files mismatched templates
+- The actual updated files on disk will be there too
+
+If the snapshot file changes are what you expected, then you can commit them.
+The rest of the file changes can be reverted.
+
+> [🚀 Feature: Add a way to apply known file changes after migration #1184](https://github.com/JoshuaKGoldberg/create-typescript-app/issues/1184) tracks turning the test snapshot into a feature.
+
+##### Unexpected File Modifications
+
+The migration test also asserts that no files were unexpectedly changed.
+If you see a failure like:
+
+```plaintext
+Oh no! Running the migrate script unexpectedly modified:
+ - ...
+```
+
+...then that means the file generated from templates differs from what's checked into the repository.
+This is most often caused by changes to templates not being applied to checked-in files too.
+
+Templates for files are generally stored in [`src/steps/writing/creation`] under a path roughly corresponding to the file they describe.
+For example, the template for `tsup.config.ts` is stored in [`src/steps/writing/creation/createTsupConfig.ts`](../src/steps/writing/creation/createTsupConfig.ts).
+If the `createTsupConfig` function were to be modified without an equivalent change to `tsup.config.ts` -or vice-versa- then the migration test would report:
+
+```plaintext
+Oh no! Running the migrate script unexpectedly modified:
+ - tsup.config.ts
+```

Diff for: docs/Creation.md

@@ -6,7 +6,15 @@ You can run `npx create-typescript-app` in your terminal to interactively create
 npx create-typescript-app
 ```
 
-Then, go through the following two steps to set up required repository tooling on GitHub:
+The creation script will by default:
+
+1. Create a new directory with the given repository name
+2. Initialize that new directory as a local Git repository
+3. Copy the template's files to that directory
+4. Create a new repository on GitHub and set it as the local repository's upstream
+5. Configure relevant settings on the GitHub repository
+
+You'll then need to manually go through the following two steps to set up tooling on GitHub:
 
 1. Create two tokens in [repository secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets) _(unless you chose to opt out of releases)_:
    - `ACCESS_TOKEN`: A [GitHub PAT](https://github.com/settings/tokens/new) with _repo_ and _workflow_ permissions
@@ -23,7 +31,7 @@ Hooray! 🥳
 You can explicitly provide some or all of the options the script would prompt for as command-line flags.
 See [Options.md](./Options.md).
 
-For example, running the creation script and skipping all GitHub APIs:
+For example, running the creation script and skipping all GitHub-related APIs:
 
 ```shell
 npx create-typescript-app --mode create --skip-all-contributors-api --skip-github-api

Diff for: docs/Initialization.md

@@ -18,7 +18,7 @@ It will then remove itself and uninstall dependencies only used for initializati
 pnpm run initialize
 ```
 
-Then, go through the following two steps to set up required repository tooling on GitHub:
+You'll then need to manually go through the following two steps to set up tooling on GitHub:
 
 1. Create two tokens in [repository secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets) _(unless you chose to opt out of releases)_:
    - `ACCESS_TOKEN`: A [GitHub PAT](https://github.com/settings/tokens/new) with _repo_ and _workflow_ permissions
@@ -37,7 +37,7 @@ See [Options.md](./Options.md).
 
 `pnpm run initialize` will set `--mode` to `initialize`.
 
-For example, running the initialization script and skipping all GitHub APIs:
+For example, running the initialization script and skipping all GitHub-related APIs:
 
 ```shell
 pnpm run initialize --skip-all-contributors-api --skip-github-api

Diff for: docs/Migration.md

@@ -6,20 +6,54 @@ If you have an existing repository that you'd like to give the files from this r
 npx create-typescript-app
 ```
 
+The migration script will:
+
+- Uninstall any known old packages that conflict with this template's tooling
+- Delete configuration files used with those old packages
+- Install any packages needed for this template's tooling
+- Create or rewrite configuration files for the new tooling
+- Run ESLint and Prettier auto-fixers to align formatting and style to the new settings
+
+For example, if the repository previously using Jest for testing:
+
+- `eslint-plugin-jest`, `jest`, and other Jest-related packages will be uninstalled
+- Any Jest config file like `jest.config.js` will be deleted
+- `eslint-plugin-vitest`, `vitest`, and other Vitest-related packages will be installed
+- A `vitest.config.ts` file will be created
+
+You'll then need to manually go through the following two steps to set up tooling on GitHub:
+
+1. Create two tokens in [repository secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets) _(unless you chose to opt out of releases)_:
+   - `ACCESS_TOKEN`: A [GitHub PAT](https://github.com/settings/tokens/new) with _repo_ and _workflow_ permissions
+   - `NPM_TOKEN`: An [npm access token](https://docs.npmjs.com/creating-and-viewing-access-tokens/) with _Automation_ permissions
+2. Install two GitHub apps:
+   - [Codecov](https://github.com/marketplace/codecov) _(unless you chose to opt out of tests)_
+   - [Renovate](https://github.com/marketplace/renovate) _(unless you chose to opt out of renovate)_
+
+Your repository will then have an approximate copy of this template's tooling ready for you to review!
+Hooray! 🥳
+
 > [!WARNING]
 > Migration will override many files in your repository.
 > You'll want to review each of the changes.
 > There will almost certainly be some incorrect changes you'll need to fix.
 
 ## Options
 
+`create-typescript-app` will detect whether it's being run in an existing repository.
+It also allows specifying `--mode migrate` if that detection misinterprets the current directory:
+
+```shell
+npx create-typescript-app --mode migrate
+```
+
 You can explicitly provide some or all of the options the script would prompt for as command-line flags.
 See [Options.md](./Options.md).
 
-For example, running the migration script and skipping all GitHub APIs:
+For example, running the migration script and skipping all GitHub-related APIs:
 
 ```shell
-npx create-typescript-app --mode migrate --skip-all-contributors-api --skip-github-api
+npx create-typescript-app --skip-all-contributors-api --skip-github-api
 ```
 
 See [Tooling.md](./Tooling.md) for details on the tooling pieces and which bases they're included in.

Diff for: docs/Options.md

@@ -13,8 +13,8 @@ The following required options will be prompted for interactively if not provide
 These required options determine how the creation script will set up and scaffold the repository:
 
 - `--base`: Whether to scaffold the repository with:
-  - `minimum`: Just the bare starter tooling most repositories should ideally include.
-  - `common`: Important additions to the minimum starters such as releases and tests.
+  - `minimum`: Just the bare starter tooling most repositories should ideally include
+  - `common`: Important additions to the minimum starters such as releases and tests
   - `everything`: The most thorough tooling imaginable: sorting, spellchecking, and more!
   - `prompt`: Fine-grained control over which tooling pieces to use
 - `--mode`: Whether to:

Diff for: script/create-test-e2e.js

@@ -10,6 +10,8 @@ const title = "Test Title";
 
 await $`rm -rf ${repository}`;
 
+// Fist we run with --mode create to create a new new local repository,
+// asserting that pnpm i passes in that repository's directory.
 await $({
 	stdio: "inherit",
 })`c8 -o ./coverage-create -r html -r lcov  --src src node ./bin/index.js --base everything --mode create --author ${author} --email ${email} --description ${description} --owner ${owner} --title ${title} --repository ${repository} --skip-all-contributors-api --skip-github-api`;
@@ -18,6 +20,7 @@ process.chdir(repository);
 
 const failures = [];
 
+// Then we run each of the CI commands to assert that they pass too.
 for (const command of [
 	`pnpm i`,
 	`pnpm run build`,

Diff for: script/initialize-test-e2e.js

@@ -8,10 +8,11 @@ const owner = "RNR1";
 const title = "New Title Test";
 const repository = "new-repository-test";
 
-// First we run initialize to modifies the local repo, so we can test the changes
+// Fist we run with --mode initialize to modify the local repository files,
+// asserting that the created package.json keeps the general description.
 await $({
 	stdio: "inherit",
-})`node ./bin/index.js --description ${description} --base everything --mode initialize --owner ${owner} --title ${title} --repository ${repository} --skip-all-contributors-api --skip-github-api --skip-restore`;
+})`pnpm run initialize --base everything --mode initialize --description ${description} --owner ${owner} --title ${title} --repository ${repository} --skip-all-contributors-api --skip-github-api --skip-restore`;
 
 const newPackageJson = JSON.parse(
 	(await fs.readFile("./package.json")).toString(),
@@ -21,6 +22,8 @@ console.log("New package JSON:", newPackageJson);
 assert.equal(newPackageJson.description, description);
 assert.equal(newPackageJson.name, repository);
 
+// Assert that the initialize script used the provided values in files,
+// except for the 'This package was templated with ...' attribution notice.
 const files = await globby(["*.*", "**/*.*"], {
 	gitignore: true,
 	ignoreFiles: ["script/initialize-test-e2e.js"],
@@ -34,11 +37,12 @@ for (const search of [`/JoshuaKGoldberg/`, "create-typescript-app"]) {
 	);
 }
 
+// Use Knip to assert that none of the template-only dependencies remain.
+// They should have been removed as part of initialization.
 try {
 	await $`pnpm run lint:knip`;
 } catch (error) {
-	console.error("Error running lint:knip:", error);
-	process.exitCode = 1;
+	throw new Error("Error running lint:knip:", { cause: error });
 }
 
 // Now that initialize has passed normal steps, we reset everything,
@@ -49,4 +53,4 @@ await $`pnpm i`;
 await $`pnpm run build`;
 await $({
 	stdio: "inherit",
-})`c8 -o ./coverage -r html -r lcov --src src node ./bin/index.js --base everything --description ${description} --mode initialize --owner ${owner} --title ${title} --repository ${repository} --skip-all-contributors-api --skip-github-api --skip-removal --skip-restore`;
+})`c8 -o ./coverage -r html -r lcov --src src node ./bin/index.js --base everything --mode initialize --description ${description} --owner ${owner} --title ${title} --repository ${repository} --skip-all-contributors-api --skip-github-api --skip-removal --skip-restore`;

Diff for: script/migrate-test-e2e.js

@@ -120,6 +120,8 @@ describe("expected file changes", () => {
 			`Looks like there were no changes to ${file} from migration?`,
 		);
 
+		// If this fails, see .github/DEVELOPMENT.md > Setup Scripts for context.
+		// Then see .github/DEVELOPMENT.md > Migration Snapshot Failures.
 		expect(contentsAfterGitMarkers).toMatchSnapshot();
 	});
 });
@@ -149,22 +151,18 @@ test("unexpected file changes", async () => {
 		const gitDiffCommand = `git diff HEAD -- ${unstagedModifiedFiles.join(
 			" ",
 		)}`;
-		console.log(
-			`Stdout from running \`${gitDiffCommand}\`:\n${
-				(await execaCommand(gitDiffCommand)).stdout
-			}`,
-		);
+		const { stdout } = await execaCommand(gitDiffCommand);
+
+		console.log(`Stdout from running \`${gitDiffCommand}\`:\n${stdout}`);
+
 		throw new Error(
 			[
 				"",
-				"Oh no! Running the migrate script modified some files:",
+				"Oh no! Running the migrate script unexpectedly modified:",
 				...unstagedModifiedFiles.map((filePath) => ` - ${filePath}`),
 				"",
-				"That likely indicates changes made to the repository without",
-				"corresponding updates to templates in src/.",
-				"",
-				"Please search for those file(s)' name(s) under src/migrate for",
-				"the corresponding template and update those as well.",
+				"See .github/DEVELOPMENT.md > Setup Scripts for context.",
+				"Then see .github/DEVELOPMENT.md > Unexpected File Modifications.",
 			]
 				.map((line) => chalk.red(line))
 				.join("\n"),

Diff for: src/steps/writing/creation/createDotESLintignore.test.ts

@@ -0,0 +1,30 @@
+import { describe, expect, it } from "vitest";
+
+import { createDotESLintignore } from "./createDotESLintignore.js";
+
+describe("createDotESLintignore", () => {
+	it("creates an ignore file with coverage when excludeTests is false", () => {
+		const actual = createDotESLintignore({ excludeTests: false });
+
+		expect(actual).toMatchInlineSnapshot(`
+			"!.*
+			coverage
+			lib
+			node_modules
+			pnpm-lock.yaml
+			"
+		`);
+	});
+
+	it("creates an ignore file without coverage when excludeTests is true", () => {
+		const actual = createDotESLintignore({ excludeTests: true });
+
+		expect(actual).toMatchInlineSnapshot(`
+			"!.*
+			lib
+			node_modules
+			pnpm-lock.yaml
+			"
+		`);
+	});
+});

Diff for: src/steps/writing/creation/createDotESLintignore.ts

@@ -0,0 +1,14 @@
+import { Options } from "../../../shared/types.js";
+import { formatIgnoreFile } from "./formatters/formatIgnoreFile.js";
+
+export function createDotESLintignore(options: Pick<Options, "excludeTests">) {
+	return formatIgnoreFile(
+		[
+			"!.*",
+			...(options.excludeTests ? [] : ["coverage"]),
+			"lib",
+			"node_modules",
+			"pnpm-lock.yaml",
+		].filter(Boolean),
+	);
+}

Diff for: src/steps/writing/creation/createDotGitignore.test.ts

@@ -0,0 +1,26 @@
+import { describe, expect, it } from "vitest";
+
+import { createDotGitignore } from "./createDotGitignore.js";
+
+describe("createDotGitignore", () => {
+	it("creates an ignore file with coverage when excludeTests is false", () => {
+		const actual = createDotGitignore({ excludeTests: false });
+
+		expect(actual).toMatchInlineSnapshot(`
+			"coverage/
+			lib/
+			node_modules/
+			"
+		`);
+	});
+
+	it("creates an ignore file without coverage when excludeTests is true", () => {
+		const actual = createDotGitignore({ excludeTests: true });
+
+		expect(actual).toMatchInlineSnapshot(`
+			"lib/
+			node_modules/
+			"
+		`);
+	});
+});

Diff for: src/steps/writing/creation/createDotGitignore.ts

@@ -0,0 +1,10 @@
+import { Options } from "../../../shared/types.js";
+import { formatIgnoreFile } from "./formatters/formatIgnoreFile.js";
+
+export function createDotGitignore(options: Pick<Options, "excludeTests">) {
+	return formatIgnoreFile([
+		...(options.excludeTests ? [] : ["coverage/"]),
+		"lib/",
+		"node_modules/",
+	]);
+}