Test strategy proposal (#960)

* Propose levels of testing and the test matrix
* Propose a single mocking library to introduce unit testing
* Add test strategy documentation

Author: award999

Diff for: .vscode-test.js

@@ -33,6 +33,8 @@ module.exports = defineConfig({
             mocha: {
                 ui: "tdd",
                 color: true,
+                // so doesn't timeout when breakpoint is hit
+                timeout: isDebugRun ? Number.MAX_SAFE_INTEGER : 3000,
                 timeout,
                 forbidOnly: isCIBuild,
                 grep: isFastTestRun ? "@slow" : undefined,
@@ -41,6 +43,19 @@ module.exports = defineConfig({
             installExtensions: ["vadimcn.vscode-lldb"],
             reuseMachineInstall: !isCIBuild,
         },
+        {
+            label: "unitTests",
+            files: ["out/test/unit-tests/**/*.test.js"],
+            version: "stable",
+            mocha: {
+                ui: "tdd",
+                color: true,
+                // so doesn't timeout when breakpoint is hit
+                timeout: isDebugRun ? Number.MAX_SAFE_INTEGER : 3000,
+                forbidOnly: isCIBuild,
+            },
+            reuseMachineInstall: !isCIBuild,
+        },
         // you can specify additional test configurations, too
     ],
     coverage: {

Diff for: .vscode/launch.json

@@ -37,6 +37,17 @@
 				"VSCODE_DEBUG": "1"
 			},
 			"preLaunchTask": "compile-tests"
+		},
+		{
+			"name": "Unit Tests",
+			"type": "extensionHost",
+			"request": "launch",
+			"testConfiguration": "${workspaceFolder}/.vscode-test.js",
+			"testConfigurationLabel": "unitTests",
+			"outFiles": [
+				"${workspaceFolder}/out/**/*.js"
+			],
+			"preLaunchTask": "compile-tests"
 		}
 	]
 }

Diff for: CONTRIBUTING.md

@@ -76,6 +76,8 @@ Please keep your PRs to a minimal number of changes. If a PR is large, try to sp
 
 Where possible any new feature should have tests that go along with it, to ensure it works and will continue to work in the future. When a PR is submitted one of the prerequisites for it to be merged is that all tests pass.
 
+For information on levels of testing done in this extension, see the [test strategy](docs/contributor/test-strategy.md).
+
 To get started running tests first import the `testing-debug.code-profile` VS Code profile used by the tests. Run the `> Profiles: Import Profile...` command then `Select File` and pick `./.vscode/testing-debug.code-profile`.
 
 Now you can run tests locally using either of the following methods:

Diff for: docs/contributor/test-strategy.md

@@ -0,0 +1,143 @@
+# Swift for Visual Studio Code test strategy
+
+## Overview
+
+The recommended way for [testing extensions](https://code.visualstudio.com/api/working-with-extensions/testing-extension) involves using either the new [vscode-test-cli](https://github.com/microsoft/vscode-test-cli) or creating your [own mocha test runner](https://code.visualstudio.com/api/working-with-extensions/testing-extension#advanced-setup-your-own-runner). Either approach results in Visual Studio Code getting downloaded, and a window spawned. This is necessary to have access the the APIs of the `vscode` namespace, to stimulate behaviour (ex. `vscode.tasks.executeTasks`) and obtain state (ex. `vscode.languages.getDiagnostics`).
+
+There are some testing gaps when only using this approach. Relying on using the `vscode` APIs makes it difficult to easily write unit tests. It ends up testing the communication between a lot of components in the `vscode-swift` extension and associated dependencies. Additionally, there is a lot of code that remains unverified. This code may get executed so that it shows up in the coverage report, but the behaviour is unobserved. Some examples of behaviour that is not observed, includes prompting the user for input, or verifying if a notification gets shown. See https://devblogs.microsoft.com/ise/testing-vscode-extensions-with-typescript/ for a more detailed overview.
+
+In addition to gaps in testing, the current approach tests at the integration level which results in slower and more brittle tests, as they rely on the communication between several components.
+
+## Unit testing
+
+For unit testing [ts-mockito](https://github.com/NagRock/ts-mockito) is used for mocking out user interaction and observing behaviour that cannot otherwise be observed. Many helpful utility functions exist in [MockUtils.ts](../../test/unit-tests/MockUtils.ts). These utility methods take care of the setup and teardown of the mocks so the developer does not need to remember to do this for each suite/test.
+
+### Mocking `vscode` namespace
+
+The `mockNamespace` function provides a mocked implementation of one of the `vscode` API namespaces. Below is an example of how to employ mocking to test if the [showReloadExtensionNotification](../../src/ui/ReloadExtension.ts) function shows a notification and mock the button click.
+
+```ts
+suite("ReloadExtension Unit Test Suite", async function () {
+    const windowMock = mockNamespace(vscode, "window");
+    const commandsMock = mockNamespace(vscode, "commands");
+
+    test('"Reload Extensions" is clicked', async () => {
+        // What happens if they click this button?
+        when(windowMock.showWarningMessage(anyString(), "Reload Extensions")).thenReturn(
+            Promise.resolve("Reload Extensions")
+        );
+        await showReloadExtensionNotification("Want to reload?");
+        verify(commandsMock.executeCommand("workbench.action.reloadWindow")).called();
+    });
+});
+```
+
+### Mocking event emitter
+
+The `eventListenerMock` function captures components listening for a given event and fires the event emitter with the provided test data. Below is an example of mocking the `onDidStartTask` event.
+
+```ts
+suite("Event emitter example", async function () {
+    const listenerMock = eventListenerMock(vscode.tasks, "onDidStartTask");
+
+    test("Fire event", async () => {
+        const mockedTask = mock(vscode.Task);
+        mockedTaskExecution = { task: instance(mockedTask), terminate: () => {} };
+
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+    });
+});
+```
+
+### Mocking global variables
+
+The `globalVariableMock` function allows for overriding the value for some global constant.
+
+```ts
+suite("Environment variable example", async function () {
+    const envMock = globalVariableMock(process, "env");
+
+    test("Linux", async () => {
+        env.setValue({ DEVELOPER_DIR: '/path/to/Xcode.app' });
+
+        // Test DEVELOPER_DIR usage
+    });
+});
+```
+
+It can also be used to mock the extension [configuration](../../src/configuration.ts).
+
+```ts
+import configuration from "../../../src/configuration";
+suite("SwiftBuildStatus Unit Test Suite", async function () {
+    const statusConfig = globalVariableMock(configuration, "showBuildStatus");
+
+    test("Shows notification", async () => {
+        statusConfig.setValue("notification");
+
+        // Test shows as notification
+    });
+
+    test("Shows status bar", async () => {
+        statusConfig.setValue("swiftStatus");
+
+        // Test shows in status bar
+    });
+});
+```
+
+## Test Pyramid
+
+Tests are grouped into 3 levels. The biggest distinguishing factors between the various levels will be the runtime of the test, and the number of "real" vs. mocked dependencies.
+
+### 1. Unit (`/test/unit`)
+
+- Employ stubbing or mocking techniques to allow for user interaction, AND to mock slow APIs like `executeTask`
+- Mocked SwiftPM commands return hardcoded output, such as compile errors
+- Any sourcekit-lsp interaction is mocked, with hardcoded responses
+- Runs with a fast timeout of 100ms
+- No usages of assets/test projects
+  - Use [mock-fs](https://www.npmjs.com/package/mock-fs) for testing fs usage
+- Run in CI build for new PRs
+- Ideally the vast majority of tests are at this level
+
+### 2. Integration (`/test/integration`)
+
+- Tests interaction between components, with some mocking for slow or fragile dependencies
+- Stimulate actions using the VS Code APIs
+- Use actual output from SwiftPM
+- Use actual responses from sourcekit-lsp
+- Use a moderate maximum timeout of up to 30s
+  - The CI job timeout is 15 minutes
+- Use curated `assets/test` projects
+- Run in CI and nightly builds
+- Test key integrations with the VS Code API and key communication between our components
+
+### 3. Smoke (`/test/smoke`)
+
+- No mocking at all
+- For now only stimulate actions using the VS Code APIs, testing via the UI is a different beast
+- Use curated `assets/test` projects
+- No need to enforce a maximum timeout (per test)
+- Only run in nightly build
+- Should only have a handful of these tests, for complex features
+
+## Test Matrix
+
+### CI Build
+
+- Run for new PRs (`@swift-server-bot test this please`)
+- Run macOS, Linux and Windows
+  - Currently only Linux, macOS and Windows is being explored
+  - Expect Windows to fail short term, annotate to disable these tests
+- Ideally run against Swift versions 5.6 - 6.0 + main
+- Run `unit` and `integration` test suites
+- Run test against latest `stable` VS Code
+
+### Nightly Build
+
+- Run macOS, Linux and Windows
+  - Currently only Linux, macOS and Windows is being explored
+- Ideally run against Swift versions 5.6 - 6.0 + main
+- Run `integration` and `smoke` test suites
+- Run test against latest `stable` and `insiders` VS Code

Diff for: package-lock.json

@@ -1215,6 +1215,7 @@
     "format": "prettier --check src test",
     "pretest": "npm run compile && find ./assets/test -type d -name '.build' -exec rm -rf {} + && find . -type d -name 'Package.resolved' -exec rm -rf {} + && tsc -p ./",
     "test": "vscode-test",
+    "unit-test": "vscode-test --label unitTests",
     "coverage": "npm run pretest && vscode-test --coverage",
     "compile-tests": "find ./assets/test -type d -name '.build' -exec rm -rf {} + && npm run compile && npm run esbuild",
     "package": "vsce package",
@@ -1243,6 +1244,7 @@
     "node-pty": "^1.0.0",
     "prettier": "3.3.2",
     "strip-ansi": "^6.0.1",
+    "ts-mockito": "^2.6.1",
     "typescript": "^5.5.3"
   },
   "dependencies": {

Diff for: package.json

@@ -0,0 +1,189 @@
+//===----------------------------------------------------------------------===//
+//
+// This source file is part of the VS Code Swift open source project
+//
+// Copyright (c) 2024 the VS Code Swift project authors
+// Licensed under Apache License v2.0
+//
+// See LICENSE.txt for license information
+// See CONTRIBUTORS.txt for the list of VS Code Swift project authors
+//
+// SPDX-License-Identifier: Apache-2.0
+//
+//===----------------------------------------------------------------------===//
+/* eslint-disable @typescript-eslint/no-explicit-any */
+import { setup, teardown } from "mocha";
+import { Disposable, Event, EventEmitter } from "vscode";
+import { instance, mock } from "ts-mockito";
+
+export function getMochaHooks(): { setup: typeof setup; teardown: typeof teardown } {
+    if (!("setup" in global && "teardown" in global)) {
+        throw new Error("MockUtils can only be used when running under mocha");
+    }
+    return {
+        setup: global.setup,
+        teardown: global.teardown,
+    };
+}
+
+/**
+ * Create a new mock for each test that gets cleaned up automatically afterwards. This function makes use of the fact that
+ * Mocha's setup() and teardown() methods can be called from anywhere. The resulting object is a proxy to the real
+ * mock since it won't be created until the test actually begins.
+ *
+ * The proxy lets us avoid boilerplate by creating a mock in one line:
+ *
+ *     const windowMock = mockNamespace(vscode, "window");
+ *
+ *     test('Some test', () => {
+ *       expect(windowMock.showErrorMessage).to.have.been.calledWith('Some error message');
+ *     })
+ *
+ * @param obj The object to create the stub inside
+ * @param property The method inside the object to be stubbed
+ */
+export function mockNamespace<T, K extends keyof T>(obj: T, property: K): T[K] {
+    let realMock: T[K];
+    const originalValue: T[K] = obj[property];
+    const mocha = getMochaHooks();
+    // Create the mock at setup
+    mocha.setup(() => {
+        realMock = mock<T[K]>();
+        Object.defineProperty(obj, property, { value: instance(realMock) });
+    });
+    // Restore original value at teardown
+    mocha.teardown(() => {
+        Object.defineProperty(obj, property, { value: originalValue });
+    });
+    // Return the proxy to the real mock
+    return new Proxy(originalValue, {
+        get: (target: any, property: string): any => {
+            if (!realMock) {
+                throw Error("Mock proxy accessed before setup()");
+            }
+            return (realMock as any)[property];
+        },
+        // eslint-disable-next-line @typescript-eslint/no-unused-vars
+        set: (target: any, property: string, value: any): boolean => {
+            // Ignore
+            return true;
+        },
+    });
+}
+
+/** Retrieves the type of event given to the generic vscode.Event<T> */
+export type EventType<T> = T extends Event<infer E> ? E : never;
+
+/** The listener function that receives events. */
+export type Listener<T> = T extends Event<infer E> ? (e: E) => any : never;
+
+/** Allows sending an event to all intercepted listeners. */
+export interface ListenerInterceptor<T> {
+    onDidAddListener: Event<(event: T) => any>;
+
+    /** Send an event notification to the intercepted listener(s). */
+    notifyAll(event: T): Promise<void>;
+}
+
+class ListenerInterceptorImpl<T> implements ListenerInterceptor<T> {
+    private listeners: ((event: T) => any)[];
+    private _onDidChangeListeners: EventEmitter<(event: T) => any>;
+
+    constructor() {
+        this.listeners = [];
+        this._onDidChangeListeners = new EventEmitter();
+        this.onDidAddListener = this._onDidChangeListeners.event;
+    }
+
+    async notifyAll(event: T): Promise<void> {
+        await Promise.all(this.listeners.map(async listener => await listener(event)));
+    }
+
+    addListener: Event<T> = (listener, thisArgs) => {
+        if (thisArgs) {
+            listener = listener.bind(thisArgs);
+        }
+        this.listeners.push(listener);
+        this._onDidChangeListeners.fire(listener);
+        return { dispose: () => mock(Disposable) };
+    };
+
+    onDidAddListener: Event<(event: T) => any>;
+}
+
+/** Retrieves all properties of an object that are of the type vscode.Event<T>. */
+type EventsOf<T> = {
+    [K in keyof T]: T[K] extends Event<any> ? K : never;
+}[keyof T];
+
+/**
+ * Create a ListenerInterceptor that intercepts all listeners attached to a VS Code event function. This function stubs out the
+ * given method during Mocha setup and restores it during teardown.
+ *
+ * This lets us avoid boilerplate by creating an interceptor in one line:
+ *
+ *     const interceptor = eventListenerMock(vscode.workspace, 'onDidChangeWorkspaceFolders');
+ *
+ *     test('Some test', () => {
+ *       interceptor.notify(event);
+ *     })
+ *
+ * @param obj The object to create the stub inside
+ * @param method The name of the method to stub within the object. Must be of the type Event<any>.
+ */
+export function eventListenerMock<T, K extends EventsOf<T>>(
+    obj: T,
+    method: K
+): ListenerInterceptor<EventType<T[K]>> {
+    const interceptor = new ListenerInterceptorImpl<EventType<T[K]>>();
+    const originalValue: T[K] = obj[method];
+    const mocha = getMochaHooks();
+    mocha.setup(() => {
+        Object.defineProperty(obj, method, { value: interceptor.addListener });
+    });
+    // Restore original value at teardown
+    mocha.teardown(() => {
+        Object.defineProperty(obj, method, { value: originalValue });
+    });
+    return interceptor;
+}
+
+/**
+ * Allows setting the global value.
+ */
+export interface GlobalVariableMock<T> {
+    setValue(value: T): void;
+}
+
+/**
+ * Create a new GlobalVariableMock that is restored after a test completes.
+ */
+export function globalVariableMock<T, K extends keyof T>(
+    obj: T,
+    property: K
+): GlobalVariableMock<T[K]> {
+    let setupComplete: boolean = false;
+    let originalValue: T[K];
+    const mocha = getMochaHooks();
+    // Grab the original value during setup
+    mocha.setup(() => {
+        originalValue = obj[property];
+        setupComplete = true;
+    });
+    // Restore the original value on teardown
+    mocha.teardown(() => {
+        Object.defineProperty(obj, property, { value: originalValue });
+        setupComplete = false;
+    });
+    // Return a GlobalVariableMock that allows for easy mocking of the value
+    return {
+        setValue(value: T[K]): void {
+            if (!setupComplete) {
+                throw new Error(
+                    `'${String(property)}' cannot be set before globalVariableMock() completes its setup through Mocha`
+                );
+            }
+            Object.defineProperty(obj, property, { value: value });
+        },
+    };
+}

Diff for: test/unit-tests/MockUtils.ts

@@ -0,0 +1,48 @@
+//===----------------------------------------------------------------------===//
+//
+// This source file is part of the VS Code Swift open source project
+//
+// Copyright (c) 2024 the VS Code Swift project authors
+// Licensed under Apache License v2.0
+//
+// See LICENSE.txt for license information
+// See CONTRIBUTORS.txt for the list of VS Code Swift project authors
+//
+// SPDX-License-Identifier: Apache-2.0
+//
+//===----------------------------------------------------------------------===//
+import { anyString, anything, verify, when } from "ts-mockito";
+import { equal } from "assert";
+import { showReloadExtensionNotification } from "../../../src/ui/ReloadExtension";
+import * as vscode from "vscode";
+import { mockNamespace } from "../MockUtils";
+
+suite("ReloadExtension Unit Test Suite", async function () {
+    const windowMock = mockNamespace(vscode, "window");
+    const commandsMock = mockNamespace(vscode, "commands");
+
+    test("Shows user a warning", async () => {
+        // No behaviour setup, let's just check if we showed them the notification
+        await showReloadExtensionNotification("Want to reload?");
+        verify(windowMock.showWarningMessage("Want to reload?", "Reload Extensions")).called();
+    });
+
+    test('"Reload Extensions" is clicked', async () => {
+        // What happens if they click this button?
+        when(windowMock.showWarningMessage(anyString(), "Reload Extensions")).thenReturn(
+            Promise.resolve("Reload Extensions")
+        );
+        await showReloadExtensionNotification("Want to reload?");
+        verify(commandsMock.executeCommand("workbench.action.reloadWindow")).called();
+    });
+
+    test("Provide a different button", async () => {
+        // What if we provide another option?
+        when(
+            windowMock.showWarningMessage("Want to reload?", "Reload Extensions", "Ignore")
+        ).thenReturn(Promise.resolve("Ignore"));
+        const result = await showReloadExtensionNotification("Want to reload?", "Ignore");
+        equal(result, "Ignore");
+        verify(commandsMock.executeCommand(anything())).never();
+    });
+});

Diff for: test/unit-tests/ui/ReloadExtension.test.ts

@@ -0,0 +1,242 @@
+//===----------------------------------------------------------------------===//
+//
+// This source file is part of the VS Code Swift open source project
+//
+// Copyright (c) 2024 the VS Code Swift project authors
+// Licensed under Apache License v2.0
+//
+// See LICENSE.txt for license information
+// See CONTRIBUTORS.txt for the list of VS Code Swift project authors
+//
+// SPDX-License-Identifier: Apache-2.0
+//
+//===----------------------------------------------------------------------===//
+import {
+    anyFunction,
+    anything,
+    capture,
+    deepEqual,
+    instance,
+    mock,
+    verify,
+    when,
+} from "ts-mockito";
+import configuration from "../../../src/configuration";
+import * as vscode from "vscode";
+import { eventListenerMock, globalVariableMock, mockNamespace } from "../MockUtils";
+import { SwiftExecution } from "../../../src/tasks/SwiftExecution";
+import { TestSwiftProcess } from "../../fixtures";
+import { StatusItem } from "../../../src/ui/StatusItem";
+import { SwiftBuildStatus } from "../../../src/ui/SwiftBuildStatus";
+
+suite("SwiftBuildStatus Unit Test Suite", async function () {
+    const windowMock = mockNamespace(vscode, "window");
+    const listenerMock = eventListenerMock(vscode.tasks, "onDidStartTask");
+    const configurationMock = globalVariableMock(configuration, "showBuildStatus");
+
+    let mockedStatusItem: StatusItem;
+    let mockedTask: vscode.Task;
+    let mockedExecution: SwiftExecution;
+    let mockedProcess: TestSwiftProcess;
+    let mockedTaskExecution: vscode.TaskExecution;
+
+    setup(() => {
+        mockedStatusItem = mock(StatusItem);
+        mockedTask = mock(vscode.Task);
+        mockedProcess = new TestSwiftProcess("swift", ["build"]);
+        mockedExecution = new SwiftExecution(
+            mockedProcess.command,
+            mockedProcess.args,
+            {},
+            mockedProcess
+        );
+        when(mockedTask.definition).thenReturn({ type: "swift" });
+        when(mockedTask.execution).thenReturn(mockedExecution);
+        when(mockedTask.name).thenReturn("My Task");
+
+        // https://github.com/NagRock/ts-mockito/issues/204
+        const task = instance(mockedTask);
+        Object.setPrototypeOf(task, vscode.Task.prototype);
+        mockedTaskExecution = { task, terminate: () => {} };
+    });
+
+    test("Never show status", async () => {
+        configurationMock.setValue("never");
+
+        new SwiftBuildStatus(instance(mockedStatusItem));
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+
+        verify(
+            mockedStatusItem.showStatusWhileRunning(mockedTaskExecution.task, anyFunction())
+        ).never();
+        verify(windowMock.withProgress(anything(), anyFunction())).never();
+    });
+
+    test("Ignore non-swift task", async () => {
+        when(mockedTask.execution).thenReturn(new vscode.ShellExecution("swift"));
+        configurationMock.setValue("swiftStatus");
+
+        new SwiftBuildStatus(instance(mockedStatusItem));
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+
+        verify(
+            mockedStatusItem.showStatusWhileRunning(mockedTaskExecution.task, anyFunction())
+        ).never();
+        verify(windowMock.withProgress(anything(), anyFunction())).never();
+    });
+
+    test("Show swift status", async () => {
+        configurationMock.setValue("swiftStatus");
+
+        new SwiftBuildStatus(instance(mockedStatusItem));
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+
+        verify(
+            mockedStatusItem.showStatusWhileRunning(mockedTaskExecution.task, anyFunction())
+        ).called();
+        verify(windowMock.withProgress(anything(), anyFunction())).never();
+    });
+
+    test("Show status bar progress", async () => {
+        configurationMock.setValue("progress");
+
+        new SwiftBuildStatus(instance(mockedStatusItem));
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+
+        verify(
+            windowMock.withProgress(
+                deepEqual({ location: vscode.ProgressLocation.Window }),
+                anyFunction()
+            )
+        ).called();
+        verify(mockedStatusItem.showStatusWhileRunning(anything(), anyFunction())).never();
+    });
+
+    test("Show notification progress", async () => {
+        configurationMock.setValue("notification");
+
+        new SwiftBuildStatus(instance(mockedStatusItem));
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+
+        verify(
+            windowMock.withProgress(
+                deepEqual({ location: vscode.ProgressLocation.Notification }),
+                anyFunction()
+            )
+        ).called();
+        verify(mockedStatusItem.showStatusWhileRunning(anything(), anyFunction())).never();
+    });
+
+    test("Update fetching", async () => {
+        // Setup progress
+        configurationMock.setValue("progress");
+        new SwiftBuildStatus(instance(mockedStatusItem));
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+
+        // Setup swiftStatus
+        configurationMock.setValue("swiftStatus");
+        new SwiftBuildStatus(instance(mockedStatusItem));
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+
+        const mockReporter = mock<vscode.Progress<{ message: string }>>();
+        const [, statusCallback] = capture<vscode.Task, () => void>(
+            mockedStatusItem.showStatusWhileRunning
+        ).last();
+        const [, progressCallback] = capture<
+            vscode.ProgressOptions,
+            (p: vscode.Progress<unknown>) => Promise<void>
+        >(windowMock.withProgress).last();
+
+        // Execute callback reporters
+        statusCallback();
+        progressCallback(instance(mockReporter));
+
+        mockedProcess.write(
+            "Fetching https://github.com/apple/example-package-figlet from cache\n" +
+                "Fetched https://github.com/apple/example-package-figlet from cache (0.43s)\n" +
+                "Fetching https://github.com/apple/swift-testing.git from cache\n" +
+                "Fetched https://github.com/apple/swift-testing.git from cache (0.77s)\n"
+        );
+
+        const expected = "My Task fetching dependencies";
+        verify(mockReporter.report(deepEqual({ message: expected }))).called();
+        verify(mockedStatusItem.update(mockedTaskExecution.task, expected)).called();
+    });
+
+    test("Update build progress", async () => {
+        // Setup progress
+        configurationMock.setValue("progress");
+        new SwiftBuildStatus(instance(mockedStatusItem));
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+
+        // Setup swiftStatus
+        configurationMock.setValue("swiftStatus");
+        new SwiftBuildStatus(instance(mockedStatusItem));
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+
+        const mockReporter = mock<vscode.Progress<{ message: string }>>();
+        const [, statusCallback] = capture<vscode.Task, () => void>(
+            mockedStatusItem.showStatusWhileRunning
+        ).last();
+        const [, progressCallback] = capture<
+            vscode.ProgressOptions,
+            (p: vscode.Progress<unknown>) => Promise<void>
+        >(windowMock.withProgress).last();
+
+        // Execute callback reporters
+        statusCallback();
+        progressCallback(instance(mockReporter));
+
+        mockedProcess.write(
+            "Fetching https://github.com/apple/example-package-figlet from cache\n" +
+                "[6/7] Building main.swift\n" +
+                "[7/7] Applying MyCLI\n"
+        );
+
+        const expected = "My Task [7/7]";
+        verify(mockReporter.report(deepEqual({ message: expected }))).called();
+        verify(mockedStatusItem.update(mockedTaskExecution.task, expected)).called();
+
+        // Ignore old stuff
+        verify(
+            mockReporter.report(deepEqual({ message: "My Task fetching dependencies" }))
+        ).never();
+        verify(mockReporter.report(deepEqual({ message: "My Task [6/7]" }))).never();
+    });
+
+    test("Build complete", async () => {
+        // Setup progress
+        configurationMock.setValue("progress");
+        new SwiftBuildStatus(instance(mockedStatusItem));
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+
+        // Setup swiftStatus
+        configurationMock.setValue("swiftStatus");
+        new SwiftBuildStatus(instance(mockedStatusItem));
+        listenerMock.notifyAll({ execution: mockedTaskExecution });
+
+        const mockReporter = mock<vscode.Progress<{ message: string }>>();
+        const [, statusCallback] = capture<vscode.Task, () => void>(
+            mockedStatusItem.showStatusWhileRunning
+        ).last();
+        const [, progressCallback] = capture<
+            vscode.ProgressOptions,
+            (p: vscode.Progress<unknown>) => Promise<void>
+        >(windowMock.withProgress).last();
+
+        // Execute callback reporters
+        statusCallback();
+        progressCallback(instance(mockReporter));
+
+        mockedProcess.write(
+            "Fetching https://github.com/apple/example-package-figlet from cache\n" +
+                "[6/7] Building main.swift\n" +
+                "[7/7] Applying MyCLI\n" +
+                "Build complete!"
+        );
+
+        // Report nothing
+        verify(mockReporter.report(anything())).never();
+        verify(mockedStatusItem.update(anything(), anything())).never();
+    });
+});