Initial implementation of Command Plugins

[abertelrud]

Initial implementation of https://forums.swift.org/t/pitch-package-manager-command-plugins/53172.

This feature is experimental until approved, so it requires a tools version of 999.0 in the package that declares the command plugin, and requires SWIFTPM_ENABLE_COMMAND_PLUGINS to be set to 1 in the environment when invoked.

This implementation temporarily uses a swift package plugin <verb> form for invoking command plugins. Per the pitch and proposal, plugin-defined commands are intended to be invocable using the shortcut swift package <verb> when the feature is complete (there is discussion of even supporting swift <verb> but that is not part of the current proposal).

Motivation:

Implementation of a new feature being pitched, kept under a feature flag until the proposal is accepted.

Staging:

This feature is being developed incrementally across this PR and follow-on PRs. This PR it is in a state where it could be merged to get most of the functionality into the mainline (behind the feature flag), including all the planned changes to PackageDescription and PackagePlugin. Remaining items of the full feature would then be landed in future PRs. The SWIFTPM_ENABLE_COMMAND_PLUGINS guard flag will only be removed once the evolution proposal has been approved and the feature has been fully implemented including any amendments.

Modifications:

• add the new enum cases for the command plugin capability to PackageDescription
• add serialization of those cases in PackageDescriptionSerializer
• deserialize them in the ManifestJSONLoader and add to the package model
• add manifest source generation for the command capability
• add ability to describe packages that have command plugins
• add a unit test to check loading a manifest with a command capability
• add protocol declarations for command plugin entry point in the plugin API
• ability to invoke the command plugin in libSwiftPM and unit tests
• add a unit test to check round trip calling into the plugin
• add ability to invoke the command plugin from CLI
• made the host-to-plugin communication be bidirectional and interactive
• ability for plugin to call back into SwiftPM for services
• ability to create symbol graphs for a target when a plugin requests it
• ability to run a build when the plugin requests it

Remaining in upcoming PRs:

• request approval to run command plugins that need special permissions
• implement the option to list the available command plugins
• support qualifiers for ambiguous plugin commands
• ability to run a test when the plugin requests it
• build any executables needed by command plugins before invoking the plugin
• provide access to toolchain executables in the map sent to plugins
• add async to all the plugin APIs (in separate PR, depends on back deployment)

[abertelrud]

@swift-ci please smoke test

[abertelrud]

@swift-ci please smoke test

[abertelrud]

@swift-ci please smoke test

[abertelrud]

@swift-ci please smoke test

[abertelrud]

@swift-ci please smoke test

[abertelrud]

@swift-ci please smoke test

[abertelrud]

@swift-ci please smoke test

[abertelrud]

Merging this after approval and successful CI tests as it is a good baseline for starting to use the feature. Further implementation in upcoming PRs.

[compnerd]

This seems to have regressed the windows build: https://ci-external.swift.org/job/oss-swift-windows-toolchain-x86_64-vs2019/379/console

[abertelrud]

This seems to have regressed the windows build: https://ci-external.swift.org/job/oss-swift-windows-toolchain-x86_64-vs2019/379/console

Thanks for letting me know, and apologies. It looks to be easy to fix forward. Investigating.

[AnthonyMDev]

Command plugins are such an awesome addition, thanks so much for making this happen.

I see that in the Remaining in Upcoming PRs sections there is:

support qualifiers for ambiguous plugin commands

Any chance that this will be coming prior to a stable release of Xcode 14? Conflicting plugin command names currently mean that it is impossible to execute either command.

[tomerd]

cc @abertelrud @neonichu

[neonichu]

Filed rdar://99887952 (Support qualifiers for ambiguous plugin commands)