Announcing System.CommandLine 2.0 Beta 3

[jonsequitur]

System.CommandLine 2.0 Beta 3

We've been getting great feedback on the changes we made for the System.CommandLine Beta 2 release. (The announcement is here if you'd like to catch up.) The Beta 3 release incorporates feedback, bug fixes, and more performance improvements. Please give it a try and let us know your thoughts.

Here's an overview of the most significant changes in this release.

Removed unnecessary interfaces

A number of interfaces have been removed in this release. While this was done during our performance work over the last couple of months, performance wasn't the main motivation for this change. These interfaces were initially introduced to provide an API that would discourage changing the configuration of a Parser once it's been instantiated and to leave open the possibility of making the underlying model immutable. An immutable model wasn't worth the cost or complexity in the end. Meanwhile, no compelling use case arose for people to create their own implementations of ICommand (for example) as opposed to inheriting Command. So while removing these interfaces does save a little time during JIT compilation and removed the need for numerous casts, the main reason they've been removed is to reduce the complexity of a redundant extension point.

These are the interfaces that have been removed and the corresponding classes that you can reference in their place :

Removed interface	Class you should reference instead
IArgument	Argument
ICommand	Command
IDirectiveCollection	DirectiveCollection
IIdentifierSymbol	IdentifierSymbol
IOption	Option
ISymbol	Symbol

Command line configuration is now validated on demand rather than up-front

Up until this release, validation of a command line configuration would happen immediately as symbols were added. The third line of the following code would throw an exception:

var root = new RootCommand();
root.Add(new Option<string>("--duplicate"));
root.Add(new Option<string>("--duplicate"));

System.ArgumentException: Alias '--duplicate' is already in use.

In typical usage this validation is wasteful because once compiled, the parser will be configured the same way every time your app starts up. If it was valid when you compiled it, it will always be valid. We identified this as a place to improve performance by moving validation to a method that you can run when appropriate: CommandLineConfiguration.ThrowIfInvalid. Maybe you'll want to use it at runtime in specialized cases such as configuring a parser based on an external configuration. We strongly encourage you use it in unit testing. Here's an example:

[Fact]
public void Parser_is_configured_correctly()
{
    var command = MyApp.CreateRootCommand(); // exposing your root command for testing is very helpful
    var configuration = new CommandLineConfiguration(command);
    configuration.ThrowIfInvalid();
}

Removed SymbolSet

The SymbolSet class was originally created to support name-based lookups and up-front checking for duplicate command or option aliases that could lead to an ambiguous parser configuration. Once we had moved the name-based binding functionality out of the core System.CommandLine library and moved validation to CommandLineConfiguration.ThrowIfInvalid, SymbolSet no longer had a role to play, so it has been removed.

Removed the [debug] directive

The [debug] directive provided a consistent way to attach a debugger to your System.CommandLine-driven application and stop on a breakpoint in your startup code. While this is clearly useful, we weren't satisfied with the security of the design. Until this is improved, we’ve removed support for this directive from the library. We plan to bring it back in the future when a better design is in place. If you have ideas or use cases for it, please let us know!

A simpler validator API

Since the earliest versions of System.CommandLine, it's been possible to add custom validation logic to a symbol using the AddValidator method on a Command, Option, or Argument, like this:

var option = new Option<int>("-x");
option.AddValidator((OptionResult result) =>
{
    // validation logic here
});

Originally, the way to signal a validation error was to return a string from your validator delegate, while returning null indicated there was no error. This was a pretty unusual interface and had been on the list of things to fix for a long time. To make matters worse, it became ambiguous with the introduction of the SymbolResult.ErrorMessage property which can be set to indicate an error. If you were to set this to one value and return a different value, which should take precedence?

In this release we've resolved these oddities by making the ValidateSymbolResult<T> delegate void-returning. Now, the only way to set an error message from a custom validator is by setting SymbolResult.ErrorMessage. Here's an example that ensures the value of the option is a number from 1 to 100:

var option = new Option<int>("-x");
option.AddValidator((OptionResult result) =>
{
    if (result.GetValueForOption(option) <= 0)
    {
        result.ErrorMessage = "The value of -x must be greater than 0.";
    }
});

Support for trimming

In .NET 6, full support was added for publishing trimmed, self-contained applications. Previous versions of System.CommandLine would produce trim warnings due to a reliance on certain reflection APIs. This is no longer the case. If you would like to use trimming to publish smaller command line apps using System.CommandLine, please give it a try and let us know what you think.

One notable breaking change that resulted from removing much of the reflection code in System.CommandLine is that we've had to remove the conventions that allowed instantiating types when:

• the type has a constructor that accepts a single string parameter, or
• the type has an associated TypeConverter.

This will be a breaking change for many custom types. A few common types will still work because we've added special cases for them, including DirectoryInfo, FileInfo, FileSystemInfo, Uri, and the common numeric types. For other types that were previously relying on these conventions, you can continue to parse them by passing a ParseArgument<T> to the Option<T> or Argument<T> constructors.

[SiliconFiend]

Beta 3 broke my configuration, specifically an optional (nullable) int argument to a command. The last Beta 2 is still working.
The invocation looks like this:
Program.exe mycommand 1234
And here's the exception output:

Unhandled exception: System.ArgumentException: The SetHandler call for command 'mycommand' is missing an Argument or Option for the parameter at position 1. Did you mean to pass one of these?
Argument<Nullable`1> my-int
   at System.CommandLine.Handler.GetValueForHandlerParameter[T](IValueDescriptor[] symbols, Int32& index, InvocationContext context)
   at System.CommandLine.Handler.<>c__DisplayClass11_0`11.<SetHandler>b__0(InvocationContext context)
   at System.CommandLine.Invocation.AnonymousCommandHandler.<>c__DisplayClass2_0.<.ctor>g__Handle|0(InvocationContext context)
   at System.CommandLine.Invocation.AnonymousCommandHandler.InvokeAsync(InvocationContext context)
   at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass20_0.<<UseParseErrorReporting>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass13_0.<<UseHelp>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass24_0.<<UseVersionOption>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<<UseTypoCorrections>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseSuggestDirective>b__21_0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass19_0.<<UseParseDirective>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<RegisterWithDotnetSuggest>b__6_0>d.MoveNext()
--- End of stack trace from previous location ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass9_0.<<UseExceptionHandler>b__0>d.MoveNext()

Here's how I'm setting up the configuration (the real config has a few more options after the nullable int):

var rootCommand = new RootCommand();
rootCommand.Description = "A root command";
rootCommand.AddGlobalOption(new Option<bool>(new string[] { "--quiet", "-Q", "-q" }, description: "Do not display informational messages"));

var myCommand = new Command("mycommand", "A command")
{
	new Argument<int?>("my-int", description: "An optional int argument")
};

IValueDescriptor[] commandParams = rootCommand.Children.OfType<IValueDescriptor>().ToArray().Concat(myCommand.Children.OfType<IValueDescriptor>().ToArray()).ToArray();

myCommand.SetHandler<InvocationContext, bool, int?>(MyCommandHandler, commandParams);
rootCommand.AddCommand(myCommand);

I have other commands with similar configurations that are still working, but this is the only one with a nullable int argument (actually it's the only one with an Argument; the others only have Options) so I'm assuming that's the problem.

[SiliconFiend]

I did some more testing and it appears the problem is with the Argument, not the nullable int. A little more debugging showed that my prior command which was building the param arrays is falling down now because it seems the command Children collection is sorted by type. The argument used to fall after the global options and before the command-specific options (i.e., in the order it was added to the command), but now it has moved to the end of the param array. Looks like this commit broke it: 07b1b17

Is there a recommended way to build the param list without having to re-type each of the params individually, while preserving the order? I'm trying to avoid declaring variables for each of the options because it feels like unnecessary clutter.

[steveberdy]

@jonsequitur great release! I do want to add that I also have the same issue that @SiliconFiend has.

[baywet]

Thanks for this new release! Have you considered creating GitHub releases with this information so it flows naturally when dependabot creates PRs in repos to update the dependency?

[chrisxfire]

Any plans to update the main documentation? For example, I can't find a description of how SetHandler works in the documentation other than some use of it in examples without further explanation. This may be common knowledge to those who have been using System.CommandLine for some time, but it would be helpful for those new to it. Similar for arity.

[jonsequitur]

Much more thorough and up-to-date documentation is now available at Microsoft Docs:

https://docs.microsoft.com/en-us/dotnet/standard/commandline/

[chrisxfire]

Thanks!

[c-s-n]

I tested the mentioned validator API with setting of the ErrorMessage and stumbled over a small bug.

var inputDirectoryArgument = new Argument<DirectoryInfo>(name: "Input directory");
inputDirectoryArgument
    .AddValidator(result =>
        {
            result.ErrorMessage = "Validation error on input directory";
        });

var outputDirectoriesArgument = new Argument<DirectoryInfo[]>(name: "Output directories");
outputDirectoriesArgument
    .AddValidator(result =>
        {
            result.ErrorMessage = "Validation error on output directories";
        });

When starting the program only with an input directory as an argument, but no output directory argument, the validation message for it is duplicated:

Validation error on input directory
Validation error on output directories
Validation error on output directories

This is not the case if I specify one or more output directory arguments.

I could file this as a separate bug if needed.

[KalleOlaviNiemitalo]

I was watching releases on this repository, but 2.0 Beta 3 was not marked as a release, so I didn't notice it before yesterday.

[jonsequitur]

I could file this as a separate bug if needed.

@c-s-n If you could open an issue it would be appreciated. Could you please include a more detailed example including the command lines that produce the bug? Thanks!

[jonsequitur]

@KalleOlaviNiemitalo I just created one: https://github.com/dotnet/command-line-api/releases/tag/v2.0.0-beta3.22114.1.

We released a few bug fix package updates on beta 3 after the announcement. This points to the code for the most recent one.

[iBicha]

@jonsequitur are there any updates around System.CommandLine.Hosting? Should I rely on it, or rely on the BindingContext for service container?

[jonsequitur]

@iBicha No updates yet. The scenario is an important one and we'll definitely address it in some form but until the APIs that System.CommandLine.Hosting depends on are stable, it's too early to say what the next steps should be.

I'd like to take a look at whether the BinderBase<T> approach can work for System.CommandLine.Hosting and allow us to remove the service provider from BindingContext, which was recommended during last week's API review.

[KalleOlaviNiemitalo]

we've had to remove the conventions that allowed instantiating types when:

* the type has a constructor that accepts a single string parameter

That feature is still documented here:

command-line-api/docs/model-binding.md

Lines 165 to 167 in 1d98a75

	## Anything with a string constructor
	But `FileInfo` and `DirectoryInfo` are not special cases. Any type having a constructor that takes a single string parameter can be bound in this way. Go back to the previous example and try using a `Uri` instead.

Is it now available from System.CommandLine.NamingConventionBinder somehow, as advertised at the top of that file? If not, it would be good to remove the section or add a warning to it.

I was using Option<HttpMethod>, which no longer works out of the box. I added a custom parser (the most relevant documentation seems to be at Custom validation and binding) but it doesn't feel entirely satisfactory because I had to hardcode the default value in the parser method, rather than let options in different commands use the same parser but choose their own default values (although I suppose I could use per-option lambda expressions for that). Also, the interaction between Arity and OnlyTake is not quite clear to me.