Skip to content

Adopt the ArgumentParser library for the command-line tools #2653

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 39 commits into from
Aug 26, 2020

Conversation

natecook1000
Copy link
Member

@natecook1000 natecook1000 commented Mar 11, 2020

This transitions the different SPM command-line tools to use the ArgumentParser library instead of the TSC parser. Among the changes is that the tools are no longer subclasses of SwiftTool. Instead, each tool creates a SwiftTool instance to use when necessary. It may be useful to follow up with a refactoring to either move some of the functionality that's in the tools into SwiftTool or to otherwise separate the CLI from the actual functions being performed.

Still to do:

  • Get tests passing
  • Add shell completion generation to ArgumentParser

@natecook1000
Copy link
Member Author

@swift-ci Please smoke test

1 similar comment
@natecook1000
Copy link
Member Author

@swift-ci Please smoke test

@natecook1000
Copy link
Member Author

@swift-ci please smoke test self hosted

@natecook1000
Copy link
Member Author

@swift-ci Please smoke test self hosted

The SwiftTool initializer performs some global configuration steps,
so it isn't really safe to require that individual commands create
an instance, even when they don't need one. This moves the
customization point to a run(_ swiftTool: SwiftTool) method.
@natecook1000
Copy link
Member Author

@swift-ci Please smoke test self hosted

@natecook1000
Copy link
Member Author

@swift-ci Please smoke test self hosted

@natecook1000
Copy link
Member Author

@swift-ci Please smoke test self hosted

# Conflicts:
#	Sources/Commands/Options.swift
#	Sources/Commands/SwiftBuildTool.swift
#	Sources/Commands/SwiftPackageTool.swift
#	Sources/Commands/SwiftRunTool.swift
#	Sources/Commands/SwiftTestTool.swift
#	Sources/Commands/SwiftTool.swift
@abertelrud
Copy link
Contributor

I think update-checkout configs will need to be updated in the swift and swift-source-compatibility-suite repos, and CI will have to be reconfigured to use the 0.3.0 tag before this is merged. Currently, 0.0.6 is the version being checked out.

Got it. For the smoke test it must be using the version that's checked out next to swiftpm rather than the dependency in the package. Although I wonder what fallout switching to 0.3.0 across the board would have on other clients...

@abertelrud
Copy link
Contributor

abertelrud commented Aug 22, 2020

I just found your existing PR swiftlang/swift-source-compat-suite#443 that would move it to 0.2.0. Were there concerns about merging that one at the time? (there's nothing in the comments)

@owenv
Copy link
Contributor

owenv commented Aug 22, 2020

Not that I remember, I just lost track of it at the time. @shahmishal does need to be cc'ed on any changes in this area though because I think there's some Jenkins configuration required to change checkout tags for security reasons.

Got it. For the smoke test it must be using the version that's checked out next to swiftpm rather than the dependency in the package. Although I wonder what fallout switching to 0.3.0 across the board would have on other clients...

This might be a problem in the future, but I don't think there are any other clients right now. The checkout was originally setup alongside the swift-driver CMake build but there were some issues preventing adoption for awhile.

@abertelrud
Copy link
Contributor

abertelrud commented Aug 22, 2020

Sounds great. Would you be willing to update your existing PR (which Mishal already approved once) to 0.3.0 and then we can ask for his re-approval with the updated version? I could open a new PR, but since you already have one open for essentially the same thing, we might as well reuse it. We'll block merging of this PR on the merging of that one.

@owenv
Copy link
Contributor

owenv commented Aug 22, 2020

I messed up a force push, so I had to re-create the PRs, but they're available here:
swiftlang/swift#33593
swiftlang/swift-source-compat-suite#454

@abertelrud
Copy link
Contributor

Thanks, very much appreciated!

@abertelrud
Copy link
Contributor

At this point this is just waiting for the CI to be configured to check out 0.3.0 of ArgumentParser (those are the PRs that @owenv mentioned; they just need to be approved and merged, then we can merge this).

@natecook1000
Copy link
Member Author

@swift-ci Please smoke test

@abertelrud
Copy link
Contributor

Now that the other two PRs are merged, we just need these tests to pass and then can merge this. 🤞

@abertelrud
Copy link
Contributor

The failure is unrelated (error: 'Triple' is ambiguous for type lookup in this context) and is being worked on, but we'll need to wait to merge until it's fixed.

@abertelrud
Copy link
Contributor

Now that #2883 is merged, we can try this CI again.

@abertelrud
Copy link
Contributor

@swift-ci please smoke test

@abertelrud
Copy link
Contributor

The Linux failure seems to involve an error in SourceKit-LSP's build-script-helper.py script. Need to investigate whether it's related.

@abertelrud
Copy link
Contributor

It does look related:

15:13:49 + /home/buildnode/jenkins/workspace/swift-package-manager-Linux-smoke-test/branch-master/sourcekit-lsp/Utilities/build-script-helper.py build --package-path /home/buildnode/jenkins/workspace/swift-package-manager-Linux-smoke-test/branch-master/sourcekit-lsp --build-path /home/buildnode/jenkins/workspace/swift-package-manager-Linux-smoke-test/branch-master/build/buildbot_incremental/sourcekitlsp-linux-x86_64 --configuration release --toolchain /home/buildnode/jenkins/workspace/swift-package-manager-Linux-smoke-test/branch-master/build/buildbot_incremental/toolchain-linux-x86_64/usr --ninja-bin /home/buildnode/jenkins/workspace/swift-package-manager-Linux-smoke-test/branch-master/build/buildbot_incremental/ninja-build/ninja --verbose
15:13:49 error: Missing value for '-c '

So this might be a difference in how this flag is being parsed.

@abertelrud
Copy link
Contributor

I don't see any problems with how the build-script-helper.py script is forming the arguments. Something odd is going on, though: the script always passes --configuration, but the error message is about -c. In the error message, I always see swift-build echoing back the form (log or short) of the flag that was actually passed to it, it's possible that it's a different invocation than then one directly in build-script-helper.py and that the lines are just intermixed.

@abertelrud
Copy link
Contributor

abertelrud commented Aug 26, 2020

I can actually reproduce this using the exact line that SourceKit-LSP is passing:

.build/debug/swift-build  --package-path /home/buildnode/jenkins/workspace/swift-package-manager-Linux-smoke-test/branch-master/sourcekit-lsp --build-path /home/buildnode/jenkins/workspace/swift-package-manager-Linux-smoke-test/branch-master/build/buildbot_incremental/sourcekitlsp-linux-x86_64 --configuration release --verbose -Xcxx -I -Xcxx /home/buildnode/jenkins/workspace/swift-package-manager-Linux-smoke-test/branch-master/build/buildbot_incremental/toolchain-linux-x86_64/usr/lib/swift -Xcxx -I -Xcxx /home/buildnode/jenkins/workspace/swift-package-manager-Linux-smoke-test/branch-master/build/buildbot_incremental/toolchain-linux-x86_64/usr/lib/swift/Block -Xlinker -rpath -Xlinker ../lib/swift/linux
error: Missing value for '-c <configuration>'

with the changes in this PR, but it doesn't reproduce without them. I'll see if I can figure out what's causing it to think it's seeing a -c.

@natecook1000
Copy link
Member Author

Looks like the C++ compiler flag option had an incorrect name. Since -Xcxx wasn't matching as a full name, the parser was treating it as a group of short options, and recognized -c, but expected a value to follow. Better diagnostics would help here!

@natecook1000
Copy link
Member Author

@swift-ci Please smoke test

@abertelrud
Copy link
Contributor

Fantastic; thank you! I guess this wasn't seen on macOS because the script only passes that flag on Linux? Not sure what happened there. I guess I also wonder whether there are other combinations of flags that might get passed and that might be misinterpreted as combined single-file flags, resulting in confusing error messages. Improving handling of incorrectly specified parameters should not prevent us from merging, of course, but I am wondering whether we should disable combining of short-from flags for SwiftPM.

@abertelrud
Copy link
Contributor

@natecook1000, assuming this passes, are you fine with merging it? I think all the other issues have been taken care of (downstream CI etc).

@natecook1000 natecook1000 merged commit b30b5e2 into swiftlang:master Aug 26, 2020
@natecook1000 natecook1000 deleted the argumentparser branch August 26, 2020 17:31
skarred14 pushed a commit to val-verde/swift-package-manager that referenced this pull request Aug 26, 2020
…g#2653)

This transitions the Commands tools to use ArgumentParser instead 
of the TSC parser. The biggest change is that the tools are no 
longer subclasses of SwiftTool. Instead, each tool conforms to the
SwiftCommand protocol, which requires the shared options and creates
a SwiftTool instance after parsing.
CodaFi added a commit to CodaFi/swift-package-manager that referenced this pull request Aug 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants