Add `PackageSearchClient` #5950

neonichu · 2022-12-06T21:17:01Z

This offers a generic interface which allows clients to find packages via exact matching of registry identities or URLs, as well as search through collections and index if there are no exact matches.

neonichu · 2022-12-06T21:17:17Z

@swift-ci please smoke test

Sources/PackageMetadata/PackageMetadata.swift

Sources/SourceControl/GitRepository.swift

neonichu · 2022-12-06T21:23:16Z

Example results:

Search for "apple.swift-argument-parser"

success([PackageMetadata.Package(identity: apple.swift-argument-parser, location: nil, branches: [], versions: [1.0.3], readmeURL: Optional(https://github.com/apple/swift-argument-parser/raw/main/README.md))])

Search for "networking" with Apple default collection configured

success([PackageMetadata.Package(identity: swift-nio, location: Optional("https://github.com/apple/swift-nio.git"), branches: [], versions: [2.43.1, 2.43.0, 2.42.1, 1.14.4, 1.14.3, 1.14.2], readmeURL: Optional(https://raw.githubusercontent.com/apple/swift-nio/main/README.md))])

Search for "https://github.com/jpsim/yams"

success([PackageMetadata.Package(identity: yams, location: Optional("https://github.com/jpsim/yams"), branches: ["fuzzer", "gh-pages", "main", "nn-key-coding-strategy-2", "rules_xcodeproj", "use-swift-dtoa", "yams-3-branch"], versions: [0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.2.0, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.3.7, 0.4.0, 0.4.1, 0.5.0, 0.6.0, 0.7.0, 1.0.0, 1.0.1, 1.0.2, 2.0.0, 3.0.0, 3.0.1, 4.0.0, 4.0.1, 4.0.2, 4.0.3, 4.0.4, 4.0.5, 4.0.6, 5.0.0, 5.0.1], readmeURL: Optional(https://github.com/jpsim/yams/raw/main/README.md))])

Sources/PackageMetadata/PackageMetadata.swift

neonichu · 2022-12-07T00:23:43Z

@swift-ci please smoke test

Sources/PackageMetadata/PackageMetadata.swift

tomerd · 2022-12-07T22:42:24Z

Sources/PackageMetadata/PackageMetadata.swift

+
+    public func findPackages(
+        _ query: String,
+        callback: @escaping (Result<[Package], Error>) -> Void


would be nice to have all new apis adopt async/await if we can

We could do that, but considering the dire state of async/await on CI, I'm not sure if we should proliferate it, yet.

Just curious, what are you referring to by "state of async/await on CI"? I was hoping I could try using it in some of the new experimental-destination subcommands, especially after we update ArgumentParser 1.1.4 where AsyncParsableCommand is available. Is there anything I should be aware before I even consider it?

We can't use it at all on macOS CI so far, in case that is necessary, we should avoid it. As long as we can scope the use so that macOS CI still works but just doesn't test a certain feature, that should be fine in my book.

tomerd · 2022-12-07T22:45:02Z

Sources/PackageMetadata/PackageMetadata.swift

+                                                                                           location: $0.location,
+                                                                                           versions: $0.versions.map { $0.version },
+                                                                                           readmeURL: $0.readmeURL) }
+                    if packages.isEmpty, let error = error {


the requirement on both is a bit odd? but perhaps i am missing something

The idea was to normally report the success for the empty search result, but if we don't get a result via search to provide any previous error. So if the query was a registry ID or URL that caused an error which also did not yield any results through search, we'd see the underlying error.

tomerd · 2022-12-07T22:49:39Z

one other thing to add here is that the result can include where (which provider) the metadata came from, I think that can be useful in a UI

neonichu · 2022-12-08T17:55:16Z

@swift-ci please smoke test

neonichu · 2022-12-08T18:34:43Z

Looks like 5.6 needs a bit more help:

[1138/1162] Compiling PackageMetadata PackageMetadata.swift
/Users/ec2-user/jenkins/workspace/swift-package-manager-with-xcode-self-hosted-PR-osx/branch-main/swiftpm/Sources/PackageMetadata/PackageMetadata.swift:96:43: error: unable to infer complex closure return type; add explicit type to disambiguate
        let fetchStandalonePackageByURL = { (error: Error?) in
                                          ^
                                                            -> <#Result#> 
/Users/ec2-user/jenkins/workspace/swift-package-manager-with-xcode-self-hosted-PR-osx/branch-main/swiftpm/Sources/PackageMetadata/PackageMetadata.swift:147:48: error: 'nil' requires a contextual type
            return fetchStandalonePackageByURL(nil)

This offers a generic interface which allows clients to find packages via exact matching of registry identities or URLs, as well as search through collections and index if there are no exact matches.

neonichu · 2022-12-08T21:28:54Z

@swift-ci please smoke test

neonichu · 2022-12-09T00:18:03Z

ugh, looks like our good friend is back :/
Time to revert #5957?

/private/var/folders/pf/p5g2vb2j0yx1x6vlmxdppc700000gn/T/TemporaryDirectory.duPOYz/Foo/Tests/package-nameTests/package_nameTests.swift:9:9: error: cannot find 'XCTAssertEqual' in scope
            XCTAssertEqual(package_name().text, "Hello, World!")
            ^~~~~~~~~~~~~~
    /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks/XCTest.framework/Headers/XCTestAssertions.h:117:9: note: macro 'XCTAssertEqual' not imported: function like macros not supported
    #define XCTAssertEqual(expression1, expression2, ...) \
            ^
    error: fatalError
    , output: "", stderr: "Building for debugging...\n[1/3] Compiling package_name package_name.swift\n[2/4] Merging module package_name\n[2/4] Linking package-name\n[4/5] Compiling package_nameTests package_nameTests.swift\n/private/var/folders/pf/p5g2vb2j0yx1x6vlmxdppc700000gn/T/TemporaryDirectory.duPOYz/Foo/Tests/package-nameTests/package_nameTests.swift:9:9: error: cannot find \'XCTAssertEqual\' in scope\n        XCTAssertEqual(package_name().text, \"Hello, World!\")\n        ^~~~~~~~~~~~~~\n/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks/XCTest.framework/Headers/XCTestAssertions.h:117:9: note: macro \'XCTAssertEqual\' not imported: function like macros not supported\n#define XCTAssertEqual(expression1, expression2, ...) \\\n        ^\nerror: fatalError\n")"Test Case '-[WorkspaceTests.InitTests testNonC99NameExecutablePackage]' failed (21.897 seconds).Test Suite 'InitTests' failed at 2022-12-08 23:26:38.876.

neonichu · 2022-12-09T00:18:50Z

@swift-ci please smoke test macOS

This reverts commit c6faab7. The failure showed up again in #5950, so it's not actually gone.

yim-lee

I think we need to let @compnerd know whenever there is a new module?

Other than that, LGTM

neonichu · 2022-12-15T04:39:52Z

@swift-ci please smoke test windows

neonichu · 2022-12-15T04:40:18Z

Looks like the Windows job wasn't run previously?

neonichu · 2022-12-15T08:39:12Z

Windows error looks unrelated:

C:\Users\swift-ci\jenkins\workspace\swiftpm-PR-windows\llvm-project\llvm\lib\IR\DebugInfoMetadata.cpp(948): error C2429: language feature 'init-statements in if/switch' requires compiler flag '/std:c++17'

neonichu · 2022-12-15T08:39:21Z

@swift-ci please smoke test windows

neonichu · 2022-12-16T06:08:09Z

@swift-ci please smoke test windows

neonichu · 2022-12-16T17:36:57Z

I think we need to let @compnerd know whenever there is a new module?

Was looking into adding the new module to the installer scripts, but actually I am not adding it to the CMake build at all for now, so this should be fine as-is.

compnerd · 2022-12-16T17:39:34Z

As long as nothing is linking against it and it doesn't diverge functionality, I think that should be fine. But we should keep the platforms in parity - so if we are adding a new module, we should be adding it to the CMake build and the swift-installer-scripts packaging IMO.

neonichu · 2022-12-16T18:25:57Z

I agree with that, but I see this particular change as an extension of the package-collections functionality which was never available on Windows to begin with (probably by mistake). We should address this, but separately from this PR.

neonichu added the WIP Work in progress label Dec 6, 2022

neonichu self-assigned this Dec 6, 2022

neonichu commented Dec 6, 2022

View reviewed changes

Sources/PackageMetadata/PackageMetadata.swift Show resolved Hide resolved

neonichu commented Dec 6, 2022

View reviewed changes

Sources/PackageMetadata/PackageMetadata.swift Show resolved Hide resolved

neonichu commented Dec 6, 2022

View reviewed changes

Sources/SourceControl/GitRepository.swift Show resolved Hide resolved

yim-lee reviewed Dec 6, 2022

View reviewed changes

Sources/PackageMetadata/PackageMetadata.swift Show resolved Hide resolved

Sources/PackageMetadata/PackageMetadata.swift Show resolved Hide resolved

Sources/PackageMetadata/PackageMetadata.swift Show resolved Hide resolved

Sources/PackageMetadata/PackageMetadata.swift Show resolved Hide resolved

neonichu force-pushed the add-package-metadata branch from 345c97e to 2677d2a Compare December 7, 2022 00:23

tomerd reviewed Dec 7, 2022

View reviewed changes

Add PackageSearchClient

28ec83e

This offers a generic interface which allows clients to find packages via exact matching of registry identities or URLs, as well as search through collections and index if there are no exact matches.

neonichu force-pushed the add-package-metadata branch from 2677d2a to 28ec83e Compare December 8, 2022 21:28

neonichu marked this pull request as ready for review December 8, 2022 21:28

neonichu requested review from abertelrud and elsh as code owners December 8, 2022 21:28

neonichu removed the WIP Work in progress label Dec 8, 2022

neonichu added a commit that referenced this pull request Dec 9, 2022

Revert "Reenable disabled tests (#5957)"

518d777

This reverts commit c6faab7. The failure showed up again in #5950, so it's not actually gone.

neonichu mentioned this pull request Dec 9, 2022

Revert "Reenable disabled tests (#5957)" #5963

Merged

neonichu added a commit that referenced this pull request Dec 9, 2022

Revert "Reenable disabled tests (#5957)" (#5963)

dd7e9cc

This reverts commit c6faab7. The failure showed up again in #5950, so it's not actually gone.

yim-lee approved these changes Dec 15, 2022

View reviewed changes

theMomax mentioned this pull request Dec 16, 2022

Fix "Add emit extension block symbols option to dump-symbol-graph and SymbolGraphOptions" #5978

Merged

neonichu merged commit 4d312bd into main Dec 16, 2022

neonichu deleted the add-package-metadata branch December 16, 2022 17:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `PackageSearchClient` #5950

Add `PackageSearchClient` #5950

neonichu commented Dec 6, 2022

neonichu commented Dec 6, 2022

neonichu commented Dec 6, 2022

neonichu commented Dec 7, 2022

tomerd Dec 7, 2022

neonichu Dec 7, 2022

MaxDesiatov Dec 15, 2022

neonichu Dec 16, 2022

tomerd Dec 7, 2022

neonichu Dec 7, 2022

tomerd commented Dec 7, 2022

neonichu commented Dec 8, 2022

neonichu commented Dec 8, 2022

neonichu commented Dec 8, 2022

neonichu commented Dec 9, 2022 •

edited

Loading

neonichu commented Dec 9, 2022

yim-lee left a comment

neonichu commented Dec 15, 2022

neonichu commented Dec 15, 2022

neonichu commented Dec 15, 2022

neonichu commented Dec 15, 2022

neonichu commented Dec 16, 2022

neonichu commented Dec 16, 2022

compnerd commented Dec 16, 2022

neonichu commented Dec 16, 2022

Add PackageSearchClient #5950

Add PackageSearchClient #5950

Conversation

neonichu commented Dec 6, 2022

neonichu commented Dec 6, 2022

neonichu commented Dec 6, 2022

neonichu commented Dec 7, 2022

tomerd Dec 7, 2022

Choose a reason for hiding this comment

neonichu Dec 7, 2022

Choose a reason for hiding this comment

MaxDesiatov Dec 15, 2022

Choose a reason for hiding this comment

neonichu Dec 16, 2022

Choose a reason for hiding this comment

tomerd Dec 7, 2022

Choose a reason for hiding this comment

neonichu Dec 7, 2022

Choose a reason for hiding this comment

tomerd commented Dec 7, 2022

neonichu commented Dec 8, 2022

neonichu commented Dec 8, 2022

neonichu commented Dec 8, 2022

neonichu commented Dec 9, 2022 • edited Loading

neonichu commented Dec 9, 2022

yim-lee left a comment

Choose a reason for hiding this comment

neonichu commented Dec 15, 2022

neonichu commented Dec 15, 2022

neonichu commented Dec 15, 2022

neonichu commented Dec 15, 2022

neonichu commented Dec 16, 2022

neonichu commented Dec 16, 2022

compnerd commented Dec 16, 2022

neonichu commented Dec 16, 2022

Add `PackageSearchClient` #5950

Add `PackageSearchClient` #5950

neonichu commented Dec 9, 2022 •

edited

Loading