[SR-8262] Swift Package Manager fails at checkout when dependent repository contains Git LFS objects

[swift-ci]

Previous ID	SR-8262
Radar	rdar://problem/42246102
Original Reporter	yifei (JIRA User)
Type	Bug

Additional Detail from JIRA

Votes	2
Component/s	Package Manager
Labels	Bug
Assignee	None
Priority	Medium

md5: 2218c2c1d09b919e983b204e24d25058

Issue Description:

The error is in the format of: (XXXXX being the name of LFS-tracked file)

Error downloading object: XXXXX (hash): Smudge error: Error downloading XXXXX (hash): batch request: missing protocol: "/path/to/project/.build/repositories/dependency.git/info/lfs"

error: external filter 'git-lfs filter-process' failed
fatal: XXXXX: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

This happens during the "cloneCheckout" phase i.e. invoking "git clone --shared". If I disable the LFS filter by editing the global .gitconfig, the project builds fine.

[aciidgh]

@swift-ci create

[jtbandes]

Has there been any update on this issue? There's a similar discussion at https://forums.swift.org/t/swiftpm-with-git-lfs/42396.

I am running into this issue as well with https://github.com/foxglove/mcap. This dependency fails during checkout: .package(url: "https://github.com/foxglove/mcap", branch: "main"),

Creating working copy for https://github.com/foxglove/mcap
error: Couldn’t check out revision ‘2dad37cad3c7875842b44f3fe927e2dcfdecf7d4’:
    Downloading docs/img/0398582-hero.png (669 KB)
    Error downloading object: docs/img/0398582-hero.png (266f9d3): Smudge error: Error downloading docs/img/0398582-hero.png (266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5): error transferring "266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5": [0] remote missing object 266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5
    Errors logged to /Users/jacob/Desktop/testpackage/.build/checkouts/mcap/.git/lfs/logs/20220923T112418.927225.log
    Use `git lfs logs last` to view the log.
    error: external filter 'git-lfs filter-process' failed
    fatal: docs/img/0398582-hero.png: smudge filter lfs failed

A simple git clone succeeds.

[shoaibsadanaut]

Is there any progress on it or someone has a solution?

[neonichu]

We're talking about an Xcode specific issue here, correct? I tried reproducing with https://github.com/foxglove/mcap and swift build and it seems to work fine for me.

[jtbandes]

Nope, it's not an Xcode specific issue. swift build within the repository works fine when you manually clone it. The problem is with swiftpm's automatic cloning.

Try creating an empty directory and add this Package.swift:

// swift-tools-version:5.5
import PackageDescription

let package = Package(
  name: "demo",
  products: [],
  dependencies: [
    .package(url: "https://github.com/foxglove/mcap", branch: "main"),
  ],
  targets: []
)

Then run swift build:

$ swift build
Fetching https://github.com/foxglove/mcap from cache
Fetched https://github.com/foxglove/mcap (2.03s)
Fetching https://github.com/apple/swift-collections from cache
Fetched https://github.com/apple/swift-collections (0.76s)
Fetching https://github.com/apple/swift-algorithms from cache
Fetched https://github.com/apple/swift-algorithms (0.33s)
Computing version for https://github.com/apple/swift-algorithms
Computed https://github.com/apple/swift-algorithms at 1.0.0 (0.32s)
Fetching https://github.com/apple/swift-numerics from cache
Fetched https://github.com/apple/swift-numerics (0.60s)
Computing version for https://github.com/apple/swift-numerics
Computed https://github.com/apple/swift-numerics at 1.0.2 (0.33s)
Creating working copy for https://github.com/apple/swift-algorithms
Working copy of https://github.com/apple/swift-algorithms resolved at 1.0.0
Creating working copy for https://github.com/apple/swift-collections
Working copy of https://github.com/apple/swift-collections resolved at 418378107c87a4b312e29a51f773ce0e4e12e199
Creating working copy for https://github.com/apple/swift-numerics
Working copy of https://github.com/apple/swift-numerics resolved at 1.0.2
Creating working copy for https://github.com/foxglove/mcap
error: Couldn’t check out revision ‘14fa18afb83e504ae03397fd4e9292a6c83a7e80’:
    Downloading docs/img/0398582-hero.png (669 KB)
    Error downloading object: docs/img/0398582-hero.png (266f9d3): Smudge error: Error downloading docs/img/0398582-hero.png (266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5): error transferring "266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5": [0] remote missing object 266f9d3dd91c1bd739e1d79ae4abd459f71e8d4c661656745504ea52913467b5
    Errors logged to /Users/jacob/Desktop/swifttest/.build/checkouts/mcap/.git/lfs/logs/20221006T101212.79192.log
    Use `git lfs logs last` to view the log.
    error: external filter 'git-lfs filter-process' failed
    fatal: docs/img/0398582-hero.png: smudge filter lfs failed

[neonichu]

Interesting, this does not happen for me. I ended up with the same commit as you, but nothing in .git/lfs:

❯ git show --oneline -s
14fa18afb (HEAD, origin/main, origin/HEAD, main) fix link in README (#636)

/tmp/foo/.build/checkouts/mcap heads/main
❯ ls -ld .git/lfs
".git/lfs": No such file or directory (os error 2)

I'm not really familiar with git-lfs, is there any global config required for it?

[jtbandes]

I'm not really familiar with git-lfs, is there any global config required for it?

Yes, normally users will run git lfs install once, as described here: https://git-lfs.github.com

It's a very common thing to have installed, since many projects use Git LFS.

What do you see when you look at /tmp/foo/.build/checkouts/mcap/docs/img/0398582-hero.png? I'm guessing you have only the LFS pointer file, which is what you'd get if you don't have LFS installed.

[jtbandes]

Also note that some folks on Discourse have dug deeper into why this happens: https://forums.swift.org/t/swiftpm-with-git-lfs/42396/13

[neonichu]

Thanks, this helps!

[jtbandes]

Thanks for looking into this! 🙌 Just wondering if there’s been any update so far on a possible resolution?

[jtbandes]

I tried adding a .lfsconfig file to the repo, as recommended in the Discourse thread, and this seems to have worked around the issue for swift build (like the small example I gave you), but it still doesn't work when adding the dependency in Xcode...

Example project that fails to build in Xcode on my machine: TestPackage.zip

[briancordanyoung]

Based on this thread and the forum discussion, I came to the same blocking errors.

Linking git-lfs did work in the sense that Xcode used it but reported a different error downloading various files that were checked in with lfs. The error included "Smudge error: Error downloading..."

> cd /Applications/Xcode-14.1.0.app/Contents/Developer
> ln -s /opt/homebrew/bin/git-lfs git-lfs

Adding a .lfsconfig (with an updated path for my repo) did not help with me when running swift build or an Xcode package resolution. I did have some GitHub authentication errors until I removed the commit that added the file.

The only work around I have is to manually clone the repo and include it as a local package—which of course negates all advantages of using a dependency manager. 😫

I did file FB11803523 for Xcode.

[jtbandes]

Not sure if it helps you, but I believe the .lfsconfig file has to be present on the repo's default branch. When SPM clones the package it starts with the default branch and then checks out whichever branch/commit you requested.

This did ultimately end up working well enough for me, I was able to use xcodebuild in CI (but not the Xcode app): https://github.com/foxglove/MCAPSpotlightImporter/actions/runs/3380647041/jobs/5613664270

However, the .lfsconfig file is causing some other issues for developers working with the repo... 😓

[iosdevben]

Is there any update to this?

[twprzybysz]

+1

[nicolascabaret]

+1

[scosman]

One option for this I wrote up here (which only applies if the LFS files are something optional for your package, like test results): https://stackoverflow.com/questions/67457594/swift-package-manager-does-not-work-with-git-lfs/77336212#77336212

[koliyo]

The workaround from @briancordanyoung seem to work, but the Xcode path has changed

cd /Applications/Xcode.app/Contents/Developer/usr/bin
ln -s /opt/homebrew/bin/git-lfs git-lfs

[yangkx1024]

Seems the workaround conflict with SIP, it's weird to include XCode folder in SIP.

[koliyo]

Are you sure it is SIP @yangkx1024? Wonder why the symlinking was allowed for me, I have not disabled SIP

csrutil status


System Integrity Protection status: enabled.

[yangkx1024]

I disabled the SIP and the ln command success, but now I stuck with the git-lfs issue, same as this post: https://forums.swift.org/t/swiftpm-with-git-lfs/42396/13

[jtbandes]

@neonichu if you learned anything about the root cause of this issue or possible workaround, I’m sure many of us would be interested to hear it, thanks :)

[dipidoo]

I am using the following workaround:

• Switch to using your system Git.

  • When you encounter error that says "Package Resolution Failed", dismiss with "Add Anyway". This will add a partially configured package dependency.

  • If Xcode Command Line Tools is never installed on your machine, install it first following instructions from Apple Developer website.

  • Run the following command from Terminal, from the directory where your .xcodeproj is located, as appropriate for your project. It will resolve the package through your system Git. Your system Git needs to already have Git LFS configured.

    xcodebuild -scmProvider system -resolvePackageDependencies
    

    Note: You can also add -project, -workspace, and -scheme arguments as appropriate to target your development setup. Refer to the xcodebuild (Command Line Tools) FAQ and -help section.

[iosdevben]

Can we get an update on this please? It's been open for 6 years but seems like a critical barrier preventing 100% adoption of Swift PM.

[briancordanyoung]

@iosdevben I would love to have this fixed. But I would not let it prevent you from adopting SPM. I've been using this work around all this time and the benefits of SPM have made it completely worth it.