Frontend: Ignore resilient binary swiftmodules under usr/lib/swift

[xymus]

Most SDKs use only swiftinterfaces under usr/lib/swift. Let's standardize this behavior and use only swiftinterface when they are present, even if there are binary swiftmodule files available next to them.

Preserve the old behavior for the stdlib for faster build-times.

Apply the same logic to SubFrameworks as well while we're at it.

rdar://145316821

[xymus]

@swift-ci Please smoke test

[nkcsgexi]

🥳

[xymus]

@swift-ci Please smoke test

[weliveindetail]

Hi @xymus, bisect shows that this PR broke the Android build with the Windows toolchain. This is coming late, because we don't have test coverage for it in mainline CI yet. However, I can reproduce the issue with the PR linked below.

Any ideas how to fix this? Does your change apply on Windows? Are binary swiftmodules resilient there? Let me know if you can recommend anything to try and I will look into it tomorrow. Thx

Here's the error:

T:/x64/Android.platform/Developer/SDKs/Android.sdk\usr\lib\swift\android\_math.swiftmodule\x86_64-unknown-linux-android.private.swiftinterface:6:19: error: underlying Objective-C module '_math' not found
 4 | // swift-module-flags-ignorable: -enable-lexical-lifetimes=false -enable-ossa-modules -interface-compiler-version 6.2
 5 | import Swift
 6 | @_exported import _math
   |                   `- error: underlying Objective-C module '_math' not found
 7 | @available(swift, deprecated: 3.0, message: "Please use 'Double.pi' or '.pi' to get the value of correct type and avoid casting.")
 8 | public let M_PI: Swift.Double

T:/x64/Android.platform/Developer/SDKs/Android.sdk\usr\lib\swift\android\_math.swiftmodule\x86_64-unknown-linux-android.private.swiftinterface:1:1: error: failed to build module '_math' for importation due to the errors above; the textual interface may be broken by project issues or a compiler bug
 1 | // swift-interface-format-version: 1.0
   | `- error: failed to build module '_math' for importation due to the errors above; the textual interface may be broken by project issues or a compiler bug
 2 | // swift-compiler-version: Swift version 6.2-dev effective-5.10 (LLVM 1b5a84a64783fb8, Swift ab36a3aab1c59b4)
 3 | // swift-module-flags: -target x86_64-unknown-linux-android -disable-objc-interop -autolink-force-load -enable-experimental-feature NoncopyableGenerics2 -enable-experimental-feature SuppressedAssociatedTypes -enable-experimental-feature SE427NoInferenceOnExtension -enable-experimental-feature NonescapableTypes -enable-experimental-feature LifetimeDependence -enable-upcoming-feature MemberImportVisibility -enable-library-evolution -module-link-name swift_math -swift-version 5 -O -library-level api -enforce-exclusivity=unchecked -disable-objc-interop -target-min-inlining-version min -module-name _math

T:/x64/Android.platform/Developer/SDKs/Android.sdk\usr\lib\swift\android\Android.swiftmodule\x86_64-unknown-linux-android.private.swiftinterface:6:19: error: no such module 'SwiftAndroid'
  4 | // swift-module-flags-ignorable: -enable-lexical-lifetimes=false -enable-ossa-modules -interface-compiler-version 6.2
  5 | import Swift
  6 | @_exported import SwiftAndroid
    |                   `- error: no such module 'SwiftAndroid'
  7 | import SwiftOverlayShims
  8 | import SwiftShims

<module-includes>:1:10: note: in file included from <module-includes>:1:
1 | #include "LibcOverlayShims.h"
  |          `- note: in file included from <module-includes>:1:
2 | 

T:/x64/Android.platform/Developer/SDKs/Android.sdk\usr\lib\swift\android\Android.swiftmodule\x86_64-unknown-linux-android.private.swiftinterface:6:19: error: no such module 'SwiftAndroid'
  4 | // swift-module-flags-ignorable: -enable-lexical-lifetimes=false -enable-ossa-modules -interface-compiler-version 6.2
  5 | import Swift
  6 | @_exported import SwiftAndroid
    |                   `- error: no such module 'SwiftAndroid'
  7 | import SwiftOverlayShims
  8 | import SwiftShims

<module-includes>:1:10: note: in file included from <module-includes>:1:
1 | #include "LibcOverlayShims.h"
  |          `- note: in file included from <module-includes>:1:
2 | 

T:/x64/Android.platform/Developer/SDKs/Android.sdk\usr\lib\swift\android\Synchronization.swiftmodule\x86_64-unknown-linux-android.private.swiftinterface:5:8: error: failed to build module 'Android' for importation due to the errors above; the textual interface may be broken by project issues or a compiler bug
   3 | // swift-module-flags: -target x86_64-unknown-linux-android -disable-objc-interop -enable-experimental-feature NoncopyableGenerics2 -enable-experimental-feature SuppressedAssociatedTypes -enable-experimental-feature SE427NoInferenceOnExtension -enable-experimental-feature NonescapableTypes -enable-experimental-feature LifetimeDependence -enable-upcoming-feature MemberImportVisibility -enable-experimental-feature RawLayout -enable-experimental-feature StaticExclusiveOnly -enable-experimental-feature Extern -enable-library-evolution -module-link-name swiftSynchronization -swift-version 5 -O -library-level api -enable-builtin-module -enforce-exclusivity=unchecked -disable-objc-interop -target-min-inlining-version min -module-name Synchronization
   4 | // swift-module-flags-ignorable: -strict-memory-safety -enable-lexical-lifetimes=false -interface-compiler-version 6.2
   5 | import Android

     |        `- error: failed to build module 'Android' for importation due to the errors above; the textual interface may be broken by project issues or a compiler bug

   6 | import Builtin

   7 | import Swift



<module-includes>:1:10: note: in file included from <module-includes>:1:

1 | #include "LibcOverlayShims.h"

  |          `- note: in file included from <module-includes>:1:

2 | 

ninja: build stopped: subcommand failed.
Error: Error: cmake.exe exited with code 1.

[xymus]

It looks like these modules are resilient / library-evolution-enabled from the flags I can see in the logs. There’s a <unknown>:0: warning: module interfaces are only supported with -enable-library-evolution that points to a misconfig, it's worrying but I can't tell which module is affected.

This change applies to modules with swiftinterfaces under the SDK/usr/lib/swift/ path, it appears to match the Android configuration. I don’t think there’s a fundamental reason for this to not work on Windows/Android, we generate the core libraries as resilient on most platforms.

It looks like building these swiftinterfaces failed because of a missing search path. Search paths aren’t written in swiftinterfaces whereas they can be written in swiftmodules depending on the configuration. I would expect that adding them in the command building the client, or as implied by the SDK search path as we do for System/Library/Frameworks, would allow the swiftinterfaces to build fine.

I could as well add an exception to disable this change or the logic preferring swiftinterfaces on Windows. This logic has two goals, avoid issues with distributed swiftmodule files and make it more consistent how we import a module (always use the swiftinterface instead of using either the swiftmodule or the swiftinterface). I've seen these be more of a problem on Apple platforms.

[weliveindetail]

Thanks for the explanation. Search paths might be a good point. Revisiting the error message:

1. usr\lib\swift\android\_math.swiftmodule lacks the "underlying Objective-C module '_math'"
2. usr\lib\swift\android\Android.swiftmodule lacks the "SwiftAndroid" module, which is a Clang Module actually

I found both of these dependencies in the android.modulemap, which ends up in the usr/lib/swift/android/x86_64 subfolder of the SDK. Though, I cannot confirm yet. As soon as I add this as an additional include directory, my swiftc hangs for a few minutes and then dumps the same errors again.

Anyway, I guess this is the right way forward.

[compnerd]

This happens to impact both Windows and Android as both require additional modulemaps to be injected via the VFS. The VFS overlays are not preserved nor are they recreated and thus resulting in the failure to find these modules.