There are a number of functional components to Binding Tools for Swift and generally speaking, they are separate pieces. This document will describe each piece and how they fit together. Before going into deep details, I will talk about the biggest components and how and why they fit together.
Demangler: This component analyzes a source Swift library and maps entry points to mangled names. It consumes a source Swift library and createsTLDefinitionobjects that contain type, mangled name, demangled name, module name, and offset within a module.Swiftmodule parser: This component aggregates the public API. It takes in a.swiftmodulefile generated by the Swift compiler, similar to a C header file, and combines it with theTLDefinitionobjects to generate a module declaration.WrappingCompilerandOverrideBuilder: These components generate Swift wrappers for cases where direct P/Invoke from C# into Swift is not possible.WrappingCompiler: Compile the wrappers to a helper library.Demangler: Analyze the output wrapping library (inventory) and map entry points to mangled names (similar to the step 1).Swiftmodule parser: Reflect on output wrapping library and aggregate the public API (similar to the step 2).NewClassCompiler: This component generates C# bindings based on the generatedTLDefinitionand module declarations from steps 5 and 3. It should implement type mapping and register lowering. Key considerations include:- Splitting out tuples into individual parameters
- Determining the return buffer size using the value witness table for the struct type
- Managing enum/struct arguments through registers vs. passing by reference
XamGlue: This component is a library that provides basic support for Swift interop that the generated code builds on. It includes bindings for common built-in types like arrays and dictionaries. It provides a single source of truth for Swift types so that they can be exposed across an assembly boundary. This can be a NuGet package, possibly referencable by FrameworkReference or automatically included when targeting macOS, Mac Catalyst, iOS, or tvOS platforms that exposes the platform APIs for each .framework that is exposed from Swift to .NET.
Tom-Swifty: This component orchestrates other components and allows users to easily generate Swift projections for a given set of .frameworks. It presents an interface on top of the tool that is user-friendly and project-system-integrated.
The flow generally goes like this:
- source library (.swift) → Demangler → Inventory
- source .swiftmodule/library → Swiftmodule parser → ModuleDeclaration(s)
- Inventory/ModuleDeclarations → WrappingCompiler → wrapper source (.swift)
- Inventory/ModuleDeclarations → OverrideBuilder → wrapper source (.swift)
- wrapper source (.swift) → WrappingCompiler → wrapper library (.dylib)
- wrapper library (.dylib) → Demangler → wrapper Inventory
- wrapper .swiftmodule/library → Swiftmodule parser → wrapper ModuleDeclaration(s)
- (ModuleDeclarations, Inventory, wrapper ModuleDeclarations, wrapper Inventory) → NewClassCompiler → C# binding
The flow starts with the source Swift library, which is processed by the Demangler to generate TLDefinition objects. Then, the Swiftmodule parser aggregates the public API from .swiftmodule file and generates a module declaration. Both the TLDefinition objects and module declaration are then used as inputs for two distinct components: the WrappingCompiler and OverrideBuilder. These components generate Swift wrappers source code. The wrappers source code is subsequently processed again by the Demangler and Swiftmodule parser leading to the generation of wrapper library TLDefinition objects and its module declaration. Finally, TLDefinition and module declarations, both for the source Swift library and Swift wrappers, are consumed by the NewClassCompiler to generate C# bindings.
Along the way, there are several things that happen as side effects of the overall. First, there is a type database which contains swift types and information associated with them:
- The kind of the type (class, struct, enum, etc)
- The name of the type including the module
- Whether or not it’s ObjC or swift
- The C# namespace and full type name
This database is built from either reading XML files or in the process reflecting on module declarations and writing the C# bindings.
Second there three compile-time marshaling engines that handle transitions from:
- C# → swift
- swift → “C safe” C#
- C safe C# → C#
Dynamo is a code generator that uses C# combinators to write source code. Dynamo is a somewhat leaky abstraction that models language structures similar to a compiler parse tree and handles writing out the code for you. This was meant as a step up from using Console.WriteLine. There are sub modules within Dynamo for writing swift source code and writing C# source code. Combinators let you use C# expressions to build up structures. For example, there is an abstract type SLBaseExpr for which many operators exist. The C# expression someBaseExpr + someOtherBaseExpr generates an SLBinaryExpr with a + operator.
Generally speaking, the types are immutable except where they are or contain collections, which are mutable. In this way, you can easily, say, make a method and add to its contents or parameters, but not modify its visibility, return type, etc.
Dynamo is used in wrapping and binding.
This is one of the oldest components in the project. I had hoped initially that I could use it to represent all of the swift data types and entry points. This turned out to be not true. Still, each type in the hierarchy represents all the majors types in swift. The types in this hierarchy are meant to be immutable.
The entry points in a swift library are mangled ASCII strings. The swift name mangling scheme has changed several times since swift 3.0. In swift 3.0, it was a small prefix based language. In swift 4.0, it changed to a postfix based language. swift 5.0 is similar to swift 4.0 with some changes. All of the demanglers generates SwiftType objects and bind them into a TLDefinition. A TLDefinition may be a function or global data etc.
The inventory hierarchy is a set of pairs of classes. An FooInventory contains zero or more FooContents. A FooContents is all the elements that make up a Foo. For example, a ModuleInventory contains one or more ModuleContents. A ModuleContents contains a ClassInventory, a FunctionInventory, a VariableInventory and so on.
In general the inventories operate like a dynamic pachinko machine. If you drop a TLDefinition which represents a method on a class onto the top ModuleInventory, it will dispatch it to a ModuleContents which will in turn find or make the appropriate ClassInventory which will find or create the appropriate ClassContents and drop it onto this, which will in turn put it into a FunctionInventory, which will then drop it into FunctionContents which will in turn go into an OverloadInventory and finally into an OverloadContents.
After all the public TLDefinition objects are dispensed, there is a more or less complete representation of all the types and their public entry points in the inventory. Unfortunately, this was not complete because there is a pile of missing information, including attributes, struct layout, etc. The inventory still proved to be useful, however, so it wasn’t scrapped.
In order to interoperate with ObjC bindings generated for Xamarin.iOS/mac/tvos/watchos, we need a way to map swift types to existing C# bindings. In order to do this, we rip through all the C# types and build up type database entries from them.
This component is deprecated. Swiftmodule parser is used instead.
The reflector is a modified version of the swift compiler that consumes swift libraries and writes out an XML representation of the public facing API. This is necessary in order to capture all the details and information associated with an API. The swift compiler provides a visitor pattern that allows virtual methods to get called for each of the nodes in the parse tree of the module. Based on this we can output XML depending on the node that we’re in.
This component is deprecated. Swiftmodule parser is used instead.
This hierarchy is very similar to SwiftType, but has methods/properties to represent information that isn’t present in the mangled signatures. In addition, there is another type thrown in called TypeSpec. When swift generates type information for a given return value or parameter type, it uses a little language to represent the type. There are roughly 4 types that are represented in this: named types, tuples, closures, and protocol lists. The little language encodes all of this. TypeSpecParser is a simple recursive descent parser that consumes the little language and generates one of the TypeSpec types representing it.
TopLevelFunctionCompiler is a set of tools that given a FunctionDeclaration can generate a C# method signature, property signature, or a delegate declaration. This used by NewClassCompiler to generate public facing API, virtual callbacks, or pinvoke definitions.
Given a public swift API (function, class, struct, extension, etc), MethodWrapping generates swift code that can be called from a pinvoke in C#
Given an open swift class or a protocol, OverrideBuilder generates either an override of the type, overriding all the virtual types and delegating them to a vtable into C#, or it generates a set of extensions on the type EveryProtocol implementing all the types in the protocol and delegating them to a vtable into C#.
The TypeMapping namespace contains classes to map types from one type to another as well as maintaining the TypeDatabase. TypeMapper maps from swift types to C# types. SwiftTypeToSLType maps from the SwiftType hierarchy to the Dynamo SLType hierarchy. There is a similar one to map from TypeSpec objects to the SLType hierarchy. The latter two get bundled into the TypeMapper object so that all three are easily accessible.
This is a chunk of code that gets used to compile swift source code into libraries and swift module files. Given source files and dependencies, it figures out how to tell the compiler about all the necessary references to compile the source files correctly. It also handles multiple target platforms and merging the output into a far framework.
NewClassCompiler handles generating the C# bindings onto the types from the swift module. It orchestrates the other components into writing wrappers and then finally writing the actual C# bindings. It is a huge file. Yes, I know this. More than anything else, this reflects the complexity of representing swift in C# as well the all the special cases involved in marshaling. If you’re looking to trace through it, a good place to start is CompileModuleContents at the highest level. If you’re looking to catch a particular type, there are methods named Compile{Classes,Structs,Enums,Extensions,etc.} which do what they say on the box. Classes handle virtual and non-virtual cases separately.
Binding Tools for Swift is heavily unit tested. The general pattern that is used is a test contains a string which is swift code. It also contains some Dynamo combinators to build the code that will use the Binding Tools for Swift binding. The call TestRunning.TestAndExecute compiled the swift code into a library then runs Binding Tools for Swift on it to generate wrapping and binding. The Dynamo combinators get written into a C# file and then the whole thing gets compiled and run using mono. The output gets collected and is compared to the assert.
In addition, if there is no platform specified, the swift source code and the Dynamo code will get aggregated into code appropriate for running on a device or simulator.
Notes:
- In any given test namespace, tests swift source should have unique names for classes, functions, etc. Name conflicts will fail the device test builds
- Avoid generating output from swift using
print. It’s a real pain to capture and redirect the output to C# from a device. There are very small number of tests that do this and they feel flimsy as a result. In addition, swift IO routines buffer the output such that if you print something in swift and then in C#, you will see the C# output first. Prefer tests that return a value to C# and let C# print the result.