-
Notifications
You must be signed in to change notification settings - Fork 1.7k
WASM FFI #37882
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
This isn't linked in to anything yet, but I've verified that it works. Bug: #37882 Change-Id: Ib65d7f11aacac1a20d93fc5d4f10ced19ecb1e51 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/113263 Reviewed-by: Ryan Macnak <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
Does wasmer have support for interface types, like wasmtime? https://hacks.mozilla.org/2019/08/webassembly-interface-types/ |
I don't think so, but it sounds like this is an early stage proposal, so we probably shouldn't rely on it yet. I investigated both wasmtime and wasmer (as well as some other stuff), and the reason I chose wasmer is that it's way easier to integrate with C++. I'm still just prototyping though. |
This CL doesn't have any tests because it's just boilerplate. I'll add a test in the follow up CLs where I add actual functionality. Bug: #37882 Change-Id: I47c81f5f1be724f8226e756ba5d01880a45f1ac7 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/112841 Reviewed-by: Siva Annamalai <[email protected]> Reviewed-by: Liam Appelbe <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
This reverts commit f39a3f1. Reason for revert: https://golem.corp.goog/BuildInfo?target=flutter-profile&machine-type=android-armv7&revision=84750 Original change's description: > Scaffolding for dart:wasm > > This CL doesn't have any tests because it's just boilerplate. I'll > add a test in the follow up CLs where I add actual functionality. > > Bug: #37882 > Change-Id: I47c81f5f1be724f8226e756ba5d01880a45f1ac7 > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/112841 > Reviewed-by: Siva Annamalai <[email protected]> > Reviewed-by: Liam Appelbe <[email protected]> > Commit-Queue: Liam Appelbe <[email protected]> [email protected],[email protected] Change-Id: I0fd0f29d66a07fc29e840ddaec2d4161c8d599cb No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: #37882 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/114044 Reviewed-by: Liam Appelbe <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
Super excited to see this going! Let us know if we can be of any help :) |
how about these? Standardizing WASI: A system interface to run WebAssembly outside the web: https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/ WebAssembly Interface Types: Interoperate with All the Things!: https://hacks.mozilla.org/2019/08/webassembly-interface-types/ |
This reverts commit 9198813. Reason for revert: Relanding with a fix Original change's description: > Revert "Scaffolding for dart:wasm" > > This reverts commit f39a3f1. > > Reason for revert: https://golem.corp.goog/BuildInfo?target=flutter-profile&machine-type=android-armv7&revision=84750 > > Original change's description: > > Scaffolding for dart:wasm > > > > This CL doesn't have any tests because it's just boilerplate. I'll > > add a test in the follow up CLs where I add actual functionality. > > > > Bug: #37882 > > Change-Id: I47c81f5f1be724f8226e756ba5d01880a45f1ac7 > > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/112841 > > Reviewed-by: Siva Annamalai <[email protected]> > > Reviewed-by: Liam Appelbe <[email protected]> > > Commit-Queue: Liam Appelbe <[email protected]> > > [email protected],[email protected] > > Change-Id: I0fd0f29d66a07fc29e840ddaec2d4161c8d599cb > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Bug: #37882 > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/114044 > Reviewed-by: Liam Appelbe <[email protected]> > Commit-Queue: Liam Appelbe <[email protected]> [email protected],[email protected] # Not skipping CQ checks because original CL landed > 1 day ago. Bug: #37882 Change-Id: Idb43cbd3a0521776ac420bfef91c8a9a4362f18e Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/114757 Reviewed-by: Liam Appelbe <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
Currently this only supports functions that take and return numeric types. Byte arrays, and callbacks will come later, in a separate wrapper package. Bug: #37882 Change-Id: I7bb82be83cbbb6062736b3e958f89d021f1af4bb Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/114586 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Samir Jindel <[email protected]> Reviewed-by: Ryan Macnak <[email protected]>
Follow up to https://dart-review.googlesource.com/c/sdk/+/117210, which was a quick fix to get the bots green. Since I've cut the FFI dependency for the native type markers, wasm.cc needs to switch to using these new classes, and we need boilerplate to associate them with some CIDs. Bug: #37882 Change-Id: Ic64de8fd08b8a84476aeaacae3f57af3f278b8c9 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117292 Reviewed-by: Alexander Markov <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
These classes just thinly wrap wasmer objects, so we don't need them. Bug: #37882 Change-Id: I06d715cc823c48101da3730ee68dd11adf5eb523 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117420 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Alexander Markov <[email protected]>
This involved refactoring so that all the throws happen in the top level native entry functions, and then making sure that all the native objects are accounted for in those locations. I ran the wasm tests under asan and verified they used to leak but now don't. Bug: #37882 Change-Id: I9991ff6b9f8af1fa2f335c73ecf11aba29ea8c72 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/117443 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Ryan Macnak <[email protected]>
Bug: #37882 Change-Id: Idee2f5383516e62ff3c228968c105835fcd954f3 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118200 Reviewed-by: Ryan Macnak <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
Fix some VM/native transition issues, and add a describe() function that prints all the imports and exports of a module. Bug: #37882 Change-Id: I68a8860857c3fcb37c49a9e92731629a13fd9449 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118363 Reviewed-by: Ryan Macnak <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
Bug: #37882 Change-Id: Iceccc5805bcdadf003d8e753c7f86f915d72bf39 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118662 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Alexander Markov <[email protected]>
Bug: #37882 Change-Id: Ia8aa9a87803d5e8b899ddab5479b8cd93e03fd71 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118204 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Ryan Macnak <[email protected]> Reviewed-by: Alexander Markov <[email protected]>
This reverts commit 7a24aec. Reason for revert: debug-mode gen_snapshot crashes on all programs Original change's description: > [vm] Wasm function imports > > Bug: #37882 > Change-Id: Ia8aa9a87803d5e8b899ddab5479b8cd93e03fd71 > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118204 > Commit-Queue: Liam Appelbe <[email protected]> > Reviewed-by: Ryan Macnak <[email protected]> > Reviewed-by: Alexander Markov <[email protected]> [email protected],[email protected],[email protected] Change-Id: Ib5d348e25854af010b1d8a64a87406da443981df No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: #37882 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119143 Reviewed-by: Samir Jindel <[email protected]> Commit-Queue: Samir Jindel <[email protected]>
Bug: #37882 Change-Id: I8f6c3f9b8d69a43ef6382a3d07579ea14d335c35 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119173 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Alexander Markov <[email protected]>
This reverts commit a7cb8ee. Reason for revert: Fixed the crash Original change's description: > Revert "[vm] Wasm function imports" > > This reverts commit 7a24aec. > > Reason for revert: debug-mode gen_snapshot crashes on all programs > > Original change's description: > > [vm] Wasm function imports > > > > Bug: #37882 > > Change-Id: Ia8aa9a87803d5e8b899ddab5479b8cd93e03fd71 > > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118204 > > Commit-Queue: Liam Appelbe <[email protected]> > > Reviewed-by: Ryan Macnak <[email protected]> > > Reviewed-by: Alexander Markov <[email protected]> > > [email protected],[email protected],[email protected] > > Change-Id: Ib5d348e25854af010b1d8a64a87406da443981df > No-Presubmit: true > No-Tree-Checks: true > No-Try: true > Bug: #37882 > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119143 > Reviewed-by: Samir Jindel <[email protected]> > Commit-Queue: Samir Jindel <[email protected]> [email protected],[email protected],[email protected],[email protected] Change-Id: I4fd073a087eae7bb95c75079446d9111e7bc8c40 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: #37882 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119168 Reviewed-by: Liam Appelbe <[email protected]> Reviewed-by: Alexander Markov <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
Bug: #37882 Change-Id: I3044d588398ccb19e3ebcebe6cd506b047a37857 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/118601 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Alexander Markov <[email protected]>
I got the hello world wasm file from the emscripten team and got it running using a fake implementation of WASI's fd_write function. This necessitated adding support for memory exports. Bug: #37882 Change-Id: I139a4e868d437e2232bf4260e5cc26d8c598ac2c Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/119061 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Alexander Markov <[email protected]>
For everybody tracking this issue. We are currently evaluating how to move multiplatform Dart<->WASM interop forward. Similarly to our FFI work we would like to collect some examples of code that people would like to use in their Dart applications via WASM. So if you have one that is open source please leave a comment here or on #37355 with a link and which platforms you would like to use it on (Flutter on iOS/Android/Desktop/Web, Dart Web), please also include information on whether you have tried to compile this library to WASM before. Thank you. |
@mraleph Thats great news! |
@maks Great to know! I have a quick question: does your Flutter app run on Web? If the app is only on non-Web platforms (Android, iOS, ...), it seems Dart FFI can support your use cases. |
@kaikai4989 well yes I could use FFI, but that's the reason I want to use WASM is as I said to avoid having to build the native lib to use with FFI for android (NDK), iOS, Linux, MacOS, etc |
In addition to the portability aspect, WASM brings sandboxing/security guarantees as well as a fully managed memory model that you don't get with FFI. WASM is simply the better, more secure, and more future-proof option for client-side shared libraries. |
So far this just builds the wasmer library, copies it into the sdk directory, loads the library, and allows you to compile WASM modules. You can't actually do anything with the modules yet. Bug: #37882 Change-Id: I7d7cfe5721bbe38a6afe76f326518e714d236ed4 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158367 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Ryan Macnak <[email protected]>
This reverts commit 2bafc32. Reason for revert: Build failures on Flutter HHH CI bots and golem (Dart VM ARMv8, Flutter). Original change's description: > [vm/wasm] Boilerplate for package:wasm > > So far this just builds the wasmer library, copies it into the sdk > directory, loads the library, and allows you to compile WASM modules. > You can't actually do anything with the modules yet. > > Bug: #37882 > Change-Id: I7d7cfe5721bbe38a6afe76f326518e714d236ed4 > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158367 > Commit-Queue: Liam Appelbe <[email protected]> > Reviewed-by: Ryan Macnak <[email protected]> [email protected],[email protected],[email protected] Change-Id: I2c5b390987bf45704a3de98e050f4bcf1cb235d8 No-Presubmit: true No-Tree-Checks: true No-Try: true Bug: #37882 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/159243 Reviewed-by: Tess Strickland <[email protected]> Commit-Queue: Tess Strickland <[email protected]>
This is a reland of 2bafc32 Original change's description: > [vm/wasm] Boilerplate for package:wasm > > So far this just builds the wasmer library, copies it into the sdk > directory, loads the library, and allows you to compile WASM modules. > You can't actually do anything with the modules yet. > > Bug: #37882 > Change-Id: I7d7cfe5721bbe38a6afe76f326518e714d236ed4 > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/158367 > Commit-Queue: Liam Appelbe <[email protected]> > Reviewed-by: Ryan Macnak <[email protected]> Bug: #37882 Change-Id: I8056df1e301acde2772ba2273148faa53d03173e Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/159321 Reviewed-by: Ryan Macnak <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
Add support for instantiating modules, getting their exported functions, and calling those functions. Function calling is tricky to implement, because we need to put the args in an array to pass to wasm, and Dart doesn't have variadic functions. So I implemented it by overriding noSuchMethod. Users can call WasmFunction like a regular function, and the args will be dynamically type checked and put in the array to pass to wasm. WasmImports is a placeholder for now. Also, it's not possible to get the function names until FFI supports passing structs by value, so for now you can only get functions by their export index. If by-value structs are delayed, I can add an intermediate C++ layer to the wasmer library work around this. Bug: #37882 Change-Id: I1214c96df324cfc9fc02a48bc09a269da1b8c6f6 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160144 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Ryan Macnak <[email protected]>
Since FFI doesn't support structs by value yet, this thin wrapper library just provides versions of the import/export name functions that return the name by pointer. Also, I've used these functions to allow looking up functions by name, and added a function that prints a module's imports and exports (which is handy for debugging). Change-Id: Iff386e0b843bd8ab3763c99d3dc445ffedb12d6a BUG: #37882 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/161765 Reviewed-by: Ryan Macnak <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
The rest of the tests rely of features I haven't implemented yet. I'll port them as I implement them. Bug: #37882 Change-Id: I3465f10055db8d82148004ea975c0ed3b2928178 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162780 Reviewed-by: Ryan Macnak <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
Also, port the old wasm memory tests, and fix some NNBD issues. Bug: #37882 Change-Id: I131ba5836bb0a3dd946cf9b0fa3f2e186b6b132e Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/162801 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Ryan Macnak <[email protected]>
Broadly speaking, the new API is very similar to the old one. The only public facing change is that you can't make a memory by itself anymore. You need to construct if from a store, which is owned by a module. The new API no longer passes structs by value, so I was able to delete the wasmer_wrapper.cc, and the files I had copied from the wasmer repo. The new API is also a lot more verbose, so it made more sense to write a script to generate the FFI boilerplate, rather that write it all manually. All the migrated tests that were passing before this refactor are still passing. Bug: #37882 Change-Id: I52ca54b982a27f1d851c630d3e09833b8810060c Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/164081 Reviewed-by: Siva Annamalai <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
I switched from a WasmImports object to a builder pattern: module.instantiate().addMemory(...).addFunction(...).build(); Bug: #37882 Change-Id: I381aa0f7df1fa006ce8d051cd5b4a1bcc1835e46 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/167460 Reviewed-by: Ryan Macnak <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
Enabling WASI when building an instance sets up the default WASI imports, and can optionally capture stdout/stderr. I'd like to set up something similar for stdin, but Wasmer doesn't support that yet. I also improved error handling in WasmRuntime by adding Wasmer's last error to the thrown exceptions where appropriate. Bug: #37882 Change-Id: I2d8546e878bcb43c7093490eac085aa62d318e96 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/167904 Reviewed-by: Ryan Macnak <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
The Dart exception is converted to a string, stored in a trap which is passed through wasm, then converted back to a Dart exception. Module instantiation can also throw a trap, because it internally invokes the initialization function, if the module defines one. So I'm handling this too. Also, update to the new version of the wasm C API, which replaces some foo_t** with foo_vec_t*. Bug: #37882 Change-Id: I1e6952a191134734a892c42b2fb8ba3506bb0844 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/169784 Reviewed-by: Ryan Macnak <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
The finalizers associate each native object with a Dart object that owns them. The finalizers need to be in native code, so I had to go back to the old approach of linking Wasmer and my native code into a single .so. TEST=I tried running the wasm tests using asan, but it didn't detect any leaks even before the finalizers. So I put printfs in all the set_finalizer and finalizer functions and manually verified that all the finalizers were being run. Bug: #37882 Change-Id: I38303ae0c6c446a23562c1b157c92ccc9af2dc34 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/172680 Reviewed-by: Daco Harkes <[email protected]> Commit-Queue: Liam Appelbe <[email protected]>
Bug: #37882 Change-Id: Ia1a268aeb08f8775573287a025b91182ca621766 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/179062 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Siva Annamalai <[email protected]>
This replaces third_party/wasmer and the related build rules, so I'll delete that stuff once the package is published. Bug: #37882 Change-Id: I33728e42c734bc8c25a2d32522112f2a3dbf4384 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/180446 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Siva Annamalai <[email protected]>
Bug: #37882 Change-Id: Ibdc64d594ac1645a566477e083d79fe23b0e3ee5 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181060 Commit-Queue: Liam Appelbe <[email protected]> Reviewed-by: Daco Harkes <[email protected]>
Why is pubspec for wasm package locked for 2.10.0 sdk: '>=2.10.0-0 <2.10.0'? |
No good reason. I'll remove that restriction before I publish it. |
@liamappelbe do you have a planned publish date for wasm? I really need as i am currently using it wasm_introp (uses JS interop) which i have updated to be null safety but i can't use it side by side with ffi. Am i correct in thinking this wasm implementation is compatible with ffi. So if we have a plugin that use ffi (native) and wasm (web) then it will work fine. |
We plan to publish soon, but it's an experimental package. The initial implementation is only for desktop platforms (windows/mac/linux), so no web support yet. I'm not quite sure what you mean by "compatible with ffi". package:wasm is built on FFI, so I guess you could use both Wasm and FFI in the same project. |
Would it be possible to try this package out now with some sort of dart runtime flag like we had before null-safety was released or is the only way to compile the Dart SDK locally? |
No runtime flags are required. We moved package:wasm to its own repo: https://github.com/dart-lang/wasm. We're publishing soon, but if you're really keen I suppose you could clone that repo. |
This package has been published, see: https://medium.com/dartlang/experimenting-with-dart-and-wasm-ef7f1c065577 |
Tracking bug for work on supporting WASM libraries in Dart.
The text was updated successfully, but these errors were encountered: