Skip to content

Add web OS and architectures #2108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Add web OS and architectures #2108

wants to merge 1 commit into from

Conversation

mosuem
Copy link
Member

@mosuem mosuem commented Mar 19, 2025

See the comment from @bkonyi at flutter/flutter#164094 (comment).

Having to work around the web platform by having a null OS and Architecture is awkward, regardless of whether the web is actually an OS or not.

For the architecture, I don't really have a use case for separating into js and wasm at the moment, but could imagine this becoming relevant when running dart compile wasm vs dart compile js (vs dart compile exe on native).


  • I’ve reviewed the contributor guide and applied the relevant portions to this PR.
Contribution guidelines:

Note that many Dart repos have a weekly cadence for reviewing PRs - please allow for some latency before initial review feedback.

@mosuem mosuem requested review from mkustermann and dcharkes and removed request for mkustermann March 19, 2025 12:24
Copy link

PR Health

Breaking changes ✔️
Package Change Current Version New Version Needed Version Looking good?
Changelog Entry ✔️
Package Changed Files

Changes to files need to be accounted for in their respective changelogs.

API leaks ✔️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package Leaked API symbols
License Headers ✔️
// Copyright (c) 2025, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
Files
no missing headers

All source files should start with a license header.

Unrelated files missing license headers
Files
pkgs/jni/lib/src/third_party/generated_bindings.dart
pkgs/objective_c/lib/src/ns_input_stream.dart

@dcharkes
Copy link
Collaborator

I don't believe this is the right solution.

We should have a WebConfig for JsAsset and WasmAsset. The WebConfig can then hold a targetLanguage enum with js and wasm for example.

Any code paths building data assets (or other non-code assets) should not have to provide a CodeConfig. And they should not have to provide an Architecture or OS. This is why we moved the targetOS to be inside the code config rather than in the config itself. @mkustermann asked for this during the prototype precisely for this reason.

We most probably didn't refactor the Data assets PR after this change though, so the target OS and target architecture might be carried around in places where they shouldn't be. If we aren't invoking a hook for code assets, we should not have an OS and architecture.

Can the Flutter code be refactored to not have to carry around architecture and OS?

If not, could you point me to why?

@coveralls
Copy link

coveralls commented Mar 19, 2025

Coverage Status

coverage: 86.676%. remained the same
when pulling ce36176 on addWebOSandArch
into 8663dfa on main.

@mosuem
Copy link
Member Author

mosuem commented Mar 19, 2025

Can the Flutter code be refactored to not have to carry around architecture and OS?

If not, could you point me to why?

In the Flutter code, we repeat a build for a list of architectures - so something like

for(final Architecture architecture in architectures) {
  build(...)
}

with web having a null architecture, this becomes

for(final Architecture? architecture in [...architectures, if(isWeb) null]) {
  build(...)
}

I am open to suggestions on how to refactor this any other way. So far, having a OS.web and Architecture.web sounds like the smoothest solution to me.

@dcharkes
Copy link
Collaborator

In the Flutter code, we repeat a build for a list of architectures - so something like

Future<DartHookResult> _runDartHooks({
  required Map<String, String> environmentDefines,
  required FlutterNativeAssetsBuildRunner buildRunner,
  required List<Architecture> architectures,
  required Uri projectUri,
  required FileSystem fileSystem,
  required OS? targetOS,
  required bool linkingEnabled,
  required bool codeAssetSupport,
  required bool dataAssetSupport,
  //TODO(mosum): Remove this parameter as soon as OS and Arch have web values.
  required bool isWeb,
}) async {

codeAssetSupport = false but required List<Architecture> architectures, and required OS? targetOS, seems very weird.

Can we package all parameters that are for both code assets and data assets in an object (defines, build runner, project uri, file system, linking enabled) and make two methods?

Or one method, but have a param AssetTypeConfig which either holds an empty FlutterDataConfig() or a FlutterCodeConfig(architecture, os)?

Let me just review the whole PR, then I have more context.

@MichealReed
Copy link
Contributor

MichealReed commented Mar 19, 2025

Nice! This would allow the native_toolchain_cmake package to build using Emscripten and add the outputs as data assets for loading via JS.

@mosuem
Copy link
Member Author

mosuem commented Mar 19, 2025

Stepping back from that PR, I think there is always a way to make this work. But the question is if adding a OS.web and Architecture.{wasm, js} would improve things even more.

@lrhn any opinion or knowledge of prior art?

@mosuem
Copy link
Member Author

mosuem commented Mar 19, 2025

Nice! This would allow the native_toolchain_cmake package to build using Emscripten and add the outputs as data assets for loading via JS.

Well, adding an OS.web and adding support for code assets in web builds are two different issues ;)

@MichealReed
Copy link
Contributor

MichealReed commented Mar 19, 2025

Well, adding an OS.web and adding support for code assets in web builds are two different issues ;)

At least for the near-term, for use with flutter, we would only need the produced wasm and js of a emscripten compiled library moved to the web build folder. This would at least allow us to build data assets. Through the html template, the library can be included and invoked.

Perhaps a similar approach could be considered where dart build web provides a html template that includes relevant js/wasm libraries (rather than trying to link it all into a fat library based on if js or wasm), which then frameworks could include where appropriate to make the calls available?

@lrhn
Copy link
Member

lrhn commented Mar 19, 2025

Opinion: Always.
I don't actually understand what you're doing, so I'll take a step back.

I was planning to ask what it means to be an OS in this context, but even more fundamentally:
Why does the OS type exist? What is its purpose?

The doc says /// An operating system the Dart VM runs on.
With that description, "web" does not fit. The VM doesn't run on the web.
Has meaning drifted since that comment was written? If so, to what?

The doc also doesn't answer my question. Why does this project need a representation of "operating systems" (whatever that may end up meaning) that Dart runs on?

Probably because it needs to make a distinction between them, and (in some cases) treat each distinguishable OS uniquely.
So will "web" fit into that?

The class exists in package:native_assets_cli and is exported by the default library and by code_assets.dart, so it's a user-facing class, which should mean that it is used in the APIs.

The class has no members, but has extension members: dylibFileName (ext),
executableFileName (ext), libraryFileName (ext), staticlibFileName (ext).

None of those look relevant to the web.

The type OS only appears in two roles in the API, as targetOS of a {Build,Hook,Link}Config and as os of a NativeCodeAsset.
If code assets are not supported on the web anyway, the latter does not seem relevant.
I don't know what the *Configs mean. Are they meaningful if OS is "web"? If not, there seem to be no role for OS.web to fill, and I'd argue that using null to represent "no useful OS" is a better choice.

(Haven't looked at architecture or how it's used, or intended to be used, but I'd ask the same questions: What problem does adding Architecture.js solve for the package it's in. The classes are not intended as a general representation of operating systems or CPU architectures that you can use for anything, they're solving a concrete problem in their own package, and shouldn't be used for anything else. I wouldn't add any values that do have a role in that. Then one can decide whether an OS.unknown value is better than null for usability. Most likely it's not if all the extension methods would throw on it.)

@dcharkes
Copy link
Collaborator

This PR not the right approach.

We don't want to add configuration meant for CodeAssets, the CodeConfig, to support web with wasm and js. Such configuration should be added in a different extension than package:code_assets, it should be in package:web_assets or alike.

Has meaning drifted since that comment was written? If so, to what?

It has not, so this is indeed the wrong place to add it.

I am very open to add support for web assets, but in the CodeConfig is not the right place.

Let's continue the design discussion for what config is needed for the web and wasm/js assets on the issues already tracking this:

I believe @mosuem's issue would be solved by flutter/flutter#164094 (comment), please let me know if it doesnt.

@mosuem
Copy link
Member Author

mosuem commented Mar 20, 2025

The doc says /// An operating system the Dart VM runs on. With that description, "web" does not fit. The VM doesn't run on the web. Has meaning drifted since that comment was written? If so, to what?

The type OS only appears in two roles in the API, as targetOS of a {Build,Hook,Link}Config and as os of a NativeCodeAsset.

That's a very good point - and why I think we should add the web as an OS and Architecture. The uses of these classes are not (only) to store where we run the build, but rather which platform we are targeting. And while we currently do not support code assets on the web, we are targeting the web with Dart/Flutter builds - which is why I keep running into issues with not having a target platform represented here.

If the issue is about placing those classes only in the code_assets API, I am fine with moving them somewhere more general, regardless of whether we require/provide them for code assets only (I can see benefits of also exposing the target OS for data assets, but that's a different discussion to be had).

@mosuem
Copy link
Member Author

mosuem commented Mar 20, 2025

A note: this is why Flutter does implement TargetPlatform.web_javascript.

As the use of these is quite widespread, perhaps implementing these OS and Architecture classes in a package, such as package:platform, might make sense, which can then be imported to Flutter and native_assets_cli.

@dcharkes
Copy link
Collaborator

The type OS only appears in two roles in the API, as targetOS of a {Build,Hook,Link}Config

They don't appear in {Build,Hook,Link}Config, it is the CodeConfig extension of those. We purposefully moved it there.

The uses of these classes are not (only) to store where we run the build, but rather which platform we are targeting.

Data assets may not depend on target architecture.

Any other asset types, Wasm, Jar, Js, should have their own config with architecture if they have a concept of architecture.

And while we currently do not support code assets on the web

And we wont support CodeAssets on the web. We will support compiling native code to WasmAsset.

Code assets are meant for dynamic and static libraries that run on a physical hardware architecture.

Wasm assets are meant for libraries that run the web assembly hardware abstraction. These are a different asset type.

I can see benefits of also exposing the target OS for data assets, but that's a different discussion to be had.

That should be added as a DataConfig.

Let's discuss in:

@mkustermann
Copy link
Member

Any code paths building data assets (or other non-code assets) should not have to provide a CodeConfig. And they should not have to provide an Architecture or OS.

I concur with @dcharkes on this one :)

There may come a time when we introduce new asset types for web (e.g. JavaScript or Wasm code). Once that time comes we can discuss how to modify the configuration to accommodate this and add e.g. emscripten compiler config, ...

For data assets this shouldn't be required atm.

(Also flutter's current asset mechanism (afaik) doesn't allow conditionally add icons/fonts/data assets based on architecture or OS, does it?)

@mosuem
Copy link
Member Author

mosuem commented Mar 21, 2025

(Also flutter's current asset mechanism (afaik) doesn't allow conditionally add icons/fonts/data assets based on architecture or OS, does it?)

This is an open issue flutter/flutter#96514.

After an offline discussion with @dcharkes, I think the bigger picture is that web should not be special-cased as a platform, regardless of whether or not to add it as an OS. For the Flutter PR kicking off this discussion, I found a solution I am quite happy with, abstracting over the target platform instead of using the OS, Architecture pair from dart-lang/native.

@MichealReed
Copy link
Contributor

MichealReed commented Mar 21, 2025

Any code paths building data assets (or other non-code assets) should not have to provide a CodeConfig. And they should not have to provide an Architecture or OS.

Some data assets may still need to be built based on the target platform, mainly generated content like headers

There may come a time when we introduce new asset types for web (e.g. JavaScript or Wasm code). Once that time comes we can discuss how to modify the configuration to accommodate this and add e.g. emscripten compiler config, ...

So this would mean that until this day comes we cannot trigger builds targeting web and would need to piggyback build web libraries or assets while targeting a different OS and include the web assets with all builds.

(Also flutter's current asset mechanism (afaik) doesn't allow conditionally add icons/fonts/data assets based on architecture or OS, does it?)

If the dart build hook controls the conditionals based on architecture or OS, flutter would not need to, it would simply use what is provided.

@mosuem
Copy link
Member Author

mosuem commented Mar 24, 2025

Just to clarify, web assets and conditional assets are definitely features we want to and hopefully will ship - this is more of an implementation detail question.

@mosuem
Copy link
Member Author

mosuem commented Apr 4, 2025

(Closing as solved differently, and the discussion about implementation details of web code assets is elsewhere)

@mosuem mosuem closed this Apr 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants