Skip to content

[native_assets_cli] Move BuildMode to [native_toolchain_c] #1800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Dec 10, 2024

Conversation

dcharkes
Copy link
Collaborator

@dcharkes dcharkes commented Dec 10, 2024

The first in a series of refactorings to address:

We align BuildMode with OptimizationLevel, it's configured by the hook author in the build hook. Likely, every package should ship with max optimization level and build mode set to release. Only while developing a package (when it is the root package or path-depsed in) would the build mode be set to debug.

This will require a manual roll into dartdev and flutter tools due to the removal of BuildMode from the native_assets_builder API.

The native assets builder will still emit the field in the JSON, until we can bump the native_assets_cli SDK constraint to 3.7 stable. Then only people with older versions of the native_assets_cli package but a newer SDK break.

@coveralls
Copy link

coveralls commented Dec 10, 2024

Coverage Status

coverage: 88.699% (-0.02%) from 88.719%
when pulling 3ab2ed8 on remove-buildmode
into d37df9c on main.

@@ -118,16 +99,13 @@ sealed class HookConfigBuilder {
required String packageName,
required OS targetOS,
required List<String> buildAssetTypes,
required BuildMode? buildMode,
// required BuildMode? buildMode,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this.

@@ -207,6 +186,10 @@ final class BuildConfigBuilder extends HookConfigBuilder {
json[_dependencyMetadataKey] = {
for (final key in metadata.keys) key: metadata[key]!.toJson(),
};
// TODO: Bump min-SDK constraint to 3.7 and remove once stable.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function should no longer take dryRun instead it should hardcode it to false in the json.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would break the tests in native_toolchain_c that check that no assets are built if dryRun is true. We can only remove such functionality once we can bump the Dart SDK constraint to 3.7 stable (assuming we get all the refactorings landed in both Dart and Flutter before the branchcut for 3.7).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code here is the builder code (i.e. code for Dart SDK & Flutter SDK - which no longer set this to true), no?

The package:native_toolchain_c is for hook writers, it can still read the dry run (possibly produced by older versions of Dart SDK & older versions of Flutter SDK).

Copy link
Collaborator Author

@dcharkes dcharkes Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The builder code is used to produce a config in the unit tests of native_toolchain_c (pkgs/native_toolchain_c/test/cbuilder/cbuilder_test.dart). Edit: It seems we don't have any expect that no asset is produced no dry run though, we should have had that. 🤓

@auto-submit auto-submit bot merged commit 00aae9e into main Dec 10, 2024
24 checks passed
@auto-submit auto-submit bot deleted the remove-buildmode branch December 10, 2024 20:44
@nmfisher
Copy link
Contributor

@dcharkes when building a Flutter plugin on Windows, we need to select the precompiled library that was built with a runtime library matching the Flutter application (i.e. we can't mix debug (/MDd) and release (/MD) versions).

Previously BuildMode was available via BuildConfig, which was correctly set to BuildMode.debug for flutter run and BuildMode.release for flutter run --release. After this PR, this is no longer available. Is there any way to retrieve the BuildMode for the Flutter application that initiated the build?

@dcharkes
Copy link
Collaborator Author

@dcharkes when building a Flutter plugin on Windows, we need to select the precompiled library that was built with a runtime library matching the Flutter application (i.e. we can't mix debug (/MDd) and release (/MD) versions).

Oh that's very interesting! Could you elaborate on why your dylib has to match the build mode of Flutter?

@nmfisher
Copy link
Contributor

@dcharkes when building a Flutter plugin on Windows, we need to select the precompiled library that was built with a runtime library matching the Flutter application (i.e. we can't mix debug (/MDd) and release (/MD) versions).

Oh that's very interesting! Could you elaborate on why your dylib has to match the build mode of Flutter?

Setting aside Windows runtime variants, the simplest example would be a Flutter application that:

  1. links against a library compiled with debug symbols/asserts/etc when run with flutter run, and
  2. links against a library compiled in release mode (no asserts/symbols/etc) when run with flutter run --release.

@dcharkes
Copy link
Collaborator Author

dcharkes commented Apr 16, 2025

a Flutter application that:

  1. links against a library compiled with debug symbols/asserts/etc when run with flutter run, and
  2. links against a library compiled in release mode (no asserts/symbols/etc) when run with flutter run --release.

This behavior makes sense for the root package (and path dependencies and packages in your pub workspace), the ones that you are developing yourself. However, for arbitrary packages in your dependencies you'd want the dylibs to be compiled in release mode. For example if you have sqlite3 or cronet dylibs somewhere in your dependencies. If those packages are stable, you don't want the asserts to be run. Those packages depending on a build-mode from Flutter would mean those would also be in debug mode in flutter run and be slow.

Because we only want to change the behavior of the root package, I believe a better approach would be to flip the BuildMode inside your hook/build.dart, and make sure you don't ship your app in debug mode.

Most likely if you're developing the Dart code of your app and not touching the native code you might want to also run your native code in release mode while developing with hot reload with flutter run. So tying it to the Flutter runtime mode is not necessarily what you want.

A second approach would be to use the user-defines in the workspace pubspec.yaml for defining the build mode. This would enable changing the build mode for your package even if you are not the root package. (Imagine you're using sqlite3 and you believe there's a bug in the C code, you can set the build mode for specifically that dylib to debug and leave cronet in release mode.)

Windows runtime variants

I don't think I have enough background information here. Why would you need the compile mode to be identical with Flutter here? Isn't it the same as for other native dependencies, you only want it in debug mode if you suspect you have an error with it.

Does this make sense? Maybe I don't fully understand your use-case or development workflow. Please elaborate if the above suggestions won't work.

edit:

i.e. we can't mix debug (/MDd) and release (/MD) versions

You mean you can't mix Flutter being in one mode and your dylib in the other mode? Or two of your own dylibs being in different modes?

@nmfisher
Copy link
Contributor

However, for arbitrary packages in your dependencies you'd want the dylibs to be compiled in release mode. For example if you have sqlite3 or cronet dylibs somewhere in your dependencies. If those packages are stable, you don't want the asserts to be run. Those packages depending on a build-mode from Flutter would mean those would also be in debug mode in flutter run and be slow.

But what if I do want them to be run? :) I was hoping this could be a package-level choice. Not all compile options need to affect speed, e.g. maybe we want to use exactly the same optimizations but strip symbols for flutter run --release and keep for flutter run. This helps debugging issues with package users.

A second approach would be to use the user-defines in the workspace pubspec.yaml for defining the build mode. This would enable changing the build mode for your package even if you are not the root package. (Imagine you're using sqlite3 and you believe there's a bug in the C code, you can set the build mode for specifically that dylib to debug and leave cronet in release mode.)

Yes, that's a viable alternative. Is this available yet? I think I may have asked about this recently.

I don't think I have enough background information here. Why would you need the compile mode to be identical with Flutter here? Isn't it the same as for other native dependencies, you only want it in debug mode if you suspect you have an error with it.

Does this make sense? Maybe I don't fully understand your use-case or development workflow. Please elaborate if the above suggestions won't work.

edit:

i.e. we can't mix debug (/MDd) and release (/MD) versions

You mean you can't mix Flutter being in one mode and your dylib in the other mode? Or two of your own dylibs being in different modes?

On Windows, a Flutter plugin is built as a DLL and can be linked against the debug (/MDd) or release (/MD) dynamic runtime. If the Flutter plugin itself also links some other precompiled static library (say sqlite.lib), the runtime flag has to match the flag used to compile the static library. The flag can be changed in the Flutter plugin CMakeLists.txt but I was looking for a way to do so automatically depending on whether flutter was invoked in debug/release mode. It's basically the same question as above.

@dcharkes
Copy link
Collaborator Author

However, for arbitrary packages in your dependencies you'd want the dylibs to be compiled in release mode. For example if you have sqlite3 or cronet dylibs somewhere in your dependencies. If those packages are stable, you don't want the asserts to be run. Those packages depending on a build-mode from Flutter would mean those would also be in debug mode in flutter run and be slow.

But what if I do want them to be run? :) I was hoping this could be a package-level choice. Not all compile options need to affect speed, e.g. maybe we want to use exactly the same optimizations but strip symbols for flutter run --release and keep for flutter run. This helps debugging issues with package users.

A second approach would be to use the user-defines in the workspace pubspec.yaml for defining the build mode. This would enable changing the build mode for your package even if you are not the root package. (Imagine you're using sqlite3 and you believe there's a bug in the C code, you can set the build mode for specifically that dylib to debug and leave cronet in release mode.)

Yes, that's a viable alternative. Is this available yet? I think I may have asked about this recently.

Yes, in the very latest release on the master branch of flutter and the latest release of package:native_assets_cli.

I don't think I have enough background information here. Why would you need the compile mode to be identical with Flutter here? Isn't it the same as for other native dependencies, you only want it in debug mode if you suspect you have an error with it.
Does this make sense? Maybe I don't fully understand your use-case or development workflow. Please elaborate if the above suggestions won't work.
edit:

i.e. we can't mix debug (/MDd) and release (/MD) versions

You mean you can't mix Flutter being in one mode and your dylib in the other mode? Or two of your own dylibs being in different modes?

On Windows, a Flutter plugin is built as a DLL and can be linked against the debug (/MDd) or release (/MD) dynamic runtime. If the Flutter plugin itself also links some other precompiled static library (say sqlite.lib), the runtime flag has to match the flag used to compile the static library. The flag can be changed in the Flutter plugin CMakeLists.txt but I was looking for a way to do so automatically depending on whether flutter was invoked in debug/release mode. It's basically the same question as above.

So, this is not a need to have the same as the Flutter runtime, but the same between two native libraries linked together statically into a dylib. (This might be relevant at some point if we ever get to statically link the Flutter/Dart AOT runtime together with a Dart AOT snapshot together with static libraries from the build hooks. But we are rather far away from that.)

Then that linking of multiple native libraries into a single dylib is happing in a single hook. So that hook should be able to make the choice correctly.

Okay, if I understand correctly, user-defines should enable your workflow. (We don't have conditional user-defines though, so you'll have to toggle between debug/release manually. Maybe add a check on your CI which checks that the user-define equals release if the app version number is a non-prerelease.)

@nmfisher
Copy link
Contributor

Yes, in the very latest release on the master branch of flutter and the latest release of package:native_assets_cli.

Ah, great news.

So, this is not a need to have the same as the Flutter runtime, but the same between two native libraries linked together statically into a dylib. (This might be relevant at some point if we ever get to statically link the Flutter/Dart AOT runtime together with a Dart AOT snapshot together with static libraries from the build hooks. But we are rather far away from that.)

Then that linking of multiple native libraries into a single dylib is happing in a single hook. So that hook should be able to make the choice correctly.

Okay, if I understand correctly, user-defines should enable your workflow. (We don't have conditional user-defines though, so you'll have to toggle between debug/release manually. Maybe add a check on your CI which checks that the user-define equals release if the app version number is a non-prerelease.)

Yes, I probably wasn't clear - this isn't about Flutter itself, it's about a Flutter plugin that builds/links some native library. I will investigate user-defines and see if that solves the issue.

Thanks for taking the time to respond.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants