[jnigen] Remove C-based bindings (#1091)

Author: HosseinYousefi

.github/workflows/jnigen.yaml

@@ -105,8 +105,6 @@ jobs:
         working-directory: ./pkgs/jnigen/java
       - name: Build summarizer
         run: dart run jnigen:setup
-      - name: Generate runtime tests
-        run: dart run tool/generate_runtime_tests.dart
       - name: Run VM tests
         run: dart test --test-randomize-ordering-seed random
       - name: Install coverage
@@ -258,8 +256,6 @@ jobs:
       - run: dart run jnigen:setup
       - name: Build summarizer
         run: dart run jnigen:setup
-      - name: Generate runtime tests
-        run: dart run tool/generate_runtime_tests.dart
       - name: Run tests
         run: dart test --test-randomize-ordering-seed random
 
@@ -309,8 +305,6 @@ jobs:
       - run: dart pub get
       - name: Build summarizer
         run: dart run jnigen:setup
-      - name: Generate runtime tests
-        run: dart run tool/generate_runtime_tests.dart
       - name: Run tests
         run: dart test --test-randomize-ordering-seed random
 

pkgs/jni/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.9.0-wip
+
+- No changes yet.
+
 ## 0.8.0
 
 - **Breaking Change** ([#981](https://github.com/dart-lang/native/issues/981)):

pkgs/jni/lib/src/third_party/global_env_extensions.dart

@@ -3,6 +3,9 @@
 // This is generated from JNI header in Android NDK. License for the same is
 // provided below.
 
+// Generation logic resides in `tool/wrapper_generators`.
+// To regenerate, run `dart run tool/generate_ffi_bindings.dart`.
+
 /*
  * Copyright (C) 2006 The Android Open Source Project
  *

pkgs/jni/pubspec.yaml

@@ -4,7 +4,7 @@
 
 name: jni
 description: A library to access JNI from Dart and Flutter that acts as a support library for package:jnigen.
-version: 0.8.0
+version: 0.9.0-wip
 repository: https://github.com/dart-lang/native/tree/main/pkgs/jni
 
 topics:

pkgs/jni/src/third_party/global_jni_env.c

@@ -3,6 +3,9 @@
 // This is generated from JNI header in Android NDK. License for the same is
 // provided below.
 
+// Generation logic resides in `tool/wrapper_generators`.
+// To regenerate, run `dart run tool/generate_ffi_bindings.dart`.
+
 /*
  * Copyright (C) 2006 The Android Open Source Project
  *

pkgs/jni/src/third_party/global_jni_env.h

@@ -3,6 +3,9 @@
 // This is generated from JNI header in Android NDK. License for the same is
 // provided below.
 
+// Generation logic resides in `tool/wrapper_generators`.
+// To regenerate, run `dart run tool/generate_ffi_bindings.dart`.
+
 /*
  * Copyright (C) 2006 The Android Open Source Project
  *

pkgs/jni/tool/wrapper_generators/ffigen_util.dart

@@ -24,6 +24,9 @@ const preamble = '''
 // This is generated from JNI header in Android NDK. License for the same is
 // provided below.
 
+// Generation logic resides in `tool/wrapper_generators`.
+// To regenerate, run `dart run tool/generate_ffi_bindings.dart`.
+
 /*
  * Copyright (C) 2006 The Android Open Source Project
  *

pkgs/jnigen/CHANGELOG.md

@@ -1,5 +1,7 @@
-## 0.8.1-wip
+## 0.9.0-wip
 
+- **Breaking Change** ([#660](https://github.com/dart-lang/native/issues/660)):
+  Removed C-based bindings. Now all bindings are Dart-only.
 - Expand constraint on `package:cli_config` to allow `^0.2.0`.
 - Ignore `use_super_parameters` lint in generated files.
 

pkgs/jnigen/README.md

@@ -6,24 +6,20 @@
 ## Introduction
 Experimental bindings generator for Java bindings through dart:ffi and JNI.
 
-`jnigen` scans compiled JAR files or Java source code to generate a description of the API, then uses it to generate Dart annd C bindings. The Dart bindings call the C bindings, which in-turn call the Java functions through JNI. Shared functionality and base classes are provided through the support library, `package:jni`.
+`jnigen` scans compiled JAR files or Java source code to generate a description of the API, then uses it to generate Dart bindings. The Dart bindings call the C bindings, which in-turn call the Java functions through JNI. Shared functionality and base classes are provided through the support library, `package:jni`.
 
 The configuration for binding generation is usually provided through YAML.
 
 Three configuration details are needed to generate the bindings. Everything else is optional:
 
 * _Inputs_: input can be Java source files (`source_path`), or compiled classes / JARs (`class_path`). Some maven / gradle based tooling is also provided to simplify obtaining dependencies.
 
-* _Outputs_: Output can be generated in package-structured (one file per class) or single file bindings. Target path to write C and Dart bindings needs to be specified.
+* _Outputs_: Output can be generated in package-structured (one file per class) or single file bindings. Target path to write Dart bindings needs to be specified.
 
 * _Classes_: Specify which classes or packages you need bindings for. Specifying a package includes all classes inside it recursively.
 
 Check out the [examples](jnigen/example/) to see some sample configurations.
 
-C code is always generated into a directory with it's own build configuration. It's built as a separate dynamic library.
-
-Lastly, [dart_only bindings](#pure-dart-bindings) mode is also available as a proof-of-concept. It does not need intermediate C bindings, only a dependency on the support library `package:jni`.
-
 ## Example
 It's possible to generate bindings for JAR libraries, or Java source files.
 
@@ -53,53 +49,50 @@ This produces the following boilerplate:
 
 ```dart
 /// Some boilerplate is omitted for clarity.
-final ffi.Pointer<T> Function<T extends ffi.NativeType>(String sym) jniLookup =
-    ProtectedJniExtensions.initGeneratedLibrary("android_utils");
-
-/// from: com.example.in_app_java.AndroidUtils
 class AndroidUtils extends jni.JObject {
-  AndroidUtils.fromReference(JReference reference) : super.fromReference(reference);
+  @override
+  late final jni.JObjType<AndroidUtils> $type = type;
+
+  AndroidUtils.fromReference(
+    jni.JReference reference,
+  ) : super.fromReference(reference);
+
+  static final _class =
+      jni.JClass.forName(r"com/example/in_app_java/AndroidUtils");
 
-  static final _showToast = jniLookup<
+  /// The type which includes information such as the signature of this class.
+  static const type = $AndroidUtilsType();
+
+  static final _id_showToast = _class.staticMethodId(
+    r"showToast",
+    r"(Landroid/app/Activity;Ljava/lang/CharSequence;I)V",
+  );
+
+  static final _showToast = ProtectedJniExtensions.lookup<
           ffi.NativeFunction<
-              jni.JniResult Function(ffi.Pointer<ffi.Void>,
-                  ffi.Pointer<ffi.Void>, ffi.Int32)>>("AndroidUtils__showToast")
+              jni.JThrowablePtr Function(
+                  ffi.Pointer<ffi.Void>,
+                  jni.JMethodIDPtr,
+                  ffi.VarArgs<
+                      (
+                        ffi.Pointer<ffi.Void>,
+                        ffi.Pointer<ffi.Void>,
+                        ffi.Int64
+                      )>)>>("globalEnv_CallStaticVoidMethod")
       .asFunction<
-          jni.JniResult Function(
+          jni.JThrowablePtr Function(ffi.Pointer<ffi.Void>, jni.JMethodIDPtr,
               ffi.Pointer<ffi.Void>, ffi.Pointer<ffi.Void>, int)>();
 
   /// from: static public void showToast(android.app.Activity mainActivity, java.lang.CharSequence text, int duration)
   static void showToast(
-          jni.JObject mainActivity, jni.JObject text, int duration) =>
-      _showToast(mainActivity.reference, text.reference, duration).check();
-}
-```
-
-#### C Bindings:
-
-```c
-// Some boilerplate is omitted for clarity.
-
-// com.example.in_app_java.AndroidUtils
-jclass _c_AndroidUtils = NULL;
-
-jmethodID _m_AndroidUtils__showToast = NULL;
-FFI_PLUGIN_EXPORT
-JniResult AndroidUtils__showToast(jobject mainActivity,
-                                  jobject text,
-                                  int32_t duration) {
-  load_env();
-  load_class_gr(&_c_AndroidUtils, "com/example/in_app_java/AndroidUtils");
-  if (_c_AndroidUtils == NULL)
-    return (JniResult){.result = {.j = 0}, .exception = check_exception()};
-  load_static_method(_c_AndroidUtils, &_m_AndroidUtils__showToast, "showToast",
-                     "(Landroid/app/Activity;Ljava/lang/CharSequence;I)V");
-  if (_m_AndroidUtils__showToast == NULL)
-    return (JniResult){.result = {.j = 0}, .exception = check_exception()};
-  (*jniEnv)->CallStaticVoidMethod(jniEnv, _c_AndroidUtils,
-                                  _m_AndroidUtils__showToast, mainActivity,
-                                  text, duration);
-  return (JniResult){.result = {.j = 0}, .exception = check_exception()};
+    jni.JObject mainActivity,
+    jni.JObject text,
+    int duration,
+  ) {
+    _showToast(_class.reference.pointer, _id_showToast as jni.JMethodIDPtr,
+            mainActivity.reference.pointer, text.reference.pointer, duration)
+        .check();
+  }
 }
 ```
 
@@ -110,9 +103,6 @@ android_sdk_config:
   add_gradle_deps: true
 
 output:
-  c:
-    library_name: android_utils
-    path: src/android_utils/
   dart:
     path: lib/android_utils.dart
     structure: single_file
@@ -145,8 +135,6 @@ More advanced features such as callbacks are not supported yet. Support for thes
 
 On Flutter targets, native libraries are built automatically and bundled. On standalone platforms, no such infrastructure exists yet. As a stopgap solution, running `dart run jni:setup` in a target directory builds all JNI native dependencies of the package into `build/jni_libs`. 
 
-By default `jni:setup` goes through pubspec configuration and builds all JNI dependencies of the project. It can be overridden to build a custom directory using `-s` switch, which can be useful when output configuration for C bindings does not follow standard FFI plugin layout.
-
 The build directory has to be passed to `Jni.spawn` call. It's assumed that all dependencies are built into the same target directory, so that once JNI is initialized, generated bindings can load their respective C libraries automatically.
 
 ## Requirements
@@ -171,10 +159,8 @@ $env:Path += ";${env:JAVA_HOME}\bin\server".
 
 If JAVA_HOME not set, find the `java.exe` executable and set the environment variable in Control Panel. If java is installed through a package manager, there may be a more automatic way to do this. (Eg: `scoop reset`).
 
-### C/C++ tooling
-CMake and a standard C toolchain are required to build `package:jni` and C bindings generated by `jnigen`.
-
-It's recommended to have `clang-format` installed for formatting the generated C bindings. On Windows, it's part of LLVM installation. On most Linux distributions it is available as a separate package. On MacOS, it can be installed using Homebrew.
+### C tooling
+CMake and a standard C toolchain are required to build `package:jni`.
 
 ## FAQs
 
@@ -225,11 +211,6 @@ A `*` denotes required configuration.
 | `classes` *            | List of qualified class / package names | List of qualified class / package names. `source_path` will be scanned assuming the sources follow standard java-ish hierarchy. That is a.b.c either maps to a directory `a/b/c` or a class file `a/b/c.java`.  |
 | `enable_experiment`    | List of experiment names:<br><ul><li>`interface_implementation`</li></ul> | List of enabled experiments. These features are still in development and their API might break.  |
 | `output:`              | (Subsection) | This subsection will contain configuration related to output files. |
-| `output:` >> `bindings_type` | `c_based` (default) or `dart_only` | Binding generation strategy. [Trade-offs](#pure-dart-bindings) are explained at the end of this document. |
-| `output:` >> `c:`      | (Subsection) | This subsection specified C output configuration. Required if `bindings_type` is `c_based`. |
-| `output:` >> `c:` >> path *       | Directory path | Directory to write C bindings. Usually `src/` in case of an FFI plugin template. |
-| `output:` >> `c:` >> subdir       | Directory path | If specified, C bindings will be written to `subdir` resolved relative to `path`. This is useful when bindings are supposed to be under source's license, and written to a subdirectory such as `third_party`. |
-| `output:` >> `c:` >> `library_name` *| Identifier (snake_case) | Name for generated C library.
 | `output:` >> `dart:` | (Subsection) | This subsection specifies Dart output configuration. |
 | `output:` >> `dart:` >> `structure` | `package_structure` / `single_file` | Whether to map resulting dart bindings to file-per-class source layout, or write all bindings to single file.
 | `output:` >> `dart:` >> `path` * | Directory path or File path | Path to write Dart bindings. Should end in `.dart` for `single_file` configurations, and end in `/` for `package_structure` (default) configuration. |
@@ -255,19 +236,6 @@ It's possible to use the programmatic API instead of YAML.
 * import `package:jnigen/jnigen.dart`
 * construct a `Config` object and pass it to `generateJniBindings` function. The parameters are similar to the ones described above.
 
-## Pure dart Bindings
-It's possible to generate bindings that do not rely on an intermediate layer of C code. Bindings will still depend on `package:jni` and its support library written in C. But this approach avoids large C bindings.
-
-To enable pure dart bindings, specify
-```
-output:
-	bindings_type: dart_only
-```
-
-Any C output configuration will be ignored.
-
-However, pure dart bindings will require additional allocations and check runtimeType of the arguments. This will be the case until Variadic arguments land in Dart FFI.
-
 ## Android core libraries
 These days, Android projects depend heavily on AndroidX and other libraries downloaded via gradle. We have a tracking issue to improve detection of android SDK and dependencies. (#31). Currently we can fetch the JAR dependencies of an android project, by running a gradle stub, if `android_sdk_config` >> `add_gradle_deps` is specified. 
 

pkgs/jnigen/android_test_runner/android/app/build.gradle

@@ -59,11 +59,6 @@ android {
             signingConfig signingConfigs.debug
         }
     }
-    externalNativeBuild {
-        cmake {
-            path 'CMakeLists.txt'
-        }
-    }
 }
 
 flutter {

pkgs/jnigen/android_test_runner/integration_test/.gitignore

@@ -0,0 +1,25 @@
+// Generated file. Do not edit or check-in to version control.
+
+// Copyright (c) 2023, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import "package:flutter_test/flutter_test.dart";
+
+import "../../test/jackson_core_test/runtime_test_registrant.dart"
+    as jackson_core_test;
+import "../../test/simple_package_test/runtime_test_registrant.dart"
+    as simple_package_test;
+import "../../test/kotlin_test/runtime_test_registrant.dart" as kotlin_test;
+
+typedef TestCaseCallback = void Function();
+
+void test(String description, TestCaseCallback testCase) {
+  testWidgets(description, (widgetTester) async => testCase());
+}
+
+void main() {
+  jackson_core_test.registerTests("jackson_core_test", test);
+  simple_package_test.registerTests("simple_package_test", test);
+  kotlin_test.registerTests("kotlin_test", test);
+}

pkgs/jnigen/android_test_runner/integration_test/runtime_test.dart

@@ -1,24 +1,28 @@
 # In-App Java Example
 
-This example shows how to write custom java code in `android/app/src` and call it using `jnigen` generated bindings.
+This example shows how to write custom java code in `android/app/src` and call
+it using `jnigen` generated bindings.
 
 #### How to run this example:
-* Run `flutter run` to run the app.
 
-* To regenerate bindings after changing Java code, run `flutter pub run jnigen --config jnigen.yaml`. This requires at least one APK build to have been run before, so that it's possible for `jnigen` to obtain classpaths of Android Gradle libraries. Therefore, once run `flutter build apk` before generating bindings for the first time, or after a `flutter clean`.
+- Run `flutter run` to run the app.
 
-#### General steps
-These are general steps to integrate Java code into a flutter project using `jnigen`.
-
-* Write Java code in suitable package folder, under `android/` subproject of the flutter app.
-
-* Create A jnigen config like `jnigen.yaml` in this example.
+- To regenerate bindings after changing Java code, run
+  `flutter pub run jnigen --config jnigen.yaml`. This requires at least one APK
+  build to have been run before, so that it's possible for `jnigen` to obtain
+  classpaths of Android Gradle libraries. Therefore, once run
+  `flutter build apk` before generating bindings for the first time, or after a
+  `flutter clean`.
 
-* Generate bindings using jnigen config.
-
-* Add an `externalNativeBuild` to gradle script (see `android/app/build.gradle` in this example).
-
-* Add proguard rules to exclude your custom classes from tree shaking, since they are always accessed reflectively in JNI.
+#### General steps
 
-* Build and run the app.
+These are general steps to integrate Java code into a flutter project using
+`jnigen`.
 
+- Write Java code in suitable package folder, under `android/` subproject of the
+  flutter app.
+- Create A jnigen config like `jnigen.yaml` in this example.
+- Generate bindings using jnigen config.
+- Add proguard rules to exclude your custom classes from tree shaking, since
+  they are always accessed reflectively in JNI.
+- Build and run the app.