Skip to content

Commit 79d733c

Browse files
camsim99camsim99
authored
[camerax] Prevent using unsupported concurrent camera use cases (flutter#6608)
Adds logic + documentation to prevent the usage of unsupported (by CameraX) concurrent camera use cases. The CameraX plugin should only support the concurrent camera use cases supported by Camerax; see [their documentation](https://developer.android.com/media/camera/camerax/architecture#combine-use-cases) for more information. To avoid the usage of unsupported concurrent use cases, this PR changes the plugin so that it behaves according to the following: * If the preview is paused (via `pausePreview`), concurrent video recording and image streaming (via `startVideoCapturing(cameraId, VideoCaptureOptions(streamCallback:...))`) is supported. * If the preview is not paused * **and** the camera device is at least supported hardware [`LIMITED`](https://developer.android.com/reference/android/hardware/camera2/CameraMetadata#INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED), then concurrent image capture and video recording is supported. * **and** the camera device is at least supported hardware [`LEVEL_3`](https://developer.android.com/reference/android/hardware/camera2/CameraMetadata#INFO_SUPPORTED_HARDWARE_LEVEL_3), then concurrent video recording and image streaming is supported, but if used, concurrent image capture alongside this is not supported. Also changes `Surface` constants to match Dart style and small documentation nits I noticed. Fixes flutter#146395, fixes flutter#144414.
1 parent 2dfe645 commit 79d733c

26 files changed

+2333
-545
lines changed

‎packages/camera/camera_android_camerax/CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
## 0.6.4
2+
3+
* Prevents usage of unsupported concurrent `UseCase`s based on the capabiliites of the camera device.
4+
15
## 0.6.3
26

37
* Shortens default interval that internal Java `InstanceManager` uses to release garbage collected weak references to

‎packages/camera/camera_android_camerax/README.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,22 @@ from your project's root directory.
2121

2222
## Limitations
2323

24+
### Concurrent preview display, video recording, image capture, and image streaming
25+
26+
The CameraX plugin only supports the concurrent camera use cases supported by Camerax; see
27+
[their documentation][6] for more information. To avoid the usage of unsupported concurrent
28+
use cases, the plugin behaves according to the following:
29+
30+
* If the preview is paused (via `pausePreview`), concurrent video recording and image capture
31+
and/or image streaming (via `startVideoCapturing(cameraId, VideoCaptureOptions(streamCallback:...))`)
32+
is supported.
33+
* If the preview is not paused
34+
* **and** the camera device is at least supported hardware [`LIMITED`][8], then concurrent
35+
image capture and video recording is supported.
36+
* **and** the camera device is at least supported hardware [`LEVEL_3`][7], then concurrent
37+
video recording and image streaming is supported, but concurrent video recording, image
38+
streaming, and image capture is not supported.
39+
2440
### 240p resolution configuration for video recording
2541

2642
240p resolution configuration for video recording is unsupported by CameraX,
@@ -45,6 +61,9 @@ For more information on contributing to this plugin, see [`CONTRIBUTING.md`](CON
4561
[3]: https://docs.flutter.dev/packages-and-plugins/developing-packages#non-endorsed-federated-plugin
4662
[4]: https://pub.dev/packages/camera_android
4763
[5]: https://github.com/flutter/flutter/issues/new/choose
64+
[6]: https://developer.android.com/media/camera/camerax/architecture#combine-use-cases
65+
[7]: https://developer.android.com/reference/android/hardware/camera2/CameraMetadata#INFO_SUPPORTED_HARDWARE_LEVEL_3
66+
[8]: https://developer.android.com/reference/android/hardware/camera2/CameraMetadata#INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED
4867
[120462]: https://github.com/flutter/flutter/issues/120462
4968
[125915]: https://github.com/flutter/flutter/issues/125915
5069
[120715]: https://github.com/flutter/flutter/issues/120715

‎packages/camera/camera_android_camerax/android/src/main/java/io/flutter/plugins/camerax/Camera2CameraInfoFlutterApiImpl.java

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
// Copyright 2013 The Flutter Authors. All rights reserved.
2+
// Use of this source code is governed by a BSD-style license that can be
3+
// found in the LICENSE file.
4+
5+
package io.flutter.plugins.camerax;
6+
7+
import androidx.annotation.NonNull;
8+
import androidx.annotation.Nullable;
9+
import androidx.camera.camera2.interop.Camera2CameraInfo;
10+
import io.flutter.plugin.common.BinaryMessenger;
11+
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.Camera2CameraInfoFlutterApi;
12+
13+
public class Camera2CameraInfoFlutterApiImpl extends Camera2CameraInfoFlutterApi {
14+
private final InstanceManager instanceManager;
15+
16+
public Camera2CameraInfoFlutterApiImpl(
17+
@Nullable BinaryMessenger binaryMessenger, @Nullable InstanceManager instanceManager) {
18+
super(binaryMessenger);
19+
this.instanceManager = instanceManager;
20+
}
21+
22+
void create(@NonNull Camera2CameraInfo camera2CameraInfo, @Nullable Reply<Void> reply) {
23+
if (!instanceManager.containsInstance(camera2CameraInfo)) {
24+
create(instanceManager.addHostCreatedInstance(camera2CameraInfo), reply);
25+
}
26+
}
27+
}

‎packages/camera/camera_android_camerax/android/src/main/java/io/flutter/plugins/camerax/Camera2CameraInfoHostApiImpl.java

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
// Copyright 2013 The Flutter Authors. All rights reserved.
2+
// Use of this source code is governed by a BSD-style license that can be
3+
// found in the LICENSE file.
4+
5+
package io.flutter.plugins.camerax;
6+
7+
import android.content.Context;
8+
import android.hardware.camera2.CameraCharacteristics;
9+
import androidx.annotation.NonNull;
10+
import androidx.annotation.OptIn;
11+
import androidx.annotation.VisibleForTesting;
12+
import androidx.camera.camera2.interop.Camera2CameraInfo;
13+
import androidx.camera.camera2.interop.ExperimentalCamera2Interop;
14+
import androidx.camera.core.CameraInfo;
15+
import io.flutter.plugin.common.BinaryMessenger;
16+
import io.flutter.plugins.camerax.GeneratedCameraXLibrary.Camera2CameraInfoHostApi;
17+
import java.util.Objects;
18+
19+
/**
20+
* Host API implementation for {@link Camera2CameraInfo}.
21+
*
22+
* <p>This class handles instantiating and adding native object instances that are attached to a
23+
* Dart instance or handle method calls on the associated native class or an instance of the class.
24+
*/
25+
public class Camera2CameraInfoHostApiImpl implements Camera2CameraInfoHostApi {
26+
private final BinaryMessenger binaryMessenger;
27+
private final InstanceManager instanceManager;
28+
private final Camera2CameraInfoProxy proxy;
29+
30+
/** Proxy for methods of {@link Camera2CameraInfo}. */
31+
@VisibleForTesting
32+
public static class Camera2CameraInfoProxy {
33+
34+
@NonNull
35+
@OptIn(markerClass = ExperimentalCamera2Interop.class)
36+
public Camera2CameraInfo createFrom(@NonNull CameraInfo cameraInfo) {
37+
return Camera2CameraInfo.from(cameraInfo);
38+
}
39+
40+
@NonNull
41+
@OptIn(markerClass = ExperimentalCamera2Interop.class)
42+
public Integer getSupportedHardwareLevel(@NonNull Camera2CameraInfo camera2CameraInfo) {
43+
return camera2CameraInfo.getCameraCharacteristic(
44+
CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);
45+
}
46+
47+
@NonNull
48+
@OptIn(markerClass = ExperimentalCamera2Interop.class)
49+
public String getCameraId(@NonNull Camera2CameraInfo camera2CameraInfo) {
50+
return camera2CameraInfo.getCameraId();
51+
}
52+
}
53+
54+
/**
55+
* Constructs an {@link Camera2CameraInfoHostApiImpl}.
56+
*
57+
* @param binaryMessenger used to communicate with Dart over asynchronous messages
58+
* @param instanceManager maintains instances stored to communicate with attached Dart objects
59+
* @param context {@link Context} used to retrieve {@code Executor}
60+
*/
61+
public Camera2CameraInfoHostApiImpl(
62+
@NonNull BinaryMessenger binaryMessenger, @NonNull InstanceManager instanceManager) {
63+
this(binaryMessenger, instanceManager, new Camera2CameraInfoProxy());
64+
}
65+
66+
/**
67+
* Constructs an {@link Camera2CameraInfoHostApiImpl}.
68+
*
69+
* @param binaryMessenger used to communicate with Dart over asynchronous messages
70+
* @param instanceManager maintains instances stored to communicate with attached Dart objects
71+
* @param proxy proxy for methods of {@link Camera2CameraInfo}
72+
*/
73+
@VisibleForTesting
74+
Camera2CameraInfoHostApiImpl(
75+
@NonNull BinaryMessenger binaryMessenger,
76+
@NonNull InstanceManager instanceManager,
77+
@NonNull Camera2CameraInfoProxy proxy) {
78+
this.instanceManager = instanceManager;
79+
this.binaryMessenger = binaryMessenger;
80+
this.proxy = proxy;
81+
}
82+
83+
@Override
84+
@NonNull
85+
public Long createFrom(@NonNull Long cameraInfoIdentifier) {
86+
final CameraInfo cameraInfo =
87+
Objects.requireNonNull(instanceManager.getInstance(cameraInfoIdentifier));
88+
final Camera2CameraInfo camera2CameraInfo = proxy.createFrom(cameraInfo);
89+
final Camera2CameraInfoFlutterApiImpl flutterApi =
90+
new Camera2CameraInfoFlutterApiImpl(binaryMessenger, instanceManager);
91+
92+
flutterApi.create(camera2CameraInfo, reply -> {});
93+
return instanceManager.getIdentifierForStrongReference(camera2CameraInfo);
94+
}
95+
96+
@Override
97+
@NonNull
98+
public Long getSupportedHardwareLevel(@NonNull Long identifier) {
99+
return Long.valueOf(proxy.getSupportedHardwareLevel(getCamera2CameraInfoInstance(identifier)));
100+
}
101+
102+
@Override
103+
@NonNull
104+
public String getCameraId(@NonNull Long identifier) {
105+
return proxy.getCameraId(getCamera2CameraInfoInstance(identifier));
106+
}
107+
108+
private Camera2CameraInfo getCamera2CameraInfoInstance(@NonNull Long identifier) {
109+
return Objects.requireNonNull(instanceManager.getInstance(identifier));
110+
}
111+
}

‎packages/camera/camera_android_camerax/android/src/main/java/io/flutter/plugins/camerax/CameraAndroidCameraxPlugin.java

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,8 @@ public void setUp(
136136
GeneratedCameraXLibrary.MeteringPointHostApi.setup(binaryMessenger, meteringPointHostApiImpl);
137137
GeneratedCameraXLibrary.ResolutionFilterHostApi.setup(
138138
binaryMessenger, new ResolutionFilterHostApiImpl(instanceManager));
139+
GeneratedCameraXLibrary.Camera2CameraInfoHostApi.setup(
140+
binaryMessenger, new Camera2CameraInfoHostApiImpl(binaryMessenger, instanceManager));
139141
}
140142

141143
@Override

‎packages/camera/camera_android_camerax/android/src/main/java/io/flutter/plugins/camerax/GeneratedCameraXLibrary.java

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4267,4 +4267,137 @@ static void setup(
42674267
}
42684268
}
42694269
}
4270+
/** Generated interface from Pigeon that represents a handler of messages from Flutter. */
4271+
public interface Camera2CameraInfoHostApi {
4272+
4273+
@NonNull
4274+
Long createFrom(@NonNull Long cameraInfoIdentifier);
4275+
4276+
@NonNull
4277+
Long getSupportedHardwareLevel(@NonNull Long identifier);
4278+
4279+
@NonNull
4280+
String getCameraId(@NonNull Long identifier);
4281+
4282+
/** The codec used by Camera2CameraInfoHostApi. */
4283+
static @NonNull MessageCodec<Object> getCodec() {
4284+
return new StandardMessageCodec();
4285+
}
4286+
/**
4287+
* Sets up an instance of `Camera2CameraInfoHostApi` to handle messages through the
4288+
* `binaryMessenger`.
4289+
*/
4290+
static void setup(
4291+
@NonNull BinaryMessenger binaryMessenger, @Nullable Camera2CameraInfoHostApi api) {
4292+
{
4293+
BasicMessageChannel<Object> channel =
4294+
new BasicMessageChannel<>(
4295+
binaryMessenger,
4296+
"dev.flutter.pigeon.Camera2CameraInfoHostApi.createFrom",
4297+
getCodec());
4298+
if (api != null) {
4299+
channel.setMessageHandler(
4300+
(message, reply) -> {
4301+
ArrayList<Object> wrapped = new ArrayList<Object>();
4302+
ArrayList<Object> args = (ArrayList<Object>) message;
4303+
Number cameraInfoIdentifierArg = (Number) args.get(0);
4304+
try {
4305+
Long output =
4306+
api.createFrom(
4307+
(cameraInfoIdentifierArg == null)
4308+
? null
4309+
: cameraInfoIdentifierArg.longValue());
4310+
wrapped.add(0, output);
4311+
} catch (Throwable exception) {
4312+
ArrayList<Object> wrappedError = wrapError(exception);
4313+
wrapped = wrappedError;
4314+
}
4315+
reply.reply(wrapped);
4316+
});
4317+
} else {
4318+
channel.setMessageHandler(null);
4319+
}
4320+
}
4321+
{
4322+
BasicMessageChannel<Object> channel =
4323+
new BasicMessageChannel<>(
4324+
binaryMessenger,
4325+
"dev.flutter.pigeon.Camera2CameraInfoHostApi.getSupportedHardwareLevel",
4326+
getCodec());
4327+
if (api != null) {
4328+
channel.setMessageHandler(
4329+
(message, reply) -> {
4330+
ArrayList<Object> wrapped = new ArrayList<Object>();
4331+
ArrayList<Object> args = (ArrayList<Object>) message;
4332+
Number identifierArg = (Number) args.get(0);
4333+
try {
4334+
Long output =
4335+
api.getSupportedHardwareLevel(
4336+
(identifierArg == null) ? null : identifierArg.longValue());
4337+
wrapped.add(0, output);
4338+
} catch (Throwable exception) {
4339+
ArrayList<Object> wrappedError = wrapError(exception);
4340+
wrapped = wrappedError;
4341+
}
4342+
reply.reply(wrapped);
4343+
});
4344+
} else {
4345+
channel.setMessageHandler(null);
4346+
}
4347+
}
4348+
{
4349+
BasicMessageChannel<Object> channel =
4350+
new BasicMessageChannel<>(
4351+
binaryMessenger,
4352+
"dev.flutter.pigeon.Camera2CameraInfoHostApi.getCameraId",
4353+
getCodec());
4354+
if (api != null) {
4355+
channel.setMessageHandler(
4356+
(message, reply) -> {
4357+
ArrayList<Object> wrapped = new ArrayList<Object>();
4358+
ArrayList<Object> args = (ArrayList<Object>) message;
4359+
Number identifierArg = (Number) args.get(0);
4360+
try {
4361+
String output =
4362+
api.getCameraId((identifierArg == null) ? null : identifierArg.longValue());
4363+
wrapped.add(0, output);
4364+
} catch (Throwable exception) {
4365+
ArrayList<Object> wrappedError = wrapError(exception);
4366+
wrapped = wrappedError;
4367+
}
4368+
reply.reply(wrapped);
4369+
});
4370+
} else {
4371+
channel.setMessageHandler(null);
4372+
}
4373+
}
4374+
}
4375+
}
4376+
/** Generated class from Pigeon that represents Flutter messages that can be called from Java. */
4377+
public static class Camera2CameraInfoFlutterApi {
4378+
private final @NonNull BinaryMessenger binaryMessenger;
4379+
4380+
public Camera2CameraInfoFlutterApi(@NonNull BinaryMessenger argBinaryMessenger) {
4381+
this.binaryMessenger = argBinaryMessenger;
4382+
}
4383+
4384+
/** Public interface for sending reply. */
4385+
@SuppressWarnings("UnknownNullness")
4386+
public interface Reply<T> {
4387+
void reply(T reply);
4388+
}
4389+
/** The codec used by Camera2CameraInfoFlutterApi. */
4390+
static @NonNull MessageCodec<Object> getCodec() {
4391+
return new StandardMessageCodec();
4392+
}
4393+
4394+
public void create(@NonNull Long identifierArg, @NonNull Reply<Void> callback) {
4395+
BasicMessageChannel<Object> channel =
4396+
new BasicMessageChannel<>(
4397+
binaryMessenger, "dev.flutter.pigeon.Camera2CameraInfoFlutterApi.create", getCodec());
4398+
channel.send(
4399+
new ArrayList<Object>(Collections.singletonList(identifierArg)),
4400+
channelReply -> callback.reply(null));
4401+
}
4402+
}
42704403
}

0 commit comments

Comments
 (0)