Skip to content

[camera_avfoundation] fix sample times not being numeric after pause/resume. #6897

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 9, 2024
Merged

[camera_avfoundation] fix sample times not being numeric after pause/resume. #6897

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
76e4f1a
fix sample times not being numeric after pause
misos1 Jun 9, 2024
b066729
update comment
misos1 Jun 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions packages/camera/camera_avfoundation/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
## 0.9.16+1

* Fixes sample times not being numeric after pause/resume.

## 0.9.16

* Converts Dart-to-host communcation to Pigeon.
Expand Down
68 changes: 68 additions & 0 deletions packages/camera/camera_avfoundation/example/ios/RunnerTests/FLTCamSampleBufferTests.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,4 +130,72 @@ - (void)testDidOutputSampleBufferIgnoreAudioSamplesBeforeVideoSamples {
CFRelease(audioSample);
}

- (void)testDidOutputSampleBufferSampleTimesMustBeNumericAfterPauseResume {
FLTCam *cam = FLTCreateCamWithCaptureSessionQueue(dispatch_queue_create("testing", NULL));
CMSampleBufferRef videoSample = FLTCreateTestSampleBuffer();
CMSampleBufferRef audioSample = FLTCreateTestAudioSampleBuffer();

id connectionMock = OCMClassMock([AVCaptureConnection class]);

id writerMock = OCMClassMock([AVAssetWriter class]);
OCMStub([writerMock alloc]).andReturn(writerMock);
OCMStub([writerMock initWithURL:OCMOCK_ANY fileType:OCMOCK_ANY error:[OCMArg setTo:nil]])
.andReturn(writerMock);
__block AVAssetWriterStatus status = AVAssetWriterStatusUnknown;
OCMStub([writerMock startWriting]).andDo(^(NSInvocation *invocation) {
status = AVAssetWriterStatusWriting;
});
OCMStub([writerMock status]).andDo(^(NSInvocation *invocation) {
[invocation setReturnValue:&status];
});

__block BOOL videoAppended = NO;
id adaptorMock = OCMClassMock([AVAssetWriterInputPixelBufferAdaptor class]);
OCMStub([adaptorMock assetWriterInputPixelBufferAdaptorWithAssetWriterInput:OCMOCK_ANY
sourcePixelBufferAttributes:OCMOCK_ANY])
.andReturn(adaptorMock);
OCMStub([adaptorMock appendPixelBuffer:[OCMArg anyPointer] withPresentationTime:kCMTimeZero])
.ignoringNonObjectArgs()
.andDo(^(NSInvocation *invocation) {
CMTime presentationTime;
[invocation getArgument:&presentationTime atIndex:3];
XCTAssert(CMTIME_IS_NUMERIC(presentationTime));
videoAppended = YES;
});

__block BOOL audioAppended = NO;
id inputMock = OCMClassMock([AVAssetWriterInput class]);
OCMStub([inputMock assetWriterInputWithMediaType:OCMOCK_ANY outputSettings:OCMOCK_ANY])
.andReturn(inputMock);
OCMStub([inputMock isReadyForMoreMediaData]).andReturn(YES);
OCMStub([inputMock appendSampleBuffer:[OCMArg anyPointer]]).andDo(^(NSInvocation *invocation) {
CMSampleBufferRef sampleBuffer;
[invocation getArgument:&sampleBuffer atIndex:2];
CMTime sampleTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
XCTAssert(CMTIME_IS_NUMERIC(sampleTime));
audioAppended = YES;
});

[cam
startVideoRecordingWithCompletion:^(FlutterError *_Nullable error) {
}
messengerForStreaming:nil];

[cam pauseVideoRecording];
[cam resumeVideoRecording];

[cam captureOutput:cam.captureVideoOutput
didOutputSampleBuffer:videoSample
fromConnection:connectionMock];
[cam captureOutput:nil didOutputSampleBuffer:audioSample fromConnection:connectionMock];
[cam captureOutput:cam.captureVideoOutput
didOutputSampleBuffer:videoSample
fromConnection:connectionMock];
[cam captureOutput:nil didOutputSampleBuffer:audioSample fromConnection:connectionMock];
XCTAssert(videoAppended && audioAppended, @"Video or audio was not appended.");

CFRelease(videoSample);
CFRelease(audioSample);
}

@end
4 changes: 4 additions & 0 deletions packages/camera/camera_avfoundation/ios/Classes/FLTCam.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -672,6 +672,10 @@ - (void)captureOutput:(AVCaptureOutput *)output
if (_videoWriter.status != AVAssetWriterStatusWriting) {
[_videoWriter startWriting];
[_videoWriter startSessionAtSourceTime:currentSampleTime];
// fix sample times not being numeric when pause/resume happens before first sample buffer
// arrives https://github.com/flutter/flutter/issues/132014
_lastVideoSampleTime = currentSampleTime;
_lastAudioSampleTime = currentSampleTime;
}

if (output == _captureVideoOutput) {
Expand Down
2 changes: 1 addition & 1 deletion packages/camera/camera_avfoundation/pubspec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: camera_avfoundation
description: iOS implementation of the camera plugin.
repository: https://github.com/flutter/packages/tree/main/packages/camera/camera_avfoundation
issue_tracker: https://github.com/flutter/flutter/issues?q=is%3Aissue+is%3Aopen+label%3A%22p%3A+camera%22
version: 0.9.16
version: 0.9.16+1

environment:
sdk: ^3.2.3
Expand Down