Replaykit，startCaptureWithHandler()未在captureHandler中发送视频类型的CMSampleBufferRef [英] Replaykit, startCaptureWithHandler() not sending CMSampleBufferRef of Video type in captureHandler
问题描述
我实现了RPScreenRecorder，它可以记录屏幕以及麦克风音频.完成多个录制后，我停止录制，并使用AVMutableComposition将音频与视频合并，然后合并所有视频以形成单个视频.
I've implemented a RPScreenRecorder, which records screen as well as mic audio. After multiple recordings are completed I stop the recording and merge the Audios with Videos using AVMutableComposition and then Merge all the videos to form Single Video.
我正在使用屏幕录制和获取视频和音频文件
For screen recording and getting the video and audio files, I am using
- (void)startCaptureWithHandler:(nullable void(^)(CMSampleBufferRef sampleBuffer, RPSampleBufferType bufferType, NSError * _Nullable error))captureHandler completionHandler:
用于停止录制.我调用此函数:
For stopping the recording. I Call this function:
- (void)stopCaptureWithHandler:(void (^)(NSError *error))handler;
这些很简单.
在大多数情况下，它都能正常运行，我同时接收了视频和音频CMSampleBuffers.
但是有时候， startCaptureWithHandler只会向我发送音频缓冲区，而不向我发送视频缓冲区.
一旦遇到此问题，直到我重新启动设备并重新安装该应用程序后，该问题才会消失.这使我的应用对用户而言如此不可靠.
我认为这是一个重放工具包问题，但无法与其他开发人员一起找到相关问题.
让我知道你们中是否有人遇到过这个问题并得到了解决方案.
Most of the times it works great, I receive both video and audio CMSampleBuffers.
But some times it so happens that startCaptureWithHandler only sends me audio buffers but not video buffers.
And once I encounter this problem, it won't go until I restart my device and reinstall the app. This makes my app so unreliable for the user.
I think this is a replay kit issue but unable to found out related issues with other developers.
let me know if any one of you came across this issue and got the solution.
我已经检查了多次，但是在配置中没有看到任何问题. 但是无论如何，这里还是.
I have check multiple times but haven't seen any issue in configuration. But here it is anyway.
NSError *videoWriterError;
videoWriter = [[AVAssetWriter alloc] initWithURL:fileString fileType:AVFileTypeQuickTimeMovie
error:&videoWriterError];
NSError *audioWriterError;
audioWriter = [[AVAssetWriter alloc] initWithURL:audioFileString fileType:AVFileTypeAppleM4A
error:&audioWriterError];
CGFloat width =UIScreen.mainScreen.bounds.size.width;
NSString *widthString = [NSString stringWithFormat:@"%f", width];
CGFloat height =UIScreen.mainScreen.boNSString *heightString = [NSString stringWithFormat:@"%f", height];unds.size.height;
NSDictionary * videoOutputSettings= @{AVVideoCodecKey : AVVideoCodecTypeH264,
AVVideoWidthKey: widthString,
AVVideoHeightKey : heightString};
videoInput = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeVideo outputSettings:videoOutputSettings];
videoInput.expectsMediaDataInRealTime = true;
AudioChannelLayout acl;
bzero( &acl, sizeof(acl));
acl.mChannelLayoutTag = kAudioChannelLayoutTag_Mono;
NSDictionary * audioOutputSettings = [ NSDictionary dictionaryWithObjectsAndKeys:
[ NSNumber numberWithInt: kAudioFormatAppleLossless ], AVFormatIDKey,
[ NSNumber numberWithInt: 16 ], AVEncoderBitDepthHintKey,
[ NSNumber numberWithFloat: 44100.0 ], AVSampleRateKey,
[ NSNumber numberWithInt: 1 ], AVNumberOfChannelsKey,
[ NSData dataWithBytes: &acl length: sizeof( acl ) ], AVChannelLayoutKey,
nil ];
audioInput = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeAudio outputSettings:audioOutputSettings];
[audioInput setExpectsMediaDataInRealTime:YES];
[videoWriter addInput:videoInput];
[audioWriter addInput:audioInput];
[[AVAudioSession sharedInstance] setCategory: AVAudioSessionCategoryPlayAndRecord withOptions:AVAudioSessionCategoryOptionDefaultToSpeaker error:nil];
[RPScreenRecorder.sharedRecorder startCaptureWithHandler:^(CMSampleBufferRef _Nonnull sampleBuffer, RPSampleBufferType bufferType, NSError * _Nullable myError) {
Block
}
startCaptureWithHandler函数也具有非常简单的功能:
The startCaptureWithHandler function has pretty straight forward functionality as well:
[RPScreenRecorder.sharedRecorder startCaptureWithHandler:^(CMSampleBufferRef _Nonnull sampleBuffer, RPSampleBufferType bufferType, NSError * _Nullable myError) {
dispatch_sync(dispatch_get_main_queue(), ^{
if(CMSampleBufferDataIsReady(sampleBuffer))
{
if (self->videoWriter.status == AVAssetWriterStatusUnknown)
{
self->writingStarted = true;
[self->videoWriter startWriting];
[self->videoWriter startSessionAtSourceTime:CMSampleBufferGetPresentationTimeStamp(sampleBuffer)];
[self->audioWriter startWriting];
[self->audioWriter startSessionAtSourceTime:CMSampleBufferGetPresentationTimeStamp(sampleBuffer)];
}
if (self->videoWriter.status == AVAssetWriterStatusFailed) {
return;
}
if (bufferType == RPSampleBufferTypeVideo)
{
if (self->videoInput.isReadyForMoreMediaData)
{
[self->videoInput appendSampleBuffer:sampleBuffer];
}
}
else if (bufferType == RPSampleBufferTypeAudioMic)
{
// printf("\n+++ bufferAudio received %d \n",arc4random_uniform(100));
if (writingStarted){
if (self->audioInput.isReadyForMoreMediaData)
{
[self->audioInput appendSampleBuffer:sampleBuffer];
}
}
}
}
});
}
此外，发生这种情况时，系统屏幕录像机也会损坏.单击系统记录器时，将显示此错误:
Also, when this situation occurs, the system screen recorder gets corrupted as well. On clicking system recorder, this error shows up:
错误显示为由于以下原因导致屏幕录制已停止:由于Mediaservices错误而导致录制期间失败".
The error says "Screen recording has stopped due to: Failure during recording due to Mediaservices error".
必须有两个原因:
- iOS Replay套件处于测试版，这就是为什么有时使用后它会出现问题的原因.
- 我实施了任何有问题的逻辑，这会导致replaykit崩溃.
如果不是问题. 1，那就没问题. 如果是这样，则不行. 2然后我必须知道我可能在哪里错了?
If it's issue no. 1, then no problem. If this is issue no. 2 then I have to know where I might be wrong?
意见和帮助将不胜感激.
Opinions and help will be appreciated.
推荐答案
因此，我遇到了某些情况，其中重播工具包完全崩溃，并且除非每次重新启动设备，否则系统记录器每次都会显示错误.
So, I have come across some scenarios where Replay kit totally crashes and System recorder shows error every time unless you restart the device.
第一种情况
开始记录并在完成处理程序中将其停止
[RPScreenRecorder.sharedRecorder startCaptureWithHandler:^(CMSampleBufferRef _Nonnull sampleBuffer, RPSampleBufferType bufferType, NSError * _Nullable error) {
printf("recording");
} completionHandler:^(NSError * _Nullable error) {
[RPScreenRecorder.sharedRecorder stopCaptureWithHandler:^(NSError * _Nullable error) {
printf("Ended");
}];
}];
第二种情况
开始录制并直接在捕获处理程序中停止录制
__block BOOL stopDone = NO;
[RPScreenRecorder.sharedRecorder startCaptureWithHandler:^(CMSampleBufferRef _Nonnull sampleBuffer, RPSampleBufferType bufferType, NSError * _Nullable error) {
if (!stopDone){
[RPScreenRecorder.sharedRecorder stopCaptureWithHandler:^(NSError * _Nullable error) {
printf("Ended");
}];
stopDone = YES;
}
printf("recording");
} completionHandler:^(NSError * _Nullable error) {}];
更多场景尚未发现，我将继续更新 回答
More Scenarios are yet to be discovered and I will keep updating the answer
更新1
确实，当我们在开始之后立即停止录制时，录制的系统屏幕会出现错误，但是在再次调用startcapture之后似乎可以正常工作.
It is true that the system screen recorded gives error when we stop recording right after the start, but it seem to work alright after we call startcapture again.
我还遇到了一种情况，我的视频缓冲区中没有视频 仅适用于应用程序，系统屏幕录像机可以正常运行，将更新 解决方案很快.
I have also encountered a scenario where I don't get video buffer in my app only and the system screen recorder works fine, will update the solution soon.
更新2
这是问题所在，我的实际应用程序很旧，正在维护中，并及时更新.当replaykit出错时，我的原始应用程序无法接收视频缓冲区，我不知道是否存在使这种情况发生的配置，或者是什么?
So here is the issue, My actual app is old and it is being maintained and getting updated timely. When the replaykit becomes erroneous, My original app can't receive video buffers, I don't know if there is a configuration that is making this happen, or what?
但是新的示例应用程序似乎可以正常工作，并且在重播工具包变得错误之后.当我下次调用startCapture时，重播工具包就可以了. 怪异
But new sample app seem to work fine and after replay kit becomes erroneous. when I call startCapture next time, the replay kit becomes fine. Weird
更新3
我观察到新问题.当权限警报显示时，该应用程序将进入后台.由于我已经编码为每当应用程序进入后台时，都会发生一些UI更改，并且记录将停止. 这导致了
I observed new issue. When the permission alert shows up, the app goes to background. Since I coded that whenever the app goes to background, some UI changes will occur and the recording will be stopped. This led to the error of
由于多任务处理和内容调整大小而中断了录制
Recording interrupted by multitasking and content resizing
我还不确定，是哪个特定的UI更改导致了此失败，但是只有在出现权限警报并进行UI更改时才会出现 . 如果有人注意到此问题的任何特殊情况，请告诉我们.
I am not yet certain, which particular UI change is creating this failure, but it only comes when permission alert shows up and the UI changes are made. If someone has noticed any particular case for this issue, please let us know.
这篇关于Replaykit，startCaptureWithHandler()未在captureHandler中发送视频类型的CMSampleBufferRef的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
