媒体codeC与表面输入：生产分块输出 [英] MediaCodec with Surface input: Producing chunked output

问题描述

我想通过来产生从摄像机preVIEW数据短期连续的MP4文件的媒体codec.createInputSurface（）。但是，重新创建媒体codeC 和它相关的表面需要停止的相机，让另一个呼叫 mCamera.set previewTexture（。 ..）。这种延迟会导致丢帧无法接受的金额。

I'm trying to produce short sequential mp4 files from CameraPreview data via MediaCodec.createInputSurface(). However, recreating the MediaCodec and it's associated Surface requires stopping the Camera to allow another call to mCamera.setPreviewTexture(...). This delay results in an unacceptable amount of dropped frames.

因此​​我需要定期生成 codeC_CONFIG 和 END_OF_STREAM 数据，而无需重新输入表面，和因此不必调用 mCamera.set previewTexture（...）。这是可能的假设 MediaFormat 是不变的？

Therefore I need to generate the CODEC_CONFIG and END_OF_STREAM data periodically without recreating the input Surface, and thus having to call mCamera.setPreviewTexture(...). Is this possible assuming the MediaFormat is unchanged?

（我适应法登的 CameraToMpegTest 的例子。我完全code是<一个href="https://github.com/OnlyInAmerica/HWEn$c$crExperiments/blob/chunked2/HWEn$c$crExperiments/src/main/java/net/openwatch/hwen$c$crexperiments/ChunkedHWRecorder.java"相对=nofollow>这里）

(I'm adapting fadden's CameraToMpegTest example. My complete code is here)

不成功的尝试：

调用媒体codec.signalEndOfInputStream（），排水媒体codeC ，然后调用媒体codec.flush（）块之间产生 IllegalStateException异常 2日呼吁媒体codec.signalEndOfInputStream（）。

Calling MediaCodec.signalEndOfInputStream(), draining the MediaCodec, and then calling MediaCodec.flush() between chunks produces an IllegalStateException on the 2nd call to MediaCodec.signalEndOfInputStream().

调用媒体codec.signalEndOfInputStream（），排水媒体codeC ，然后调用媒体codec.stop（）;媒体codec.configure（...），媒体codec.start（）块之间不会再次调用媒体codec.createInputSurface（）生成以下错误：

Calling MediaCodec.signalEndOfInputStream(), draining the MediaCodec, and then calling MediaCodec.stop(); MediaCodec.configure(...), MediaCodec.start() between chunks without again calling MediaCodec.createInputSurface() produces the following error:

    09-30 13:12:49.889  17638-17719/x.xx.xxxx E/Surface﹕ queueBuffer: error queuing buffer to SurfaceTexture, -19
09-30 13:12:49.889  17638-17719/x.xx.xxxx E/IMGSRV﹕ :0: UnlockPostBuffer: Failed to queue buffer 0x592e1e70
09-30 13:12:49.889  17638-17719/x.xx.xxxx E/CameraToMpegTest﹕ Encoding loop exception!
09-30 13:12:49.889  17638-17719/x.xx.xxxx W/System.err﹕ java.lang.RuntimeException: eglSwapBuffers: EGL error: 0x300b
09-30 13:12:49.896  17638-17719/x.xx.xxxx W/System.err﹕ at x.xx.xxxx.ChunkedHWRecorder$CodecInputSurface.checkEglError(ChunkedHWRecorder.java:731)
09-30 13:12:49.896  17638-17719/x.xx.xxxx W/System.err﹕ at x.xx.xxxx.ChunkedHWRecorder$CodecInputSurface.swapBuffers(ChunkedHWRecorder.java:713)
09-30 13:12:49.896  17638-17719/x.xx.xxxx W/System.err﹕ at x.xx.xxxx.ChunkedHWRecorder.startRecording(ChunkedHWRecorder.java:164)
09-30 13:12:49.896  17638-17719/x.xx.xxxx W/System.err﹕ at x.xx.xxxx.HWRecorderActivity$CameraToMpegWrapper.run(HWRecorderActivity.java:76)
09-30 13:12:49.896  17638-17719/x.xx.xxxx W/System.err﹕ at java.lang.Thread.run(Thread.java:841)

解决感谢法登。完整的解决方案来源是<一个href="https://github.com/OnlyInAmerica/HWEn$c$crExperiments/blob/chunked2/HWEn$c$crExperiments/src/main/java/net/openwatch/hwen$c$crexperiments/ChunkedHWRecorder.java"相对=nofollow>这里。

Solved Thanks fadden. The complete solution source is here.

推荐答案

在 signalEndOfInputStream（）呼叫更新在媒体codeC协议栈各层的状态。你可以得到一个什么样的操作是从上面的<一个测试的意见有效某种意义上href="https://android.googlesource.com/platform/cts/+/jb-mr2-release/tests/tests/media/src/android/media/cts/Media$c$ccTest.java">Media$c$ccTest,但总的来说，对媒体codeC的行为根本就没有定义的不寻常的使用。

The signalEndOfInputStream() call updates the state of various layers in the MediaCodec stack. You can get some sense of what operations are valid from the comments above the tests in MediaCodecTest, but by and large the behavior of MediaCodec is simply not defined for "unusual" uses.

所以，你必须看看code。输入表面的使用寿命是绑到的<一个href="https://android.googlesource.com/platform/frameworks/av/+/jb-mr2-release/media/libstagefright/include/OMXNodeInstance.h">OMXNodeInstance;它是由<一个psented重新$ P $ href="https://android.googlesource.com/platform/frameworks/av/+/jb-mr2-release/media/libstagefright/omx/GraphicBufferSource.h">GraphicBufferSource.一旦信号EOS的GraphicBufferSource会忽略其它帧（请参阅<一href="https://android.googlesource.com/platform/frameworks/av/+/jb-mr2-release/media/libstagefright/omx/GraphicBufferSource.cpp">line 426 ）。有没有办法重新设置EOS标志不拆除GraphicBufferSource，但是当你这样做，它断开缓冲队列underlies表面。

So you have to look at the code. The lifetime of the input surface is tied to that of the OMXNodeInstance; it's represented by GraphicBufferSource. Once you signal EOS, the GraphicBufferSource will ignore additional frames (see line 426). There's no way to reset the EOS flag without tearing down the GraphicBufferSource, but when you do that it disconnects the buffer queue that underlies the Surface.

所以，我不认为你将能够停止/重新启动介质codeC和继续使用的表面。

So I don't think you're going to be able to stop/restart the MediaCodec and continue to use the Surface.

不过......你不应该需要。 CameraToMpegTest路线相机preVIEW到表面纹理，然后使纹理到与GLES的EN codeR的输入表面。所述表面纹理是从烯codeR解耦并且不应需要改变。我认为，需要改变的是codecInputSurface，这就要求 eglCreateWindowSurface（）从媒体codeC的面告诉GLES在哪里画。如果添加了新的更新面API有（销毁旧EGLSurface，创造新的EGLSurface，eglMakeCurrent），并调用它，只要你旋转了一个新的媒体codeC，我的认为的，它会一切都只是工作。

However... you shouldn't need to. CameraToMpegTest routes the camera preview to a SurfaceTexture, and then renders the texture onto the encoder's input surface with GLES. The SurfaceTexture is decoupled from the encoder and shouldn't need to change. I think what needs to change is CodecInputSurface, which calls eglCreateWindowSurface() with the Surface from the MediaCodec to tell GLES where to draw. If you add a new "update Surface" API there (destroy old EGLSurface, create new EGLSurface, eglMakeCurrent), and call it whenever you spin up a new MediaCodec, I think it'll all just work.

更新，以解决意见

这一点很重要，你只能修改 EGLSurface 。在<一个的 checkAndUpdateEglStateLocked（）功能href="https://android.googlesource.com/platform/frameworks/native/+/jb-mr2-release/libs/gui/GLConsumer.cpp">GLConsumer.cpp检查以确保 EGLDisplay 和 EGLContext 一旦被设置不会改变。你可以不叫发布（） / eglSetup（）在codecInputSurface，因为它改变了 EGLContext 。你只是想破坏并重新创建 EGLSurface 。

It's important that you only change the EGLSurface. The checkAndUpdateEglStateLocked() function in GLConsumer.cpp checks to make sure the EGLDisplay and EGLContext don't change once they've been set. You can't call release()/eglSetup() in CodecInputSurface because it changes the EGLContext. You just want to destroy and recreate the EGLSurface.

这篇关于媒体codeC与表面输入：生产分块输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！