MediaCodec.configure因1080p视频的IllegalStateException失败 [英] MediaCodec.configure fails with IllegalStateException for 1080p videos

问题描述

我一直在使用grafika的 MoviePlayer 和bigFlake的 ExtractMpegFramesTest 来在我们的应用中实现搜索和提取框架功能.这些对于我们大多数用户来说都可以正常工作，但是其中一些在设置ExtractMpegFramesTest时在MediaCodec.configure上遇到IllegalStateException(在Samsung Galaxy S4 mini，Samsung Galaxy J7，Samsung Galaxy A5，Huawei Ascend G7上会发生这种情况) .相关代码如下:

I've been using grafika's MoviePlayer and bigFlake's ExtractMpegFramesTest to implement a seeking and extract frame feature in our app. These work fine for most of our user, however some of them encounter a IllegalStateException on MediaCodec.configure when setting up the ExtractMpegFramesTest (this happens on Samsung Galaxy S4 mini, Samsung Galaxy J7, Samsung Galaxy A5, Huawei Ascend G7). The relevant code is as the following:

MoviePlayer

public void prepareResources() throws IOException {
    // The MediaExtractor error messages aren't very useful.  Check to see if the input
    // file exists so we can throw a better one if it's not there.
    if (!mSourceFile.canRead()) {
        throw new FileNotFoundException("Unable to read " + mSourceFile);
    }

    try {
        mExtractor = new MediaExtractor();
        mExtractor.setDataSource(mSourceFile.toString());
        mTrackIndex = selectTrack(mExtractor);
        if (mTrackIndex < 0) {
            throw new RuntimeException("No video track found in " + mSourceFile);
        }
        mExtractor.selectTrack(mTrackIndex);

        MediaFormat format = mExtractor.getTrackFormat(mTrackIndex);

        // Create a MediaCodec decoder, and configure it with the MediaFormat from the
        // extractor.  It's very important to use the format from the extractor because
        // it contains a copy of the CSD-0/CSD-1 codec-specific data chunks.
        String mime = format.getString(MediaFormat.KEY_MIME);
        mDecoder = MediaCodec.createDecoderByType(mime);
        mDecoder.configure(format, mOutputSurface, null, 0);
        mDecoder.start();
        mDecoderInputBuffers = mDecoder.getInputBuffers();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

ExtractMpegFramesTest

private void extractMpegFrames() throws IOException {
    MediaCodec decoder = null;
    CodecOutputSurface outputSurface = null;
    MediaExtractor extractor = null;
    /*int saveWidth = 640;
    int saveHeight = 480;*/

    try {
        long start1 = System.currentTimeMillis();

        File inputFile = new File(mInputVideoPath);   // must be an absolute path
        // The MediaExtractor error messages aren't very useful.  Check to see if the input
        // file exists so we can throw a better one if it's not there.
        if (!inputFile.canRead()) {
            throw new FileNotFoundException("Unable to read " + inputFile);
        }

        extractor = new MediaExtractor();
        extractor.setDataSource(inputFile.toString());
        int trackIndex = selectTrack(extractor);
        if (trackIndex < 0) {
            throw new RuntimeException("No video track found in " + inputFile);
        }
        extractor.selectTrack(trackIndex);

        MediaFormat format = extractor.getTrackFormat(trackIndex);
        if (VERBOSE) {
            Log.d(TAG, "Video size is " + format.getInteger(MediaFormat.KEY_WIDTH) + "x" +
                    format.getInteger(MediaFormat.KEY_HEIGHT));
        }

        // Could use width/height from the MediaFormat to get full-size frames.
        outputSurface = new CodecOutputSurface(format.getInteger(MediaFormat.KEY_WIDTH), format.getInteger(MediaFormat.KEY_HEIGHT));

        // Create a MediaCodec decoder, and configure it with the MediaFormat from the
        // extractor.  It's very important to use the format from the extractor because
        // it contains a copy of the CSD-0/CSD-1 codec-specific data chunks.
        String mime = format.getString(MediaFormat.KEY_MIME);
        decoder = MediaCodec.createDecoderByType(mime);
        decoder.configure(format, outputSurface.getSurface(), null, 0); //fails right here
        decoder.start();

        long end1 = System.currentTimeMillis();
        Timber.d("extractMpegFrames(): FRAME_EXTRACT setup in: %d millis", (end1-start1));

        long start2 = System.currentTimeMillis();
        doExtract(extractor, trackIndex, decoder, outputSurface);
        long end2 = System.currentTimeMillis();
        Timber.d("extractMpegFrames(): FRAME_EXTRACT doExtract in: %d millis", (end2-start2));
    } finally {
        // release everything we grabbed
        if (outputSurface != null) {
            outputSurface.release();
            outputSurface = null;
        }
        if (decoder != null) {
            decoder.stop();
            decoder.release();
            decoder = null;
        }
        if (extractor != null) {
            extractor.release();
            extractor = null;
        }
    }
}

我知道这个问题，但是我不明白的是为什么可以配置MediaCodec在MoviePlayer中确定(视频运行正常)，但在ExtractMpegFramesTest中失败.首先，我认为设备在OpenGL设置方面存在一些问题，但事实证明这是MediaCodec问题.

I'm aware of this question, however what I don't understand is why MediaCodec can be configured OK in MoviePlayer (the video runs just fine) but it fails in ExtractMpegFramesTest. At first, I think the device has some issues with the OpenGL set up, but it turn out to be MediaCodec problem.

任何见识将不胜感激.

Any insight would be greatly appreciated.

编辑:获得Galaxy S4 mini测试设备后，我已经获得了更有用的日志:

After obtaining a Galaxy S4 mini testing device, I've been able to get a much more helpful log:

D/MoviePlayer: Extractor selected track 0 (video/avc): {max-input-size=1572864, height=1080, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20], width=1920, durationUs=5131211, csd-1=java.nio.ByteArrayBuffer[position=0,limit=9,capacity=9], mime=video/avc, isDMCMMExtractor=1}
D/MoviePlayer: Video size is 1920x1080
D/MoviePlayer: Extractor selected track 0 (video/avc): {max-input-size=1572864, height=1080, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20], width=1920, durationUs=5131211, csd-1=java.nio.ByteArrayBuffer[position=0,limit=9,capacity=9], mime=video/avc, isDMCMMExtractor=1}
I/OMXClient: Using client-side OMX mux.
E/ACodec: [OMX.qcom.video.decoder.avc] storeMetaDataInBuffers failed w/ err -2147483648
E/ACodec:  configureCodec multi window instance fail  appPid : 3283
E/ACodec: [OMX.qcom.video.decoder.avc] configureCodec returning error -38
E/MediaCodec: Codec reported an error. (omx error 0x80001001, internalError -38)
W/System.err: java.lang.IllegalStateException
W/System.err:     at android.media.MediaCodec.native_configure(Native Method)
W/System.err:     at android.media.MediaCodec.configure(MediaCodec.java:262)
W/System.err:     at co.test.testing.player.MoviePlayer.prepareResources(MoviePlayer.java:237)
W/System.err:     at co.test.testing.activities.VideoActivity.surfaceCreated(VideoActivity.java:276)
W/System.err:     at android.view.SurfaceView.updateWindow(SurfaceView.java:602)
W/System.err:     at android.view.SurfaceView.access$000(SurfaceView.java:94)
W/System.err:     at android.view.SurfaceView$3.onPreDraw(SurfaceView.java:183)
W/System.err:     at android.view.ViewTreeObserver.dispatchOnPreDraw(ViewTreeObserver.java:891)
W/System.err:     at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:2201)
W/System.err:     at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:1256)
W/System.err:     at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:6635)
W/System.err:     at android.view.Choreographer$CallbackRecord.run(Choreographer.java:813)
W/System.err:     at android.view.Choreographer.doCallbacks(Choreographer.java:613)
W/System.err:     at android.view.Choreographer.doFrame(Choreographer.java:583)
W/System.err:     at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:799)
W/System.err:     at android.os.Handler.handleCallback(Handler.java:733)
W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:95)
W/System.err:     at android.os.Looper.loop(Looper.java:146)
W/System.err:     at android.app.ActivityThread.main(ActivityThread.java:5593)
W/System.err:     at java.lang.reflect.Method.invokeNative(Native Method)
W/System.err:     at java.lang.reflect.Method.invoke(Method.java:515)
W/System.err:     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283)
W/System.err:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099)
W/System.err:     at dalvik.system.NativeStart.main(Native Method)

这些是从出现此问题的设备中检索到的编解码器信息:

These are the codec info retrieved from devices having this problem:

• OMX.qcom.video.encoder.avc

• OMX.qcom.video.encoder.avc

OMX.SEC.avc.enc

OMX.SEC.avc.enc

OMX.Exynos.AVC.Encoder

OMX.Exynos.AVC.Encoder

奇怪的是，Galaxy S4 mini可以很好地处理720p视频，而1080p视频只是有问题.

The strange thing is the Galaxy S4 mini can handle 720p videos just fine, it's just having problem with 1080p videos.

推荐答案

好，这里再次回答我自己的问题:

Ok, here to answer my own question again:

• 首先，为什么在使用MoviePlayer显示视频后为什么不能使用ExtractMpegFramesTest提取帧:

• Firstly, why I can't extract frame with ExtractMpegFramesTest after using MoviePlayer to display videos:

似乎某些设备无法同时处理2个MediaCodec实例的高分辨率视频(对于Galaxy S4 mini，> 720p)，因此您必须正确释放一个MediaCodec实例在开始另一个之前，是这样的:

It appears that some devices can't handle 2 instances of MediaCodec at the same time for high res videos (>720p in the case of the Galaxy S4 mini) so you have to properly release one instance of MediaCodec before starting the other one, something like this:

public void releaseResources() {
    // release everything we grabbed
    if (mDecoder != null) {
        try {
            mDecoder.stop();
            mDecoder.release();
            mDecoder = null;
        } catch (Exception e) {
            Timber.d("releaseResources(): message %s cause %s", e.getMessage(),  e.getCause());
            e.printStackTrace();
        }
    }
    if (mExtractor != null) {
        try {
            mExtractor.release();
            mExtractor = null;
        } catch (Exception e) {
            Timber.d("releaseResources(): message %s cause %s", e.getMessage(),  e.getCause());
            e.printStackTrace();
        }
    }
}

第二，为什么MoviePlayer无法为1080p视频配置.就我而言，因为我是从媒体选择器活动开始的，所以该活动的预览用VideoView进行注册，其中注册了MediaPlayer.OnErrorListener()以检测错误的视频.问题是onError()回调在Galaxy S4 mini上有一些非常奇怪的行为，而在我的开发设备上却可以正常工作.我很奇怪，此MediaPlayer.OnErrorListener()使MediaCodec处于不良状态，以后会引起很多头痛.

Secondly, why the MoviePlayer fails to configure for 1080p videos. In my case, since I start this activity from a media picker activity with a VideoView for preview which has a MediaPlayer.OnErrorListener() registered to detect faulty video. The problem is the onError() callback has some really weird behavior on the Galaxy S4 mini while it work perfectly fine on my development devices. My wild guess this MediaPlayer.OnErrorListener() leaves the MediaCodec in a bad state that cause a lot of headache later on.

因此解决方案是设置空白的OnErrorListener()并正确释放VideoView，然后再使用MediaCodec进行其他操作.像这样:

So the solution is to set a blank OnErrorListener() and properly release the VideoView before doing anything else with MediaCodec. Something like this:

mVideoView.setOnErrorListener(new MediaPlayer.OnErrorListener() {
        @Override
        public boolean onError(MediaPlayer mp, int what, int extra) {
            return true;
        }
});
.....
mVideoView.stopPlayback();
doOtherThingWithMediaCodec();

这说明了在不同设备上发生的许多奇怪的事情.我希望这可以帮助其他人调试他们的应用程序.

This explain a lot of weird thing that happen on different devices. I hope this help someone else debug their app.

这篇关于MediaCodec.configure因1080p视频的IllegalStateException失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！