如何利用媒体codeC摆脱视频位图（帧） [英] How to get bitmap (frames) from video using MediaCodec
问题描述
我试图让使用Media codeC从视频文件的所有帧。如果我尝试SurfaceView显示视频，一切正常。但是，如果表面为空，当我试图摆脱字节数组位图，alwaus GET null或运行时异常。
I'm trying to get all frames from video file using MediaCodec. If I try display video on SurfaceView, everything is ok. But if surface is null, and when I try get Bitmap from byte array, alwaus get null or runtime exception.
这是我的code：
private class PlayerThread extends Thread {
private MediaExtractor extractor;
private MediaCodec decoder;
private Surface surface;
public PlayerThread(Surface surface) {
this.surface = surface;
}
@Override
public void run() {
extractor = new MediaExtractor();
extractor.setDataSource(videoPath);
for (int i = 0; i < extractor.getTrackCount(); i++) {
MediaFormat format = extractor.getTrackFormat(i);
String mime = format.getString(MediaFormat.KEY_MIME);
if (mime.startsWith("video/")) {
extractor.selectTrack(i);
decoder = MediaCodec.createDecoderByType(mime);
decoder.configure(format, /*surface*/ null, null, 0);
break;
}
}
if (decoder == null) {
Log.e("DecodeActivity", "Can't find video info!");
return;
}
decoder.start();
ByteBuffer[] inputBuffers = decoder.getInputBuffers();
ByteBuffer[] outputBuffers = decoder.getOutputBuffers();
BufferInfo info = new BufferInfo();
boolean isEOS = false;
while (!Thread.interrupted()) {
++numFrames;
if (!isEOS) {
int inIndex = decoder.dequeueInputBuffer(10000);
if (inIndex >= 0) {
ByteBuffer buffer = inputBuffers[inIndex];
int sampleSize = extractor.readSampleData(buffer, 0);
if (sampleSize < 0) {
// We shouldn't stop the playback at this point,
// just pass the EOS
// flag to decoder, we will get it again from the
// dequeueOutputBuffer
Log.d("DecodeActivity",
"InputBuffer BUFFER_FLAG_END_OF_STREAM");
decoder.queueInputBuffer(inIndex, 0, 0, 0,
MediaCodec.BUFFER_FLAG_END_OF_STREAM);
isEOS = true;
} else {
decoder.queueInputBuffer(inIndex, 0, sampleSize,
extractor.getSampleTime(), 0);
extractor.advance();
}
}
}
int outIndex = decoder.dequeueOutputBuffer(info, 10000);
switch (outIndex) {
case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
Log.d("DecodeActivity", "INFO_OUTPUT_BUFFERS_CHANGED");
outputBuffers = decoder.getOutputBuffers();
break;
case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
Log.d("DecodeActivity",
"New format " + decoder.getOutputFormat());
break;
case MediaCodec.INFO_TRY_AGAIN_LATER:
Log.d("DecodeActivity", "dequeueOutputBuffer timed out!");
break;
default:
// I tried get Bitmap on few ways
//1.
//ByteBuffer buffer = outputBuffers[outIndex];
//byte[] ba = new byte[buffer.remaining()];
//buffer.get(ba);
//final Bitmap bmp = BitmapFactory.decodeByteArray(ba, 0, ba.length);// this return null
//2.
//ByteBuffer buffer = outputBuffers[outIndex];
//final Bitmap bmp = Bitmap.createBitmap(1920, 1080, Config.ARGB_8888);//using MediaFormat object I know width and height
//int a = bmp.getByteCount(); //8294400
//buffer.rewind();
//int b = buffer.capacity();//3137536
//int c = buffer.remaining();//3137536
//bmp.copyPixelsFromBuffer(buffer); // java.lang.RuntimeException: Buffer not large enough for pixels
//I know what exception mean, but i don't know why xception occurs
//In this place I need bitmap
// We use a very simple clock to keep the video FPS, or the
// video
// playback will be too fast
while (info.presentationTimeUs / 1000 > System
.currentTimeMillis() - startMs) {
try {
sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
break;
}
}
decoder.releaseOutputBuffer(outIndex, true);
break;
}
// All decoded frames have been rendered, we can stop playing
// now
if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
Log.d("DecodeActivity",
"OutputBuffer BUFFER_FLAG_END_OF_STREAM");
break;
}
}
decoder.stop();
decoder.release();
extractor.release();
}
}
我不知道如何解决它。
I have no idea how to solve it.
萨摩斯
推荐答案
有几个与你的code的问题（或者，可以说，与媒体codeC）。
There are a couple of problems with your code (or, arguably, with MediaCodec).
首先，ByteBuffer的在媒体codeC处理不好，所以你必须手动设置从由填写在BufferInfo对象的值的缓冲参数dequeueOutputBuffer（）。
First, the ByteBuffer handling in MediaCodec is poor, so you have to manually set the buffer parameters from the values in the BufferInfo object that is filled out by dequeueOutputBuffer().
其次，出来的媒体codeC的值在YUV格式，而不是RGB。随着Android 4.4系统的，Android框架没有提供，将转换输出为位图的功能。您需要提供自己的YUV到RGB转换器（复数 - 有多种格式）。某些设备使用专有的，无证的色彩格式。
Second, the values that come out of the MediaCodec are in YUV format, not RGB. As of Android 4.4, the Android framework does not provide a function that will convert the output to Bitmap. You will need to provide your own YUV-to-RGB converters (plural -- there are multiple formats). Some devices use proprietary, undocumented color formats.
您可以看到在CTS提取和检查媒体codeC德codeR缓冲区内容的恩codeDE codeTEST 缓冲区到缓冲方法（如 checkFrame（） ）。
You can see an example of extracting and examining MediaCodec decoder buffer contents in the CTS EncodeDecodeTest buffer-to-buffer methods (e.g. checkFrame()).
一个更可靠的方式去了解，这是要回去解码到表面，但与提取的OpenGL ES的像素。该bigflake ExtractMpegFramesTest 展示了如何做到这一点。
A more reliable way to go about this is to go back to decoding to a Surface, but extract the pixels with OpenGL ES. The bigflake ExtractMpegFramesTest shows how to do this.
这篇关于如何利用媒体codeC摆脱视频位图（帧）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
