如何使用 MediaCodec 从视频中获取位图(帧) [英] How to get bitmap (frames) from video using MediaCodec
问题描述
我正在尝试使用 MediaCodec 从视频文件中获取所有帧.如果我尝试在 SurfaceView 上显示视频，则一切正常.但是如果表面为空，并且当我尝试从字节数组中获取位图时，总是会获取空值或运行时异常.
I'm trying to get all frames from video file using MediaCodec. If I try display video on SurfaceView, everything is ok. But if surface is null, and when I try get Bitmap from byte array, alwaus get null or runtime exception.
这是我的代码:
private class PlayerThread extends Thread {
private MediaExtractor extractor;
private MediaCodec decoder;
private Surface surface;
public PlayerThread(Surface surface) {
this.surface = surface;
}
@Override
public void run() {
extractor = new MediaExtractor();
extractor.setDataSource(videoPath);
for (int i = 0; i < extractor.getTrackCount(); i++) {
MediaFormat format = extractor.getTrackFormat(i);
String mime = format.getString(MediaFormat.KEY_MIME);
if (mime.startsWith("video/")) {
extractor.selectTrack(i);
decoder = MediaCodec.createDecoderByType(mime);
decoder.configure(format, /*surface*/ null, null, 0);
break;
}
}
if (decoder == null) {
Log.e("DecodeActivity", "Can't find video info!");
return;
}
decoder.start();
ByteBuffer[] inputBuffers = decoder.getInputBuffers();
ByteBuffer[] outputBuffers = decoder.getOutputBuffers();
BufferInfo info = new BufferInfo();
boolean isEOS = false;
while (!Thread.interrupted()) {
++numFrames;
if (!isEOS) {
int inIndex = decoder.dequeueInputBuffer(10000);
if (inIndex >= 0) {
ByteBuffer buffer = inputBuffers[inIndex];
int sampleSize = extractor.readSampleData(buffer, 0);
if (sampleSize < 0) {
// We shouldn't stop the playback at this point,
// just pass the EOS
// flag to decoder, we will get it again from the
// dequeueOutputBuffer
Log.d("DecodeActivity",
"InputBuffer BUFFER_FLAG_END_OF_STREAM");
decoder.queueInputBuffer(inIndex, 0, 0, 0,
MediaCodec.BUFFER_FLAG_END_OF_STREAM);
isEOS = true;
} else {
decoder.queueInputBuffer(inIndex, 0, sampleSize,
extractor.getSampleTime(), 0);
extractor.advance();
}
}
}
int outIndex = decoder.dequeueOutputBuffer(info, 10000);
switch (outIndex) {
case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
Log.d("DecodeActivity", "INFO_OUTPUT_BUFFERS_CHANGED");
outputBuffers = decoder.getOutputBuffers();
break;
case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
Log.d("DecodeActivity",
"New format " + decoder.getOutputFormat());
break;
case MediaCodec.INFO_TRY_AGAIN_LATER:
Log.d("DecodeActivity", "dequeueOutputBuffer timed out!");
break;
default:
// I tried get Bitmap on few ways
//1.
//ByteBuffer buffer = outputBuffers[outIndex];
//byte[] ba = new byte[buffer.remaining()];
//buffer.get(ba);
//final Bitmap bmp = BitmapFactory.decodeByteArray(ba, 0, ba.length);// this return null
//2.
//ByteBuffer buffer = outputBuffers[outIndex];
//final Bitmap bmp = Bitmap.createBitmap(1920, 1080, Config.ARGB_8888);//using MediaFormat object I know width and height
//int a = bmp.getByteCount(); //8294400
//buffer.rewind();
//int b = buffer.capacity();//3137536
//int c = buffer.remaining();//3137536
//bmp.copyPixelsFromBuffer(buffer); // java.lang.RuntimeException: Buffer not large enough for pixels
//I know what exception mean, but i don't know why xception occurs
//In this place I need bitmap
// We use a very simple clock to keep the video FPS, or the
// video
// playback will be too fast
while (info.presentationTimeUs / 1000 > System
.currentTimeMillis() - startMs) {
try {
sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
break;
}
}
decoder.releaseOutputBuffer(outIndex, true);
break;
}
// All decoded frames have been rendered, we can stop playing
// now
if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
Log.d("DecodeActivity",
"OutputBuffer BUFFER_FLAG_END_OF_STREAM");
break;
}
}
decoder.stop();
decoder.release();
extractor.release();
}
}
我不知道如何解决它.
萨摩斯
推荐答案
您的代码(或者，可以说是 MediaCodec)存在一些问题.
There are a couple of problems with your code (or, arguably, with MediaCodec).
首先，MediaCodec 中的 ByteBuffer 处理很差，因此您必须根据 dequeueOutputBuffer() 填充的 BufferInfo 对象中的值手动设置缓冲区参数.
First, the ByteBuffer handling in MediaCodec is poor, so you have to manually set the buffer parameters from the values in the BufferInfo object that is filled out by dequeueOutputBuffer().
其次，MediaCodec 输出的值是 YUV 格式，而不是 RGB.从 Android 4.4 开始，Android 框架不提供将输出转换为 Bitmap 的功能.您需要提供自己的 YUV 到 RGB 转换器(复数 - 有多种格式).某些设备使用专有的、未记录在案的颜色格式.
Second, the values that come out of the MediaCodec are in YUV format, not RGB. As of Android 4.4, the Android framework does not provide a function that will convert the output to Bitmap. You will need to provide your own YUV-to-RGB converters (plural -- there are multiple formats). Some devices use proprietary, undocumented color formats.
您可以在 CTS EncodeDecodeTest 缓冲区到缓冲区方法(例如 checkFrame()).
You can see an example of extracting and examining MediaCodec decoder buffer contents in the CTS EncodeDecodeTest buffer-to-buffer methods (e.g. checkFrame()).
解决此问题的更可靠方法是返回解码到 Surface，但使用 OpenGL ES 提取像素.bigflake ExtractMpegFramesTest 展示了如何做到这一点.
A more reliable way to go about this is to go back to decoding to a Surface, but extract the pixels with OpenGL ES. The bigflake ExtractMpegFramesTest shows how to do this.
这篇关于如何使用 MediaCodec 从视频中获取位图(帧)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
