Android使用渲染脚本将ImageReader Image转换为YCbCr_420_SP(NV21)字节数组? [英] Android convert ImageReader Image to YCbCr_420_SP (NV21) byte array using render script?
问题描述
我目前正在使用 Javacv ,它使用了 public void onPreviewFrame(byte []数据,相机相机)
相机功能。
I am currently working with Javacv which makes use of the public void onPreviewFrame(byte[] data, Camera camera)
camera function.
由于相机已被弃用,我一直在研究 camera2 和 MediaProjection 。这两个库都使用了 ImageReader类。
Since camera is deprecated, I have been looking into camera2 and MediaProjection. Both of these libraries make use of the ImageReader class.
目前我使用以下代码实例化这样的 ImageReader
:
Currently I instantiate such an ImageReader
with the following code:
ImageReader.newInstance(DISPLAY_WIDTH,DISPLAY_HEIGHT,PixelFormat.RGBA_8888,2);
并附上 OnImageAvailableListener
像这样:
private final ImageReader.OnImageAvailableListener mOnImageAvailableListener
= new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(ImageReader reader) {
mBackgroundHandler.post(new processImage(reader.acquireNextImage()));
}
我尝试过使用 RGBA_8888
按照此主题格式使用Javacv格式: https://github.com/bytedeco/javacv / issues / 298 但这对我不起作用。
I have tried using the RGBA_8888
format with Javacv as per this thread: https://github.com/bytedeco/javacv/issues/298 but that doesn't work for me.
所以我正在考虑使用渲染脚本以转换这些图片到 NV21(YUV_420_SP)
格式(这是<$ c中相机的默认输出$ c> onPreviewFrame function)因为我使用相机
库。
So instead I was thinking about using Render script to convert these Images to NV21(YUV_420_SP)
format (which is the default output of camera in the onPreviewFrame
function) as that worked for me with the camera
library.
我还阅读了这个和这个网站进行转换,但这些对我不起作用,我担心它们会太慢。
此外,我对C的了解非常有限。
基本上看起来我想要 https://的反向操作developer.android.com/reference/android/renderscript/ScriptIntrinsicYuvToRGB.html
I have also read posts such as this one and this website to do the conversion, but these didn't work for me and I fear they will be too slow. Furthermore, my knowledge of C is severely limited. Basically it looks like I want the reverse operation of https://developer.android.com/reference/android/renderscript/ScriptIntrinsicYuvToRGB.html
那你怎么能从图片
到一个字节数组,它匹配 onPreviewFrame
函数的输出,即 NV21(YUV_420_SP)
格式?最好使用Renderscript,因为它更快。
So how can you go from an Image
to a byte array that would match the output of the onPreviewFrame
function i.e. NV21(YUV_420_SP)
format? Preferably using Renderscript as it's faster.
我尝试过使用 ImageFormat.YUV_420_888
,但无济于事。我一直收到像这样的错误。生产者输出缓冲区格式0x1与ImageReader配置的缓冲区格式
不匹配。我切换回 PixelFormat.RGBA_8888
,发现 ImageObject
中只有一架飞机。该平面的字节缓冲区大小为宽*高* 4(分别为R,G,B,A的一个字节)。所以我尝试将其转换为 NV21
格式。
I have tried using ImageFormat.YUV_420_888
but to no avail. I kept getting errors like The producer output buffer format 0x1 doesn't match the ImageReader's configured buffer format
. I switched back to PixelFormat.RGBA_8888
and discovered that there is only one plane in the ImageObject
. The byte buffer of this plane is of size width*height*4 (one byte for R,G,B,A respectively). So I tried to convert this to NV21
format.
我修改了代码来自这个答案产生以下功能:
I modified code from this answer to produce the following function:
void RGBtoNV21(byte[] yuv420sp, byte[] argb, int width, int height) {
final int frameSize = width * height;
int yIndex = 0;
int uvIndex = frameSize;
int A, R, G, B, Y, U, V;
int index = 0;
int rgbIndex = 0;
for (int i = 0; i < height; i++) {
for (int j = 0; j < width; j++) {
R = argb[rgbIndex++];
G = argb[rgbIndex++];
B = argb[rgbIndex++];
A = argb[rgbIndex++];
// RGB to YUV conversion according to
// https://en.wikipedia.org/wiki/YUV#Y.E2.80.B2UV444_to_RGB888_conversion
Y = ((66 * R + 129 * G + 25 * B + 128) >> 8) + 16;
U = ((-38 * R - 74 * G + 112 * B + 128) >> 8) + 128;
V = ((112 * R - 94 * G - 18 * B + 128) >> 8) + 128;
// NV21 has a plane of Y and interleaved planes of VU each sampled by a factor
// of 2 meaning for every 4 Y pixels there are 1 V and 1 U.
// Note the sampling is every other pixel AND every other scanline.
yuv420sp[yIndex++] = (byte) ((Y < 0) ? 0 : ((Y > 255) ? 255 : Y));
if (i % 2 == 0 && index % 2 == 0) {
yuv420sp[uvIndex++] = (byte) ((V < 0) ? 0 : ((V > 255) ? 255 : V));
yuv420sp[uvIndex++] = (byte) ((U < 0) ? 0 : ((U > 255) ? 255 : U));
}
index++;
}
}
}
并使用以下方式调用它:
and invoke it using:
int mWidth = mImage.getWidth();
int mHeight = mImage.getHeight();
byte[] rgbaBytes = new byte[mWidth * mHeight * 4];
mImage.getPlanes()[0].getBuffer().get(rgbaBytes);
mImage.close();
byte[] yuv = new byte[mWidth * mHeight * 3 / 2];
RGBtoNV21(yuv, rgbaBytes, mWidth, mHeight);
此处 mImage
是图像
我的 ImageReader产生的对象
。
然而这会产生类似的结果对此图片:
Yet this produces a result similar to this image:
显然是畸形的。好像我的转换已关闭,但我无法确定究竟是什么。
which is clearly malformed. Seems like my conversion is off but I cannot figure what exactly.
推荐答案
@TargetApi(19)
public static byte[] yuvImageToByteArray(Image image) {
assert(image.getFormat() == ImageFormat.YUV_420_888);
int width = image.getWidth();
int height = image.getHeight();
Image.Plane[] planes = image.getPlanes();
byte[] result = new byte[width * height * 3 / 2];
int stride = planes[0].getRowStride();
if (stride == width) {
planes[0].getBuffer().get(result, 0, width);
}
else {
for (int row = 0; row < height; row++) {
planes[0].getBuffer().position(row*stride);
planes[0].getBuffer().get(result, row*width, width);
}
}
stride = planes[1].getRowStride();
assert (stride == planes[2].getRowStride());
byte[] rowBytesCb = new byte[stride];
byte[] rowBytesCr = new byte[stride];
for (int row = 0; row < height/2; row++) {
int rowOffset = width*height + width/2 * row;
planes[1].getBuffer().position(row*stride);
planes[1].getBuffer().get(rowBytesCb, 0, width/2);
planes[2].getBuffer().position(row*stride);
planes[2].getBuffer().get(rowBytesCr, 0, width/2);
for (int col = 0; col < width/2; col++) {
result[rowOffset + col*2] = rowBytesCr[col];
result[rowOffset + col*2 + 1] = rowBytesCb[col];
}
}
return result;
}
这篇关于Android使用渲染脚本将ImageReader Image转换为YCbCr_420_SP(NV21)字节数组?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!