YUV_420_888解读三星Galaxy S7(相机2) [英] YUV_420_888 interpretation on Samsung Galaxy S7 (Camera2)

问题描述

我写了一个从 YUV_420_888 到 Bitmap 的转换，考虑到以下逻辑(据我所知):

I wrote a conversion from YUV_420_888 to Bitmap, considering the following logic (as I understand it):

总结该方法:内核的坐标 x 和 y 与 Y 平面(2d 分配)的非填充部分的 x 和 y 以及输出位图的 x 和 y 一致.然而，U-Plane 和 V-Plane 具有与 Y-Plane 不同的结构，因为它们使用 1 个字节来覆盖 4 个像素，此外，可能有一个以上的 PixelStride，此外它们可能也有一个可以与 Y 平面不同的填充.因此，为了让内核有效地访问 U 和 V，我将它们放入一维分配中并创建了一个索引uvIndex"，该索引给出了相应的 U 和 V 在该一维分配中的位置，对于给定的(x,y) 坐标(未填充)Y 平面(以及输出位图).

To summarize the approach: the kernel’s coordinates x and y are congruent both with the x and y of the non-padded part of the Y-Plane (2d-allocation) and the x and y of the output-Bitmap. The U- and V-Planes, however, have a different structure than the Y-Plane, because they use 1 byte for coverage of 4 pixels, and, in addition, may have a PixelStride that is more than one, in addition they might also have a padding that can be different from that of the Y-Plane. Therefore, in order to access the U’s and V’s efficiently by the kernel I put them into 1-d allocations and created an index "uvIndex" that gives the position of the corresponding U- and V within that 1-d allocation, for given (x,y) coordinates in the (non-padded) Y-plane (and, so, the output Bitmap).

为了保持 rs-Kernel 精简，我通过 LaunchOptions 限制 x 范围(这反映了 y 平面的 RowStride，因此可以在内核中忽略)，从而排除了 yPlane 中的填充区域.所以我们只需要考虑 uvIndex 中的 uvPixelStride 和 uvRowStride，即用于访问 u 和 v 值的索引.

In order to keep the rs-Kernel lean, I excluded the padding area in the yPlane by capping the x-range via LaunchOptions (this reflects the RowStride of the y-plane which thus can be ignored WITHIN the kernel). So we just need to consider the uvPixelStride and uvRowStride within the uvIndex, i.e. the index used in order to access to the u- and v-values.

这是我的代码:

Renderscript 内核，命名为 yuv420888.rs

Renderscript Kernel, named yuv420888.rs

  #pragma version(1)
  #pragma rs java_package_name(com.xxxyyy.testcamera2);
  #pragma rs_fp_relaxed

  int32_t width;
  int32_t height;

  uint picWidth, uvPixelStride, uvRowStride ;
  rs_allocation ypsIn,uIn,vIn;

 // The LaunchOptions ensure that the Kernel does not enter the padding  zone of Y, so yRowStride can be ignored WITHIN the Kernel.
 uchar4 __attribute__((kernel)) doConvert(uint32_t x, uint32_t y) {

 // index for accessing the uIn's and vIn's
uint uvIndex=  uvPixelStride * (x/2) + uvRowStride*(y/2);

// get the y,u,v values
uchar yps= rsGetElementAt_uchar(ypsIn, x, y);
uchar u= rsGetElementAt_uchar(uIn, uvIndex);
uchar v= rsGetElementAt_uchar(vIn, uvIndex);

// calc argb
int4 argb;
    argb.r = yps + v * 1436 / 1024 - 179;
    argb.g =  yps -u * 46549 / 131072 + 44 -v * 93604 / 131072 + 91;
    argb.b = yps +u * 1814 / 1024 - 227;
    argb.a = 255;

uchar4 out = convert_uchar4(clamp(argb, 0, 255));
return out;
}

Java 端:

    private Bitmap YUV_420_888_toRGB(Image image, int width, int height){
    // Get the three image planes
    Image.Plane[] planes = image.getPlanes();
    ByteBuffer buffer = planes[0].getBuffer();
    byte[] y = new byte[buffer.remaining()];
    buffer.get(y);

    buffer = planes[1].getBuffer();
    byte[] u = new byte[buffer.remaining()];
    buffer.get(u);

    buffer = planes[2].getBuffer();
    byte[] v = new byte[buffer.remaining()];
    buffer.get(v);

    // get the relevant RowStrides and PixelStrides
    // (we know from documentation that PixelStride is 1 for y)
    int yRowStride= planes[0].getRowStride();
    int uvRowStride= planes[1].getRowStride();  // we know from   documentation that RowStride is the same for u and v.
    int uvPixelStride= planes[1].getPixelStride();  // we know from   documentation that PixelStride is the same for u and v.


    // rs creation just for demo. Create rs just once in onCreate and use it again.
    RenderScript rs = RenderScript.create(this);
    //RenderScript rs = MainActivity.rs;
    ScriptC_yuv420888 mYuv420=new ScriptC_yuv420888 (rs);

    // Y,U,V are defined as global allocations, the out-Allocation is the Bitmap.
    // Note also that uAlloc and vAlloc are 1-dimensional while yAlloc is 2-dimensional.
    Type.Builder typeUcharY = new Type.Builder(rs, Element.U8(rs));

    //using safe height
    typeUcharY.setX(yRowStride).setY(y.length / yRowStride);

    Allocation yAlloc = Allocation.createTyped(rs, typeUcharY.create());
    yAlloc.copyFrom(y);
    mYuv420.set_ypsIn(yAlloc);

    Type.Builder typeUcharUV = new Type.Builder(rs, Element.U8(rs));
    // note that the size of the u's and v's are as follows:
    //      (  (width/2)*PixelStride + padding  ) * (height/2)
    // =    (RowStride                          ) * (height/2)
    // but I noted that on the S7 it is 1 less...
    typeUcharUV.setX(u.length);
    Allocation uAlloc = Allocation.createTyped(rs, typeUcharUV.create());
    uAlloc.copyFrom(u);
    mYuv420.set_uIn(uAlloc);

    Allocation vAlloc = Allocation.createTyped(rs, typeUcharUV.create());
    vAlloc.copyFrom(v);
    mYuv420.set_vIn(vAlloc);

    // handover parameters
    mYuv420.set_picWidth(width);
    mYuv420.set_uvRowStride (uvRowStride);
    mYuv420.set_uvPixelStride (uvPixelStride);

    Bitmap outBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
    Allocation outAlloc = Allocation.createFromBitmap(rs, outBitmap, Allocation.MipmapControl.MIPMAP_NONE, Allocation.USAGE_SCRIPT);

    Script.LaunchOptions lo = new Script.LaunchOptions();
    lo.setX(0, width);  // by this we ignore the y’s padding zone, i.e. the right side of x between width and yRowStride
    //using safe height
    lo.setY(0, y.length / yRowStride);

    mYuv420.forEach_doConvert(outAlloc,lo);
    outAlloc.copyTo(outBitmap);

    return outBitmap;
}

在 Nexus 7 (API 22) 上测试这会返回漂亮的彩色位图.但是，该设备具有微不足道的像素步幅(= 1)并且没有填充(即行步幅=宽度).在全新的三星 S7 (API 23) 上测试我得到的图片颜色不正确 - 除了绿色的.但是图片并没有显示出对绿色的普遍偏见，只是似乎没有正确再现非绿色.请注意，S7 应用了 2 的 u/v 像素步幅，并且没有填充.

Testing on Nexus 7 (API 22) this returns nice color Bitmaps. This device, however, has trivial pixelstrides (=1) and no padding (i.e. rowstride=width). Testing on the brandnew Samsung S7 (API 23) I get pictures whose colors are not correct - except of the green ones. But the Picture does not show a general bias towards green, it just seems that non-green colors are not reproduced correctly. Note, that the S7 applies an u/v pixelstride of 2, and no padding.

由于最关键的代码行在 rs-code 中，因此 u/v 平面的访问 uint uvIndex= (...) 我认为，可能存在问题，可能是此处对像素步幅的考虑不正确.有没有人看到解决方案?谢谢.

Since the most crucial code line is within the rs-code the Access of the u/v planes uint uvIndex= (...) I think, there could be the problem, probably with incorrect consideration of pixelstrides here. Does anyone see the solution? Thanks.

更新:我检查了所有内容，我很确定有关 y,u,v 访问的代码是正确的.所以问题一定出在 u 和 v 值本身.非绿色有紫色倾斜，从 u,v 值来看，它们似乎在大约 110-150 的相当窄的范围内.我们真的有可能需要处理特定于设备的 YUV ->RBG 转换...?！我错过了什么吗?

UPDATE: I checked everything, and I am pretty sure that the code regarding the access of y,u,v is correct. So the problem must be with the u and v values themselves. Non green colors have a purple tilt, and looking at the u,v values they seem to be in a rather narrow range of about 110-150. Is it really possible that we need to cope with device specific YUV -> RBG conversions...?! Did I miss anything?

更新 2:已更正代码，现在可以使用了，感谢 Eddy 的反馈.

UPDATE 2: have corrected code, it works now, thanks to Eddy's Feedback.

推荐答案

看

floor((float) uvPixelStride*(x)/2)

从 Y x 坐标计算您的 U、V 行偏移 (uv_row_offset).

which calculates your U,V row offset (uv_row_offset) from the Y x-coordinate.

如果 uvPixelStride = 2，那么随着 x 的增加:

if uvPixelStride = 2, then as x increases:

x = 0, uv_row_offset = 0
x = 1, uv_row_offset = 1
x = 2, uv_row_offset = 2
x = 3, uv_row_offset = 3

这是不正确的.在 uv_row_offset = 1 或 3 处没有有效的 U/V 像素值，因为 uvPixelStride = 2.

and this is incorrect. There's no valid U/V pixel value at uv_row_offset = 1 or 3, since uvPixelStride = 2.

你想要

uvPixelStride * floor(x/2)

(假设您不相信自己会记住整数除法的关键舍入行为，如果您这样做了):

(assuming you don't trust yourself to remember the critical round-down behavior of integer divide, if you do then):

uvPixelStride * (x/2)

应该够了

这样，您的映射就变成了:

With that, your mapping becomes:

x = 0, uv_row_offset = 0
x = 1, uv_row_offset = 0
x = 2, uv_row_offset = 2
x = 3, uv_row_offset = 2

看看是否可以修复颜色错误.实际上，这里的错误寻址意味着所有其他颜色样本都来自错误的颜色平面，因为底层的 YUV 数据很可能是半平面的(因此 U 平面从 V 平面 + 1 个字节开始，两个平面交错)

See if that fixes the color errors. In practice, the incorrect addressing here would mean every other color sample would be from the wrong color plane, since it's likely that the underlying YUV data is semiplanar (so the U plane starts at V plane + 1 byte, with the two planes interleaved)

这篇关于YUV_420_888解读三星Galaxy S7(相机2)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！