在三星Galaxy S7(Camera2)上的YUV_420_888解释 [英] YUV_420_888 interpretation on Samsung Galaxy S7 (Camera2)
问题描述
考虑到以下逻辑(据我所知)，我编写了从YUV_420_888到位图的转换:
I wrote a conversion from YUV_420_888 to Bitmap, considering the following logic (as I understand it):
总结一下这种方法:内核的坐标x和y与Y平面(2d分配)的未填充部分的x和y以及输出位图的x和y都相同.但是，U平面和V平面的结构与Y平面不同，因为它们使用1个字节来覆盖4个像素，此外，它们的PixelStride可能不止一个，此外，也有可能与Y平面不同的填充.因此，为了通过内核高效地访问U和V，我将它们放入1-d分配中，并创建了一个索引"uvIndex"，该索引给出了该1-d分配中相应U-和V的位置，对于给定( x，y)在(非填充的)Y平面中(因此，在输出位图上)坐标.
To summarize the approach: the kernel’s coordinates x and y are congruent both with the x and y of the non-padded part of the Y-Plane (2d-allocation) and the x and y of the output-Bitmap. The U- and V-Planes, however, have a different structure than the Y-Plane, because they use 1 byte for coverage of 4 pixels, and, in addition, may have a PixelStride that is more than one, in addition they might also have a padding that can be different from that of the Y-Plane. Therefore, in order to access the U’s and V’s efficiently by the kernel I put them into 1-d allocations and created an index "uvIndex" that gives the position of the corresponding U- and V within that 1-d allocation, for given (x,y) coordinates in the (non-padded) Y-plane (and, so, the output Bitmap).
为了保持rs-kernel的精简，我通过LaunchOptions限制了x范围，从而排除了yPlane中的填充区域(这反映了y平面的RowStride，因此可以在内核中忽略它).因此，我们只需要考虑uvIndex中的uvPixelStride和uvRowStride，即用于访问u值和v值的索引.
In order to keep the rs-Kernel lean, I excluded the padding area in the yPlane by capping the x-range via LaunchOptions (this reflects the RowStride of the y-plane which thus can be ignored WITHIN the kernel). So we just need to consider the uvPixelStride and uvRowStride within the uvIndex, i.e. the index used in order to access to the u- and v-values.
这是我的代码:
渲染脚本内核，名为yuv420888.rs
Renderscript Kernel, named yuv420888.rs
#pragma version(1)
#pragma rs java_package_name(com.xxxyyy.testcamera2);
#pragma rs_fp_relaxed
int32_t width;
int32_t height;
uint picWidth, uvPixelStride, uvRowStride ;
rs_allocation ypsIn,uIn,vIn;
// The LaunchOptions ensure that the Kernel does not enter the padding zone of Y, so yRowStride can be ignored WITHIN the Kernel.
uchar4 __attribute__((kernel)) doConvert(uint32_t x, uint32_t y) {
// index for accessing the uIn's and vIn's
uint uvIndex= uvPixelStride * (x/2) + uvRowStride*(y/2);
// get the y,u,v values
uchar yps= rsGetElementAt_uchar(ypsIn, x, y);
uchar u= rsGetElementAt_uchar(uIn, uvIndex);
uchar v= rsGetElementAt_uchar(vIn, uvIndex);
// calc argb
int4 argb;
argb.r = yps + v * 1436 / 1024 - 179;
argb.g = yps -u * 46549 / 131072 + 44 -v * 93604 / 131072 + 91;
argb.b = yps +u * 1814 / 1024 - 227;
argb.a = 255;
uchar4 out = convert_uchar4(clamp(argb, 0, 255));
return out;
}
Java方面:
private Bitmap YUV_420_888_toRGB(Image image, int width, int height){
// Get the three image planes
Image.Plane[] planes = image.getPlanes();
ByteBuffer buffer = planes[0].getBuffer();
byte[] y = new byte[buffer.remaining()];
buffer.get(y);
buffer = planes[1].getBuffer();
byte[] u = new byte[buffer.remaining()];
buffer.get(u);
buffer = planes[2].getBuffer();
byte[] v = new byte[buffer.remaining()];
buffer.get(v);
// get the relevant RowStrides and PixelStrides
// (we know from documentation that PixelStride is 1 for y)
int yRowStride= planes[0].getRowStride();
int uvRowStride= planes[1].getRowStride(); // we know from documentation that RowStride is the same for u and v.
int uvPixelStride= planes[1].getPixelStride(); // we know from documentation that PixelStride is the same for u and v.
// rs creation just for demo. Create rs just once in onCreate and use it again.
RenderScript rs = RenderScript.create(this);
//RenderScript rs = MainActivity.rs;
ScriptC_yuv420888 mYuv420=new ScriptC_yuv420888 (rs);
// Y,U,V are defined as global allocations, the out-Allocation is the Bitmap.
// Note also that uAlloc and vAlloc are 1-dimensional while yAlloc is 2-dimensional.
Type.Builder typeUcharY = new Type.Builder(rs, Element.U8(rs));
typeUcharY.setX(yRowStride).setY(height);
Allocation yAlloc = Allocation.createTyped(rs, typeUcharY.create());
yAlloc.copyFrom(y);
mYuv420.set_ypsIn(yAlloc);
Type.Builder typeUcharUV = new Type.Builder(rs, Element.U8(rs));
// note that the size of the u's and v's are as follows:
// ( (width/2)*PixelStride + padding ) * (height/2)
// = (RowStride ) * (height/2)
// but I noted that on the S7 it is 1 less...
typeUcharUV.setX(u.length);
Allocation uAlloc = Allocation.createTyped(rs, typeUcharUV.create());
uAlloc.copyFrom(u);
mYuv420.set_uIn(uAlloc);
Allocation vAlloc = Allocation.createTyped(rs, typeUcharUV.create());
vAlloc.copyFrom(v);
mYuv420.set_vIn(vAlloc);
// handover parameters
mYuv420.set_picWidth(width);
mYuv420.set_uvRowStride (uvRowStride);
mYuv420.set_uvPixelStride (uvPixelStride);
Bitmap outBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
Allocation outAlloc = Allocation.createFromBitmap(rs, outBitmap, Allocation.MipmapControl.MIPMAP_NONE, Allocation.USAGE_SCRIPT);
Script.LaunchOptions lo = new Script.LaunchOptions();
lo.setX(0, width); // by this we ignore the y’s padding zone, i.e. the right side of x between width and yRowStride
lo.setY(0, height);
mYuv420.forEach_doConvert(outAlloc,lo);
outAlloc.copyTo(outBitmap);
return outBitmap;
}
在Nexus 7(API 22)上进行测试，这将返回漂亮的彩色位图.但是，此设备的像素点很小(= 1)，没有填充(即rowtride = width).在全新的Samsung S7(API 23)上进行测试，我得到的图片颜色不正确-绿色除外.但是，图片未显示出对绿色的一般偏见，似乎非正确复制的颜色不是绿色.请注意，S7的u/v像素间隔为2，没有填充.
Testing on Nexus 7 (API 22) this returns nice color Bitmaps. This device, however, has trivial pixelstrides (=1) and no padding (i.e. rowstride=width). Testing on the brandnew Samsung S7 (API 23) I get pictures whose colors are not correct - except of the green ones. But the Picture does not show a general bias towards green, it just seems that non-green colors are not reproduced correctly. Note, that the S7 applies an u/v pixelstride of 2, and no padding.
由于最关键的代码行在rs代码内，因此u/v平面的访问uint uvIndex =(...)我认为，可能存在问题，可能是由于此处不正确地考虑了像素条纹.有人看到解决方案了吗?谢谢.
Since the most crucial code line is within the rs-code the Access of the u/v planes uint uvIndex= (...) I think, there could be the problem, probably with incorrect consideration of pixelstrides here. Does anyone see the solution? Thanks.
更新:我检查了所有内容，而且我很确定有关y，u，v访问的代码是正确的.因此问题必须出在u和v值本身上.非绿色具有紫色倾斜，从u，v值看，它们似乎在110-150左右的较窄范围内.我们是否真的有可能需要处理特定于设备的YUV-> RBG转换...?我有什么想念吗?
UPDATE: I checked everything, and I am pretty sure that the code regarding the access of y,u,v is correct. So the problem must be with the u and v values themselves. Non green colors have a purple tilt, and looking at the u,v values they seem to be in a rather narrow range of about 110-150. Is it really possible that we need to cope with device specific YUV -> RBG conversions...?! Did I miss anything?
更新2:感谢Eddy的反馈，已经更正了代码，现在可以正常工作了.
UPDATE 2: have corrected code, it works now, thanks to Eddy's Feedback.
推荐答案
查看
floor((float) uvPixelStride*(x)/2)
根据Y x坐标计算您的U，V行偏移量(uv_row_offset).
which calculates your U,V row offset (uv_row_offset) from the Y x-coordinate.
如果uvPixelStride = 2，则随着x的增加:
if uvPixelStride = 2, then as x increases:
x = 0, uv_row_offset = 0
x = 1, uv_row_offset = 1
x = 2, uv_row_offset = 2
x = 3, uv_row_offset = 3
，这是不正确的.由于uvPixelStride = 2，因此在uv_row_offset = 1或3时没有有效的U/V像素值.
and this is incorrect. There's no valid U/V pixel value at uv_row_offset = 1 or 3, since uvPixelStride = 2.
您想要
uvPixelStride * floor(x/2)
(假设您不相信自己会记住整数除法的关键舍入行为，如果这样做的话):
(assuming you don't trust yourself to remember the critical round-down behavior of integer divide, if you do then):
uvPixelStride * (x/2)
应该足够
这样，您的映射将变为:
With that, your mapping becomes:
x = 0, uv_row_offset = 0
x = 1, uv_row_offset = 0
x = 2, uv_row_offset = 2
x = 3, uv_row_offset = 2
查看是否可以解决颜色错误.实际上，这里不正确的寻址将意味着所有其他颜色样本都将来自错误的颜色平面，因为底层的YUV数据很可能是半平面的(因此，U平面从V平面+ 1字节开始，两个平面是交错的)
See if that fixes the color errors. In practice, the incorrect addressing here would mean every other color sample would be from the wrong color plane, since it's likely that the underlying YUV data is semiplanar (so the U plane starts at V plane + 1 byte, with the two planes interleaved)
这篇关于在三星Galaxy S7(Camera2)上的YUV_420_888解释的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
