Camera2 api Imageformat.yuv_420_888旋转图像的结果 [英] Camera2 api Imageformat.yuv_420_888 results on rotated image

问题描述

关于camera2 api和RAW图像格式已经提出了很多问题，但是在网上搜索我仍然找不到答案(这就是为什么我在这里顺便说一句).

Lots of questions have been made about camera2 api and RAW image format, but searching online I have still not found the answer (that's why I am here btw).

我正在尝试使用

I am trying to do some real-time image processing on camera-captured frames using ImageReader and setRepeatingRequest with the front-facing camera. As suggested in some previous posts, I am acquiring the image in a RAW format (specifically Imageformat.yuv_420_888) in order to have a frame-rate around 30fps:

  imageReader = ImageReader.newInstance(width, height, ImageFormat.YUV_420_888, 2); 

我的图像处理算法需要RGB图像作为输入，因此我需要将YUV转换为RGB.为此，我使用 ScriptIntrinsicYuvToRGB

My image-processing algorithm requires an RGB image as input, so I need to convert from YUV to RGB. To do that I use ScriptIntrinsicYuvToRGB

  private static Bitmap YUV_420_888_toRGBIntrinsics(Image image) {

    if (image == null) return null;

    int W = image.getWidth();
    int H = image.getHeight();

    Image.Plane Y = image.getPlanes()[0];
    Image.Plane U = image.getPlanes()[1];
    Image.Plane V = image.getPlanes()[2];

    int Yb = Y.getBuffer().remaining();
    int Ub = U.getBuffer().remaining();
    int Vb = V.getBuffer().remaining();

    byte[] data = new byte[Yb + Ub + Vb];


    Y.getBuffer().get(data, 0, Yb);
    V.getBuffer().get(data, Yb, Vb);
    U.getBuffer().get(data, Yb + Vb, Ub);



    rs = RenderScript.create(context);
    ScriptIntrinsicYuvToRGB yuvToRgbIntrinsic = ScriptIntrinsicYuvToRGB.create(rs, Element.U8_4(rs));

    Type.Builder yuvType = new Type.Builder(rs, Element.U8(rs)).setX(data.length);
    Allocation in = Allocation.createTyped(rs, yuvType.create(), Allocation.USAGE_SCRIPT);

    Type.Builder rgbaType = new Type.Builder(rs, Element.RGBA_8888(rs)).setX(W).setY(H);
    Allocation out = Allocation.createTyped(rs, rgbaType.create(), Allocation.USAGE_SCRIPT);


    final Bitmap bmpout = Bitmap.createBitmap(W, H, Bitmap.Config.ARGB_8888); 

    in.copyFromUnchecked(data);

    yuvToRgbIntrinsic.setInput(in);
    yuvToRgbIntrinsic.forEach(out);
    out.copyTo(bmpout);
    image.close();
    return bmpout ;
}

此方法相当快，因为​​我可以在不到20ms的时间内转换1080p图像.唯一的问题是图像结果旋转了270度(即以横向模式拍摄照片).即使在相机构建器设置中设置了JPEG_ORIENTATION，captureRequestBuilder.set(CaptureRequest.JPEG_ORIENTATION, characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION));的结果仍然相同.

This method is quite fast since I can convert a 1080p image in less than 20ms. The only issue is that the image result is rotated by 270 degrees (i.e. picture is taken in landscape mode). Even if I set JPEG_ORIENTATION in the camera builder settings,captureRequestBuilder.set(CaptureRequest.JPEG_ORIENTATION, characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION)); the result is still the same.

这是我的问题:

• 有没有一种方法可以通过renderscript内在函数来获取旋转后的图像?
• 是否有旋转"位置?不分配内存的功能?
• 是否有YUV类型图像旋转的设置? 我尝试过的其他解决方案-矩阵旋转，YUV阵列旋转-相当慢.此外，我认为将图像旋转90/180/270是一件容易的事，只要完成拍摄，只需保存行而不是列即可.

• Is there a way to get back a rotated image through renderscript intrinsics?
• Is there a "rotate" function that does not allocate memory ?
• Are there settings for image rotation of a YUV type? Other the solutions that I have tried - Matrix rotation, YUV array rotation - are quite slow. Moreover I think that rotating an image by 90/180/270 is an easy task if done after having taken it, just need to save rows instead of columns (somehow).

推荐答案

不，YUV输出没有内置旋转.为了最大程度地减少开销，它总是按原样从图像传感器产生的.您可以读取SENSOR_ORIENTATION字段来确定如何将图像传感器放置在设备上；通常，图像传感器的长边与Android设备的长边对齐，但是仍然留下两个有效的旋转.

No, there's no built-in rotation for YUV output. To minimize overhead, it's always produced as-is from the image sensor. You can read the SENSOR_ORIENTATION field to determine how the image sensor is placed on the device; typically the long edge of the image sensor lines up with the long edge of the Android device, but that still leaves two rotations that are valid.

此外，如果您的目标是使图像直立"，那么您还需要从加速度计读取设备的方向，并将其添加到旋转中.

Also, if your goal is to have the image 'upright', then you also need to read what the device's orientation is from the accelerometer, and add that in to the rotation.

您正在执行的复制操作已经将帧从图像传递到分配中，因此进行90/180/270度旋转相对比较简单，尽管会占用大量内存带宽.

You're doing a copy already getting the frame from the Image into the Allocation, so doing a 90/180/270 degree rotation then is relatively straightforward, though memory-bandwidth-intensive.

您还可以查看Google的示例应用程序之一， HdrViewfinderDemo 将相机数据转换为RenderScript，而无需执行中间副本，然后将其转换为RGB以绘制到SurfaceView.现在它没有旋转功能，但是您可以通过rsGetElementAtYuv_uchar_ *调整查找以进行90增量操作.

You can also take a look at one of Google's sample apps, HdrViewfinderDemo, which pipes camera data into RenderScript without the intermediate copy you're doing, and then converts to RGB to draw to a SurfaceView. It doesn't have a rotation in it now, but you could adjust the lookup done via rsGetElementAtYuv_uchar_* to do 90-increments.

这篇关于Camera2 api Imageformat.yuv_420_888旋转图像的结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！