如何从 ARCore [Android Studio] 中的 session.update() 获取位图 [英] How to get Bitmap from session.update() in ARCore [Android Studio]

问题描述

我正在尝试使用 ARCore 从我的 ARSession 的当前帧中获取位图.但它始终等于 null.我已经在网上搜索了一段时间，但无法弄清楚我做错了什么.

I am trying to get a Bitmap from the current frame of my ARSession with ARCore. But it always equals null. I've already been searching the web for quite a while but cannot figure out what I am doing wrong.

try {
    capturedImage = mFrame.acquireCameraImage();

    ByteBuffer buffer = capturedImage.getPlanes()[0].getBuffer();

    byte[] bytes = new byte[buffer.capacity()];

    Bitmap bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.length,null);

    if (bitmap == null) 
        Log.e(TAG,"Bitmap was NOT initialized!");

} catch(Exception e){

}

我从我的 GLSurfaceView 的 onDrawFrame 获取 mFrame，我用它来显示相机图像.一切正常，只是我的位图等于 null.

I am getting mFrame from onDrawFrame of my GLSurfaceView which I use to display the camera image. Everything works just fine except that my Bitmap equals null.

我正在使用一个按钮，因此只使用了一个框架，如下所示:

I am using a Button, so that only a single Frame is being used, as follows:

scanButton = (Button) findViewById(R.id.scanButton);
scanButton.setOnClickListener(new View.OnClickListener() {
    @Override
    public void onClick(View view) {
        checkbox = false;
        if (capturedImage!=null) capturedImage.close();
            BitmapMethod();
    }
});

capturedImage、buffer 和 bytes 都不等于 null.

capturedImage, buffer and bytes all do not equal null.

mFrame.acquireCameraImage() 可能有问题吗?

非常感谢

推荐答案

mFrame.acquireCameraImage() 可能有问题吗?

Is there probably something wrong with mFrame.acquireCameraImage()?

不，mFrame.acquireCameraImage() 按预期工作.

但它总是等于空

位图将始终等于 null，因为位图工厂不理解传递给它的图像数据.

The Bitmap will always equal null since bitmap factory does not understand the image data that is passed to it.

方法 mFrame.acquireCameraImage() 以 YUV 格式或 YCbCr 类型的 Image 对象作为响应.这些类型的图像有 3 个平面，解释如下 此处 非常好.这些平面中包含的 ByteArray 可以由 CPU/GPU 以 native 代码直接读取.BitmapFactory 无法读取此类数据.因此，您需要将此 YUV 图像转换为其他图像.

The method mFrame.acquireCameraImage() responds with an object of type Image that is in the YUV format or YCbCr. These types of images have 3 planes which is explained here very nicely. The ByteArray contained in these planes may be read directly by a CPU/GPU in native code. BitmapFactory cannot read this type of data. Hence, you need to convert this YUV image into something else.

为此，您需要使用 YuvImage 类来创建 YUV & 的实例然后使用 compressToJpeg 方法将其转换为 JPEG.一旦你有了这个字节数组，你就可以简单地做你在上面做的事情.使用 BitmapFactory 将其转换为 Bitmap 并将其添加到您的 ImageView.

For that, you need to use YuvImage class to create an instance of YUV & then convert it into JPEG using the compressToJpeg method. Once you have the byteArray from this, you can simply do what you're doing above. Use BitmapFactory to convert it into Bitmap and add it to your ImageView.

注意:YUV 有 3 个平面.从所有平面创建一个字节数组 &然后将其传递给 YUV 构造函数.虽然没有详细说明，但它看起来应该与此类似:

Note : YUV has 3 planes. Create a single bytearray from all planes & then pass it to YUV constructor. Though not elaborate, it should look something similar to this :

//The camera image received is in YUV YCbCr Format. Get buffers for each of the planes and use them to create a new bytearray defined by the size of all three buffers combined
val cameraPlaneY = cameraImage.planes[0].buffer
val cameraPlaneU = cameraImage.planes[1].buffer
val cameraPlaneV = cameraImage.planes[2].buffer

//Use the buffers to create a new byteArray that 
val compositeByteArray = ByteArray(cameraPlaneY.capacity() + cameraPlaneU.capacity() + cameraPlaneV.capacity())

cameraPlaneY.get(compositeByteArray, 0, cameraPlaneY.capacity())
cameraPlaneU.get(compositeByteArray, cameraPlaneY.capacity(), cameraPlaneU.capacity())
cameraPlaneV.get(compositeByteArray, cameraPlaneY.capacity() + cameraPlaneU.capacity(), cameraPlaneV.capacity())

val baOutputStream = ByteArrayOutputStream()
val yuvImage: YuvImage = YuvImage(compositeByteArray, ImageFormat.NV21, cameraImage.width, cameraImage.height, null)
yuvImage.compressToJpeg(Rect(0, 0, cameraImage.width, cameraImage.height), 75, baOutputStream)
val byteForBitmap = baOutputStream.toByteArray()
val bitmap = BitmapFactory.decodeByteArray(byteForBitmap, 0, byteForBitmap.size)
imageView.setImageBitmap(bitmap)

这只是一个粗略的代码.或许还有改进的余地.另请参阅此处.

That's just a rough code. It has scope for improvement perhaps. Also refer here.

这篇关于如何从 ARCore [Android Studio] 中的 session.update() 获取位图的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！