将 AcquireCameraImageBytes() 从 Unity ARCore 保存为图像存储 [英] Save AcquireCameraImageBytes() from Unity ARCore to storage as an image

问题描述

使用 unity 和新的 1.1 版 ARCore，该 API 公开了一些获取相机信息的新方法.但是，例如，我找不到将其作为文件保存到本地存储为 jpg 的任何好的示例.

Using unity, and the new 1.1 version of ARCore, the API exposes some new ways of getting the camera information. However, I can't find any good examples of saving this as a file to local storage as a jpg, for example.

ARCore 示例有一个很好的示例，用于检索相机数据，然后在此处对其进行处理:https://github.com/google-ar/arcore-unity-sdk/blob/master/Assets/GoogleARCore/Examples/ComputerVision/Scripts/ComputerVisionController.cs#L212 并且有一些在该类中检索相机数据的示例，但没有保存该数据.

The ARCore examples have a nice example of retrieving the camera data and then doing something with it here: https://github.com/google-ar/arcore-unity-sdk/blob/master/Assets/GoogleARCore/Examples/ComputerVision/Scripts/ComputerVisionController.cs#L212 and there are a few examples of retrieving the camera data in that class, but nothing around saving that data.

我看过这个:如何拍摄&使用 Unity ARCore SDK 保存图片/屏幕截图?它使用旧的 API 获取数据的方式，并且也没有真正详细介绍保存.

I've seen this: How to take & save picture / screenshot using Unity ARCore SDK? which uses the older API way of getting data, and doesn't really go into detail on saving, either.

我理想中想要的是一种通过 Unity 将 API 中 Frame.CameraImage.AcquireCameraImageBytes() 中的数据转换为磁盘上存储的 jpg 的方法.

What I ideally want is a way to turn the data from Frame.CameraImage.AcquireCameraImageBytes() in the API into a stored jpg on disk, through Unity.

更新

我主要是通过在 ARCore github 页面上挖掘这个问题来解决这个问题的:https://github.com/google-ar/arcore-unity-sdk/issues/72#issuecomment-355134812 并修改下面桑尼的回答，所以公平一个被接受.

I've since got it working mainly through digging through this issue on the ARCore github page: https://github.com/google-ar/arcore-unity-sdk/issues/72#issuecomment-355134812 and modifying Sonny's answer below, so it's only fair that one gets accepted.

如果其他人试图这样做，我必须执行以下步骤:

In case anyone else is trying to do this I had to do the following steps:

1. 向 Start 方法添加回调以在图像可用时运行您的 OnImageAvailable 方法:

public void Start()
{
    TextureReaderComponent.OnImageAvailableCallback += OnImageAvailable;
}

将 TextureReader(来自 SDK 提供的计算机视觉示例)添加到您的相机和脚本

Add a TextureReader (from the computer vision example provided with the SDK) to your camera and your script

你的 OnImageAvailable 应该看起来像这样:

Your OnImageAvailable should look a bit like this:

/// <summary>
/// Handles a new CPU image.
/// </summary>
/// <param name="format">The format of the image.</param>
/// <param name="width">Width of the image, in pixels.</param>
/// <param name="height">Height of the image, in pixels.</param>
/// <param name="pixelBuffer">Pointer to raw image buffer.</param>
/// <param name="bufferSize">The size of the image buffer, in bytes.</param>
private void OnImageAvailable(TextureReaderApi.ImageFormatType format, int width, int height, IntPtr pixelBuffer, int bufferSize)
{
    if (m_TextureToRender == null || m_EdgeImage == null || m_ImageWidth != width || m_ImageHeight != height)
    {
        m_TextureToRender = new Texture2D(width, height, TextureFormat.RGBA32, false, false);
        m_EdgeImage = new byte[width * height * 4];
        m_ImageWidth = width;
        m_ImageHeight = height;
    }

    System.Runtime.InteropServices.Marshal.Copy(pixelBuffer, m_EdgeImage, 0, bufferSize);

    // Update the rendering texture with the sampled image.
    m_TextureToRender.LoadRawTextureData(m_EdgeImage);
    m_TextureToRender.Apply();

    var encodedJpg = m_TextureToRender.EncodeToJPG();
    var path = Application.persistentDataPath;

    File.WriteAllBytes(path + "/test.jpg", encodedJpg);
}

推荐答案

在 Unity 中，应该可以将原始图像数据加载到纹理中，然后使用 UnityEngine.ImageConversion.EncodeToJPG.示例代码:

In Unity, it should be possible to load the raw image data into a texture and then save it to a JPG using UnityEngine.ImageConversion.EncodeToJPG. Example code:

public class Example : MonoBehaviour
{
    private Texture2D _texture;
    private TextureFormat _format = TextureFormat.RGBA32;

    private void Awake()
    {
        _texture = new Texture2D(width, height, _format, false);
    }

    private void Update()
    {
        using (var image = Frame.CameraImage.AcquireCameraImageBytes())
        {
            if (!image.IsAvailable) return;

            // Load the data into a texture 
            // (this is an expensive call, but it may be possible to optimize...)
            _texture.LoadRawTextureData(image);
            _texture.Apply();
        }
    }

    public void SaveImage()
    {
        var encodedJpg = _texture.EncodeToJPG();
        File.WriteAllBytes("test.jpg", encodedJpg)
    }
}

但是，我不确定 TextureFormat 是否对应于与 Frame.CameraImage.AcquireCameraImageBytes() 一起使用的格式.(我熟悉 Unity 但不熟悉 ARCore.)请参阅 TextureFormat 上的 Unity 文档，以及是否与 ARCore 的 ImageFormatType 兼容.

However, I'm not sure if the TextureFormat corresponds to a format that works with Frame.CameraImage.AcquireCameraImageBytes(). (I'm familiar with Unity but not ARCore.) See Unity's documentation on TextureFormat, and whether that is compatible with ARCore's ImageFormatType.

此外，测试代码的性能是否足以满足您的应用程序.

Also, test whether the code is performant enough for your application.

正如用户@Lece 解释的那样，使用File.WriteAllBytes 保存编码数据.我已经更新了上面的代码示例，因为我最初省略了这一步.

As user @Lece explains, save the encoded data with File.WriteAllBytes. I've updated my code example above as I omitted that step originally.

编辑 #2:有关特定于 ARCore 的完整答案，请参阅问题帖子的更新.这里的评论也可能有用 - Jordan 指出主要部分是使用来自 此处为计算机视觉 sdk 示例".

EDIT #2: For the complete answer specific to ARCore, see the update to the question post. The comments here may also be useful - Jordan specified that "the main part was to use the texture reader from the computer vision sdk example here".

这篇关于将 AcquireCameraImageBytes() 从 Unity ARCore 保存为图像存储的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！