HoloLens - 如何从 Vuforia 获取网络摄像头纹理 2D [英] HoloLens - How to get webcam texture 2D from Vuforia

问题描述

我正在使用 Unity 为 HoloLens 开发 Vuforia 应用程序.

I'm developing a Vuforia app for HoloLens by using Unity.

当检测到图像目标时，此应用会显示一个简单的 3D 对象.

This app displays a simple 3D Object when an image target is detected.

我还使用了 Unity Asset Store 中的 fm Exhibition Tool Pack hololens，以便将在 HoloLens 上运行的应用程序流式传输到 PC.

I'm also using the fm Exhibition Tool Pack hololens from the Unity Asset Store in order to stream the app running on HoloLens to a PC.

一切正常，但是当我将应用程序流式传输到 PC 时，我看到的是 3D Unity 场景而不是房间.

Everything works fine but when i stream the app to PC i see the 3D Unity scene instead of the room.

所以我尝试获取网络摄像头纹理并将其附加到场景内的立方体，但是 vuforia ARCamera 与它发生了某种冲突，我在立方体上看不到任何东西.相反，当我在 Unity Simulator 中运行应用程序时，我看到自己在立方体上.

So i've tried to get the webcam texture and attach it to a cube inside the scene but the vuforia ARCamera get somehow conflict with it and i can't see anything on the cube. Instead when i run the app inside the Unity Simulator i see myself on the cube.

有没有办法从 Vuforia 获取网络摄像头纹理 2D 并将其附加到场景内的游戏对象?也许使用 Vuforia.Image 类?但我不知道它是如何工作的.

Is there a way to get the webcam texture 2D from Vuforia and attach it to a GameObject inside the scene? Maybe with the Vuforia.Image class? But i don't know how it works.

推荐答案

以下脚本与 FMETP STREAM 兼容.脚本在移动设备上进行了测试.

Below scripts are compatible with FMETP STREAM. The scripts are tested on mobile.

using UnityEngine;
using System.Collections;
using Vuforia;
using UnityEngine.UI;

public class VuforiaCamAccess : MonoBehaviour
{
    private bool mAccessCameraImage = true;
    public RawImage rawImage;
    public GameObject Mesh;
    private Texture2D texture;

#if UNITY_EDITOR
    private Vuforia.PIXEL_FORMAT mPixelFormat = Vuforia.PIXEL_FORMAT.GRAYSCALE;
#else
    private Vuforia.PIXEL_FORMAT mPixelFormat =  Vuforia.PIXEL_FORMAT.RGB888;
#endif

    private bool mFormatRegistered = false;

    void Start()
    {
#if UNITY_EDITOR
        texture = new Texture2D(Screen.width, Screen.height, TextureFormat.R8, false);
#else
        texture = new Texture2D(Screen.width, Screen.height, TextureFormat.RGB24, false);
#endif
        // Register Vuforia life-cycle callbacks:
        Vuforia.VuforiaARController.Instance.RegisterVuforiaStartedCallback(OnVuforiaStarted);
        Vuforia.VuforiaARController.Instance.RegisterOnPauseCallback(OnPause);
        Vuforia.VuforiaARController.Instance.RegisterTrackablesUpdatedCallback(OnTrackablesUpdated);
    }

    private void OnVuforiaStarted()
    {
        // Try register camera image format
        if (CameraDevice.Instance.SetFrameFormat(mPixelFormat, true))
        {
            Debug.Log("Successfully registered pixel format " + mPixelFormat.ToString());
            mFormatRegistered = true;
        }
        else
        {
            Debug.LogError("Failed to register pixel format " + mPixelFormat.ToString() +
                "\n the format may be unsupported by your device;" +
                "\n consider using a different pixel format.");
            mFormatRegistered = false;
        }
    }

    private void OnPause(bool paused)
    {
        if (paused)
        {
            Debug.Log("App was paused");
            UnregisterFormat();
        }
        else
        {
            Debug.Log("App was resumed");
            RegisterFormat();
        }
    }

    private void OnTrackablesUpdated()
    {
        //skip if still loading image to texture2d
        if (LoadingTexture) return;

        if (mFormatRegistered)
        {
            if (mAccessCameraImage)
            {
                Vuforia.Image image = CameraDevice.Instance.GetCameraImage(mPixelFormat);
                //if (image != null && image.IsValid())
                if (image != null)
                {
                    byte[] pixels = image.Pixels;
                    int width = image.Width;
                    int height = image.Height;
                    StartCoroutine(SetTexture(pixels, width, height));
                }
            }
        }
    }

    bool LoadingTexture = false;
    IEnumerator SetTexture(byte[] pixels, int width, int height)
    {
        if (!LoadingTexture)
        {
            LoadingTexture = true;
            if (pixels != null && pixels.Length > 0)
            {
                if (texture.width != width || texture.height != height)
                {
#if UNITY_EDITOR
                    texture = new Texture2D(width, height, TextureFormat.R8, false);
#else
                    texture = new Texture2D(width, height, TextureFormat.RGB24, false);
#endif
                }

                texture.LoadRawTextureData(pixels);
                texture.Apply();

                if (rawImage != null)
                {
                    rawImage.texture = texture;
                    rawImage.material.mainTexture = texture;
                }

                if (Mesh != null) Mesh.GetComponent<Renderer>().material.mainTexture = texture;
            }
            yield return null;
            LoadingTexture = false;
        }
    }

    private void UnregisterFormat()
    {
        Debug.Log("Unregistering camera pixel format " + mPixelFormat.ToString());
        CameraDevice.Instance.SetFrameFormat(mPixelFormat, false);
        mFormatRegistered = false;
    }

    private void RegisterFormat()
    {
        if (CameraDevice.Instance.SetFrameFormat(mPixelFormat, true))
        {
            Debug.Log("Successfully registered camera pixel format " + mPixelFormat.ToString());
            mFormatRegistered = true;
        }
        else
        {
            Debug.LogError("Failed to register camera pixel format " + mPixelFormat.ToString());
            mFormatRegistered = false;
        }
    }
}

这篇关于HoloLens - 如何从 Vuforia 获取网络摄像头纹理 2D的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！