如何访问Hololens前置摄像头 [英] How to access Hololens front camera

问题描述

我正在与hololens一起工作，并且试图获取前置摄像头的图像.我唯一需要做的就是拍摄该相机每一帧的图像并将其转换为字节数组.

I'm working with hololens, and I'm trying to get the image of the front camera. The only thing I need is to take the image of each frame of that camera and transform it into a byte array.

推荐答案

1. 遵循文档中的Unity示例.它有一个写得很好的例子: https://docs.unity3d.com/Manual/windowsholographic-photocapture.html

1. Follow Unity Example in the documentation. It has a well written example: https://docs.unity3d.com/Manual/windowsholographic-photocapture.html

从以上链接的统一文档中复制:

using UnityEngine;
using System.Collections;
using System.Linq;
using UnityEngine.XR.WSA.WebCam;

public class PhotoCaptureExample : MonoBehaviour {
PhotoCapture photoCaptureObject = null;
Texture2D targetTexture = null;

// Use this for initialization
void Start() {
    Resolution cameraResolution = PhotoCapture.SupportedResolutions.OrderByDescending((res) => res.width * res.height).First();
    targetTexture = new Texture2D(cameraResolution.width, cameraResolution.height);

    // Create a PhotoCapture object
    PhotoCapture.CreateAsync(false, delegate (PhotoCapture captureObject) {
        photoCaptureObject = captureObject;
        CameraParameters cameraParameters = new CameraParameters();
        cameraParameters.hologramOpacity = 0.0f;
        cameraParameters.cameraResolutionWidth = cameraResolution.width;
        cameraParameters.cameraResolutionHeight = cameraResolution.height;
        cameraParameters.pixelFormat = CapturePixelFormat.BGRA32;

        // Activate the camera
        photoCaptureObject.StartPhotoModeAsync(cameraParameters, delegate (PhotoCapture.PhotoCaptureResult result) {
            // Take a picture
            photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
        });
    });
}

void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame) {
    // Copy the raw image data into the target texture
    photoCaptureFrame.UploadImageDataToTexture(targetTexture);

    // Create a GameObject to which the texture can be applied
    GameObject quad = GameObject.CreatePrimitive(PrimitiveType.Quad);
    Renderer quadRenderer = quad.GetComponent<Renderer>() as Renderer;
    quadRenderer.material = new Material(Shader.Find("Custom/Unlit/UnlitTexture"));

    quad.transform.parent = this.transform;
    quad.transform.localPosition = new Vector3(0.0f, 0.0f, 3.0f);

    quadRenderer.material.SetTexture("_MainTex", targetTexture);

    // Deactivate the camera
    photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}

void OnStoppedPhotoMode(PhotoCapture.PhotoCaptureResult result) {
    // Shutdown the photo capture resource
    photoCaptureObject.Dispose();
    photoCaptureObject = null;
}
}

获取字节:替换以下方法:

    void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame) {
    // Copy the raw image data into the target texture
    photoCaptureFrame.UploadImageDataToTexture(targetTexture);

    // Create a GameObject to which the texture can be applied
    GameObject quad = GameObject.CreatePrimitive(PrimitiveType.Quad);
    Renderer quadRenderer = quad.GetComponent<Renderer>() as Renderer;
    quadRenderer.material = new Material(Shader.Find("Custom/Unlit/UnlitTexture"));

    quad.transform.parent = this.transform;
    quad.transform.localPosition = new Vector3(0.0f, 0.0f, 3.0f);

    quadRenderer.material.SetTexture("_MainTex", targetTexture);

    // Deactivate the camera
    photoCaptureObject.StopPhotoModeAsync(OnStoppedPhotoMode);
}

使用以下方法:

void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
{
    List<byte> imageBufferList = new List<byte>();
    imageBufferList.Clear();
    photoCaptureFrame.CopyRawImageDataIntoBuffer(imageBufferList);
    var bytesArray = imageBufferList.ToArray();
}

1. 如果您现在正在使用Unity 2018.1或2018.2(我也认为是2017.4)，那么它将无法正常工作.Unity有公共跟踪器来解决此问题: https://issuetracker.unity3d.com/问题/windowsmr无法在全息图中拍摄照片
2. 我创建了一个解决方法，直到Unity修复了该错误: https://github.com/MSAlshair/HoloLensMediaCapture

1. If you are using Unity 2018.1 or 2018.2 (I think also 2017.4) now, then it will NOT work. Unity has public tracker to resolve it: https://issuetracker.unity3d.com/issues/windowsmr-failure-to-take-photo-capture-in-hololens
2. I created a workaround until Unity fixes the bug: https://github.com/MSAlshair/HoloLensMediaCapture

不使用PhotoCapture统一解决方法的基本示例:上面的链接中有更多详细信息

您必须添加#if WINDOWS_UWP才能使用MediaCapture:理想情况下，您希望统一使用PhotoCapture来避免这种情况，但是在Unity解决此问题之前，您可以使用类似的方法.

Basic sample without using PhotoCapture from unity as a workaround: More details in the link above

You must add #if WINDOWS_UWP to be able to use MediaCapture: Ideally, you want to use PhotoCapture from unity to avoid this, but until Unity resolve the issue, you can use something like this.

#if WINDOWS_UWP
        public async System.Threading.Tasks.Task<byte[]> GetPhotoAsync()
        {
            //Get available devices info
            var devices = await Windows.Devices.Enumeration.DeviceInformation.FindAllAsync(
                Windows.Devices.Enumeration.DeviceClass.VideoCapture);
            var numberOfDevices = devices.Count;

            byte[] photoBytes = null;

            //Check if the device has camera
            if (devices.Count > 0)
            {
                Windows.Media.Capture.MediaCapture mediaCapture = new Windows.Media.Capture.MediaCapture();
                await mediaCapture.InitializeAsync();

                //Get Highest available resolution
                var highestResolution = mediaCapture.VideoDeviceController.GetAvailableMediaStreamProperties(
                    Windows.Media.Capture.MediaStreamType.Photo).
                    Select(item => item as Windows.Media.MediaProperties.ImageEncodingProperties).
                    Where(item => item != null).
                    OrderByDescending(Resolution => Resolution.Height * Resolution.Width).
                    ToList().First();

                using (var photoRandomAccessStream = new Windows.Storage.Streams.InMemoryRandomAccessStream())
                {
                    await mediaCapture.CapturePhotoToStreamAsync(highestResolution, photoRandomAccessStream);

                    //Covnert stream to byte array
                    photoBytes = await ConvertFromInMemoryRandomAccessStreamToByteArrayAsync(photoRandomAccessStream);
                }
            }
            else
            {
                System.Diagnostics.Debug.WriteLine("No camera device detected!");
            }

            return photoBytes;
        }

        public static async System.Threading.Tasks.Task<byte[]> ConvertFromInMemoryRandomAccessStreamToByteArrayAsync(
            Windows.Storage.Streams.InMemoryRandomAccessStream inMemoryRandomAccessStream)
        {
            using (var dataReader = new Windows.Storage.Streams.DataReader(inMemoryRandomAccessStream.GetInputStreamAt(0)))
            {
                var bytes = new byte[inMemoryRandomAccessStream.Size];
                await dataReader.LoadAsync((uint)inMemoryRandomAccessStream.Size);
                dataReader.ReadBytes(bytes);

                return bytes;
            }
        }
#endif

别忘了为清单添加功能:

1. WebCam
2. 麦克风:我不确定是否需要麦克风，因为我们只是在拍照还是不在拍照，但是无论如何我还是添加了它.
3. 图片库(如果要保存到图片库中)

这篇关于如何访问Hololens前置摄像头的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！