将实时镜头从摄像机流式传输到 Unity3D [英] Streaming live footage from camera to Unity3D

问题描述

假设我有一台无线摄像机，我想将镜头实时传输到 Unity.有没有办法做到这一点?

Let's say I have a wireless camera that I want to stream footage from to unity in real-time. Is there a way to achieve this?

额外问题:

• 如果相机的角度更广(180 度，甚至 360 度)
• 如果这是我想与之互动的片段，延迟会有多大的问题
• 除了常规镜头之外，是否可以发送更多数据，例如深度感知(使用深度感知相机)?
• 我是疯了还是已经这样做了?

提前致谢

推荐答案

我假设这是一个带有以太网端口或 Wi-Fi 的相机，您可以连接到它并从中实时流式传输图像.

I assume this is a camera with Ethernet port or Wi-Fi that you can connect to and stream images from it live.

如果是这样，那么是的，可以使用 Unity 完成.

If so, then yes, it can be done with Unity.

如何在没有外部库的情况下完成:

连接到相机:

1.连接到与相机相同的本地网络，或者如果支持unpn，您也可以通过互联网连接到它.通常，您需要摄像机的 IP 和端口来执行此操作.假设摄像机 IP 地址为 192.168.1.5，端口号为 900.要连接的 url 是 http://192.168.1.5:900.

1.Connect to the-same local network with the camera or if unpn is supported, you can also connect to it through internet. Usually, you need the IP and the port of the camera to do this. Let's say that the Camera IP Address is 192.168.1.5 and the port number is 900. The url to connect to is http://192.168.1.5:900.

有时，它只是一个以 .mjpg 或 .bin 结尾的 url，例如 http://192.168.1.5/mjpg/video.mjpg 和 http://192.168.1.5/mjpg/video.bin

Sometimes, it is simply an url that ends with .mjpg or .bin such as http://192.168.1.5/mjpg/video.mjpg and http://192.168.1.5/mjpg/video.bin

每台相机都是不同的.找到该网址的唯一方法是阅读其手册.如果该手册不可用，请使用其官方应用程序连接到它，然后使用 Wireshark 发现相机图像的 url.username 和 password(如果需要)也可以在手册中找到.如果没有，请在 Google 上搜索型号，您需要的所有东西都应找到.

Every camera is different. The only way to find the url is to read its manual. If the manual is not available, connect to it with its official Application then use Wireshark to discover the url of the camera Image. The username and the password(if required) can also be found in the manual. If not available, google the model number and everything you need should be found.

从相机中提取 JPEG:

连接到相机后，相机会不断地向您发送数据.您可以扫描这些数据并从中检索图像.

When connected to the camera, the camera will be sending endless data to you. This data you can scan and retrieve the image from it.

2.搜索 0xFF 后跟 0xD8 的 JPEG 标头.如果这两个字节彼此相邻，则开始读取字节并继续将它们保存到数组中.您可以使用 index(int) 变量来计算收到的字节数.

2.Search for the JPEG header which is 0xFF followed by 0xD8. If these two bytes are next to each other then start reading the bytes and continue to save them to an array. You can use an index(int) variable to keep count of how many bytes you have received.

int counter = 0;
byte[] completeImageByte = new byte[500000];
byte[] receivedBytes = new byte[500000];
receivedBytes[counter] = byteFromCamera;
counter++;

3.从摄像头读取数据时，检查接下来的两个字节是否是JPEG页脚，即0xFF后跟0xD9.如果这是真的，那么您已经收到了完整的图像(1 帧).

3.While reading the data from the camera, check if the next two bytes is the JPEG footer which is 0xFF followed by 0xD9. If this is true then you have received the complete image(1 frame).

您的图像字节应如下所示:

Your image bytes should look something like:

0xFF 0xD8 someotherbytes(数千个)..... 然后是 0xFF 0xD9

0xFF 0xD8 someotherbytes(thousands of them)..... then 0xFF 0xD9

将 receivedBytes 复制到 completeImageByte 变量中，以便以后可以使用它来显示图像.将 counter 变量重置为 0.

Copy receivedBytes to the completeImageByte variable so that it can be used to display the image later on. Reset counter variable to 0.

Buffer.BlockCopy(receivedBytes, 0, completeImageByte, 0, counter);
counter = 0;

在屏幕上显示 JPEG 图像:

4.将图像显示到屏幕

由于您每秒将收到许多图像，因此我发现显示此图像的最有效方式是使用RawImage 组件.因此，如果您希望它在移动设备上运行，请不要为此使用 Image 或 Sprite Renderer.

Since you will be receiving many images per second, the most efficient way I found to display this is to use RawImage Component. So, don't use Image or Sprite Renderer for this if you want this to run on mobile devices.

public RawImage screenDisplay;
if(updateFrame){
Texture2D camTexture = new Texture2D(2, 2);
camTexture.LoadImage(completeImageByte);
screenDisplay.texture = camTexture;
}

camTexture = new Texture2D(2, 2); 只需在 Start() 函数中执行一次即可.

You only need to do camTexture = new Texture2D(2, 2); once in the Start() function.

5.跳回步骤2并继续执行，直到您想要完全为止.

5.Jump back to step 2 and continue doing it until you want to quite.

用于连接相机的 API:.

如果相机需要身份验证(用户名和密码)，请使用 HttpWebRequest.

Use HttpWebRequest if the camera requires authentication (username and password).

对于不需要身份验证的用户，请使用 UnityWebRequest.使用 UnityWebRequest 时，您必须从 派生自己的类DownloadHandlerScript 否则您的应用程序将崩溃，因为您将不停地接收数据.

For those that does not require authentication, use UnityWebRequest. When using UnityWebRequest, you must derive your own classes from DownloadHandlerScript or your app will crash as you will be receiving data non stop.

从DownloadHandlerScript派生你自己的类的例子:

Example of deriving your own class from DownloadHandlerScript:

using UnityEngine;
using System.Collections;
using UnityEngine.Networking;

public class CustomWebRequest : DownloadHandlerScript
{    
    // Standard scripted download handler - will allocate memory on each ReceiveData callback
    public CustomWebRequest()
        : base()
    {
    }

    // Pre-allocated scripted download handler
    // Will reuse the supplied byte array to deliver data.
    // Eliminates memory allocation.
    public CustomWebRequest(byte[] buffer)
        : base(buffer)
    {
    }

    // Required by DownloadHandler base class. Called when you address the 'bytes' property.
    protected override byte[] GetData() { return null; }

    // Called once per frame when data has been received from the network.
    protected override bool ReceiveData(byte[] byteFromCamera, int dataLength)
    {
        if (byteFromCamera == null || byteFromCamera.Length < 1)
        {
            //Debug.Log("CustomWebRequest :: ReceiveData - received a null/empty buffer");
            return false;
        }

        //Search of JPEG Image here

        return true;
    }

    // Called when all data has been received from the server and delivered via ReceiveData
    protected override void CompleteContent()
    {
        //Debug.Log("CustomWebRequest :: CompleteContent - DOWNLOAD COMPLETE!");
    }

    // Called when a Content-Length header is received from the server.
    protected override void ReceiveContentLength(int contentLength)
    {
        //Debug.Log(string.Format("CustomWebRequest :: ReceiveContentLength - length {0}", contentLength));
    }
}

用法:

using UnityEngine;
using System.Collections;
using UnityEngine.Networking;

public class Test : MonoBehaviour
{

    CustomWebRequest camImage;
    UnityWebRequest webRequest;
    byte[] bytes = new byte[90000];

    void Start()
    {
        string url = "http://camUrl/mjpg/video.mjpg";
        webRequest = new UnityWebRequest(url);
        webRequest.downloadHandler = new CustomWebRequest(bytes);
        webRequest.Send();
    }
}

您可以让他们在ReceiveData<中执行步骤2、3、4和5CustomWebRequest 脚本中的/code> 函数.

You can them perform step 2,3,4 and 5 in the ReceiveData function from the CustomWebRequest script.

控制摄像头:

相机具有平移、旋转、翻转、镜像和执行其他功能的命令.这在每个相机中都不同，但它很简单，只需向相机的 url 发出 GET/POST 请求并提供查询.这些命令可以在相机手册中找到.

Cameras have commands to pan, rotate, flip, mirror and perform other function.This is different in every camera but it is simple as making GET/POST request to a url of the camera and providing the queries. These commands can be found in the camera manual.

例如:http://192.168.1.5?pan=50&rotate=90

其他框架:

AForge - 一个可以同时处理 JPEG/MJPES 和 FFMPEG 来自相机.您必须修改它以使用 Unity，如果您不能执行步骤 2、3、4 和 5.

AForge - A free framework that can handle both JPEG/MJPES and FFMPEG from the camera. You have to modify it to work with Unity and you should if you can't do step 2,3,4 and 5.

这篇关于将实时镜头从摄像机流式传输到 Unity3D的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！