从摄像机将实时素材流式传输到Unity3D [英] Streaming live footage from camera to Unity3D

问题描述

比方说，我有一个无线摄像头，我想实时地将素材传输成一体.有没有办法做到这一点?

Let's say I have a wireless camera that I want to stream footage from to unity in real-time. Is there a way to achieve this?

奖金问题:

• 宽角度相机(180度，甚至360度)怎么办
• 如果这是我要与之交互的素材，那么延迟会有多少问题
• 除了常规镜头之外，是否可以发送更多数据，例如深度感知(使用深度感知相机)?
• 我疯了还是这样做?

预先感谢

推荐答案

我认为这是一台具有以太网端口或Wi-Fi的相机，您可以连接到该相机并实时传输图像.

I assume this is a camera with Ethernet port or Wi-Fi that you can connect to and stream images from it live.

如果是这样，那么可以，可以使用Unity来完成.

If so, then yes, it can be done with Unity.

如何在没有外部库的情况下完成:

连接到相机:

1 .使用相机连接到相同的本地网络，或者如果支持unpn，也可以通过Internet连接到它.通常，您需要摄像机的IP和端口来执行此操作.假设摄像机IP地址为192.168.1.5，端口号为900.要连接的网址是http://192.168.1.5:900.

1.Connect to the-same local network with the camera or if unpn is supported, you can also connect to it through internet. Usually, you need the IP and the port of the camera to do this. Let's say that the Camera IP Address is 192.168.1.5 and the port number is 900. The url to connect to is http://192.168.1.5:900.

有时候，它只是一个以 .mjpg 或 .bin 结尾的网址，例如http://192.168.1.5/mjpg/video.mjpg和http://192.168.1.5/mjpg/video.bin

Sometimes, it is simply an url that ends with .mjpg or .bin such as http://192.168.1.5/mjpg/video.mjpg and http://192.168.1.5/mjpg/video.bin

每个相机都不同.查找网址的唯一方法是阅读其手册.如果该手册不可用，请使用其官方应用程序连接至该手册，然后使用Wireshark查找摄像机图像的URL. username和password(如果需要)也可以在手册中找到.如果不可用，请在Google上找到其型号以及您需要的所有物品.

Every camera is different. The only way to find the url is to read its manual. If the manual is not available, connect to it with its official Application then use Wireshark to discover the url of the camera Image. The username and the password(if required) can also be found in the manual. If not available, google the model number and everything you need should be found.

从相机中提取JPEG :

连接到相机后，相机将向您发送无尽的数据.您可以扫描这些数据并从中检索图像.

When connected to the camera, the camera will be sending endless data to you. This data you can scan and retrieve the image from it.

2 .搜索JPEG标头，该标头是0xFF，后跟0xD8.如果这两个字节彼此相邻，则开始读取字节并将其继续保存到数组中.您可以使用index(int)变量来计数已接收的字节数.

2.Search for the JPEG header which is 0xFF followed by 0xD8. If these two bytes are next to each other then start reading the bytes and continue to save them to an array. You can use an index(int) variable to keep count of how many bytes you have received.

int counter = 0;
byte[] completeImageByte = new byte[500000];
byte[] receivedBytes = new byte[500000];
receivedBytes[counter] = byteFromCamera;
counter++;

3 .从相机读取数据时，请检查接下来的两个字节是否是JPEG页脚(0xFF，后跟0xD9).如果是这样，则您已收到完整的图像(1帧).

3.While reading the data from the camera, check if the next two bytes is the JPEG footer which is 0xFF followed by 0xD9. If this is true then you have received the complete image(1 frame).

您的图像字节应类似于:

Your image bytes should look something like:

0xFF 0xD8其他字节(成千上万个).....然后是0xFF 0xD9

0xFF 0xD8 someotherbytes(thousands of them)..... then 0xFF 0xD9

将receivedBytes复制到completeImageByte变量，以便以后可用于显示图像.将counter变量重置为0.

Copy receivedBytes to the completeImageByte variable so that it can be used to display the image later on. Reset counter variable to 0.

Buffer.BlockCopy(receivedBytes, 0, completeImageByte, 0, counter);
counter = 0;

在屏幕上显示JPEG图像:

4 .在屏幕上显示图像

由于您每秒将收到许多图像，所以我发现显示此图像的最最有效的方法是使用RawImage组件.因此，如果您希望它在移动设备上运行，请不要使用Image或Sprite Renderer.

Since you will be receiving many images per second, the most efficient way I found to display this is to use RawImage Component. So, don't use Image or Sprite Renderer for this if you want this to run on mobile devices.

public RawImage screenDisplay;
if(updateFrame){
Texture2D camTexture = new Texture2D(2, 2);
camTexture.LoadImage(completeImageByte);
screenDisplay.texture = camTexture;
}

您只需在Start()函数中执行一次camTexture = new Texture2D(2, 2);.

You only need to do camTexture = new Texture2D(2, 2); once in the Start() function.

5 .跳回到步骤 2 并继续进行操作，直到您愿意为止.

5.Jump back to step 2 and continue doing it until you want to quite.

用于连接相机的API:.

如果相机需要身份验证(用户名和密码)，请使用 HttpWebRequest .

Use HttpWebRequest if the camera requires authentication (username and password).

对于不需要身份验证的用户，请使用 UnityWebRequest .使用UnityWebRequest时，必须从 DownloadHandlerScript 派生自己的类您的应用程序将崩溃，因为您将不停地接收数据.

For those that does not require authentication, use UnityWebRequest. When using UnityWebRequest, you must derive your own classes from DownloadHandlerScript or your app will crash as you will be receiving data non stop.

从DownloadHandlerScript派生自己的班级的示例:

Example of deriving your own class from DownloadHandlerScript:

using UnityEngine;
using System.Collections;
using UnityEngine.Networking;

public class CustomWebRequest : DownloadHandlerScript
{    
    // Standard scripted download handler - will allocate memory on each ReceiveData callback
    public CustomWebRequest()
        : base()
    {
    }

    // Pre-allocated scripted download handler
    // Will reuse the supplied byte array to deliver data.
    // Eliminates memory allocation.
    public CustomWebRequest(byte[] buffer)
        : base(buffer)
    {
    }

    // Required by DownloadHandler base class. Called when you address the 'bytes' property.
    protected override byte[] GetData() { return null; }

    // Called once per frame when data has been received from the network.
    protected override bool ReceiveData(byte[] byteFromCamera, int dataLength)
    {
        if (byteFromCamera == null || byteFromCamera.Length < 1)
        {
            //Debug.Log("CustomWebRequest :: ReceiveData - received a null/empty buffer");
            return false;
        }

        //Search of JPEG Image here

        return true;
    }

    // Called when all data has been received from the server and delivered via ReceiveData
    protected override void CompleteContent()
    {
        //Debug.Log("CustomWebRequest :: CompleteContent - DOWNLOAD COMPLETE!");
    }

    // Called when a Content-Length header is received from the server.
    protected override void ReceiveContentLength(int contentLength)
    {
        //Debug.Log(string.Format("CustomWebRequest :: ReceiveContentLength - length {0}", contentLength));
    }
}

用法:

using UnityEngine;
using System.Collections;
using UnityEngine.Networking;

public class Test : MonoBehaviour
{

    CustomWebRequest camImage;
    UnityWebRequest webRequest;
    byte[] bytes = new byte[90000];

    void Start()
    {
        string url = "http://camUrl/mjpg/video.mjpg";
        webRequest = new UnityWebRequest(url);
        webRequest.downloadHandler = new CustomWebRequest(bytes);
        webRequest.Send();
    }
}

您可以在ReceiveData功能中执行 2 ， 3 ， 4 和 5 步骤CustomWebRequest脚本.

You can them perform step 2,3,4 and 5 in the ReceiveData function from the CustomWebRequest script.

控制相机:

相机具有平移，旋转，翻转，镜像和执行其他功能的命令，这在每个相机中都不同，但是很简单，只需向相机的url发出GET/POST请求并提供查询即可.这些命令可以在相机手册中找到.

Cameras have commands to pan, rotate, flip, mirror and perform other function.This is different in every camera but it is simple as making GET/POST request to a url of the camera and providing the queries. These commands can be found in the camera manual.

例如:http://192.168.1.5?pan=50&rotate=90

其他框架:

AForge -可以同时处理

AForge - A free framework that can handle both JPEG/MJPES and FFMPEG from the camera. You have to modify it to work with Unity and you should if you can't do step 2,3,4 and 5.

这篇关于从摄像机将实时素材流式传输到Unity3D的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！