c ++ opencv网络摄像头流到html [英] c++ opencv webcam stream to html
问题描述
我目前正在为我的研究开发一个项目，我必须在其中获取网络摄像头流、检测一些对象并在此流上添加一些附加信息.这一切都是在服务器端完成的.
I am currently developing a project for my studies where I have to fetch a webcam stream, detect some objects and put some additional information on this stream. This is all done on the server side.
现在我还必须向客户端提供流的修改图像.客户端只需打开一个包含以下内容的 HTML 文件:
Now I also have to provide the modified image of the stream to the clients. The clients just open a HTML file with the following content:
<html>
<head>
<title></title>
</head>
<body>
<h1>It works!</h1>
<video width="320" height="240" src="http://127.0.0.1:4711/videostream" type="video/quicktime" autoplay controls>
Your browser does not support the video tag.
</video>
</body>
</html>
这将在服务器上产生一个/videostream 的 HTTP 请求.为了在服务器端处理这个请求，我将使用 Boost 1.56.
This will result in a HTTP request on the server for /videostream. To handle this request on the server side I will use Boost 1.56.
目前我的网络摄像头流的每一帧都是 IplImage 类型.我是否必须将 IplImage 转换为视频 MIME 类型特定格式?
Currently each frame of my webcam stream is of type IplImage. Do I have to convert the IplImage into a video MIME-Typespecific format?
我试图自己弄清楚整个事情是如何运作的，但我无法理解.我用 Wireshark 来分析通信，但它没有意义.出于测试目的，我已将视频上传到我的网站空间并在本地打开上述文件.视频的 src 是我的网络服务器的地址.首先是 TCP 握手的内容，然后是此消息:
I have tried to figure it out myself, how the whole thing is working, but I couldn't get it. I used Wireshark to analyze the communication, but it doesn't make sense. For testing purpose I have uploaded a video to my webspace and open the above file locally. The src of the video was the address of my webserver. First there is the TCP handshake stuff followed by this message:
HTTP 765 GET /MOV_4198.MOV HTTP/1.1
遵循以下消息(它包含连接:HTTP 部分中的 Keep-Alive):
Followed the following message (it contains connection: Keep-Alive in the HTTP part):
HTTP 279 HTTP/1.1 304 Not Modified
之后只有 TCP ACK 和 SYN 跟随，但没有数据.请看下图:见图片
Afterwards only TCP ACK and SYN folow, but no data. See the following picture: see picture
视频的真实数据在哪里以及如何发送?我在这里错过了什么?
Where and how are the real data of the video sent? What have I missed here?
如果您能提供有关浏览器(视频标签)和 C++ 套接字连接之间连接的一些信息，那就太好了.
Would be great if you could provide me some information about the connection between the Browser (video-Tag) and the C++ socket connection.
谢谢，斯蒂芬
推荐答案
我想分享我的经验 - 也许它也能帮助其他人.为了从网络摄像头获取我的流，我使用了 OpenCV 2.4.9 作为协议，我使用了 mjpeg 流媒体协议(另见MJPEG over HTTP)-谢谢致@berak - 他在我的问题帖子下的评论中提到了 MJPEG.
I want to share my made experiences - maybe it will help others too. To get my stream from the webcam I used OpenCV 2.4.9 and as protocol I used mjpeg streaming protocol (see also MJPEG over HTTP) - thanks to @berak - he mentioned MJPEG in his comment under my question-post.
下面的代码只是提供了一个概览——我不会深入讨论线程细节.由于这是一个学生项目并且我们使用的是 GitHub，您可以在 GitHub 上找到完整的源代码 这里 -Swank Rats项目我想在这里提一下，我不是 C++、OpenCV 或 Boost 大师.这个项目是我第一次使用所有三个.
The following code just gives an overview - I do not go into threading details. Since this is a students project and we are using GitHub, you can find the whole source code here on GitHub - project Swank Rats I want to mention here, that I am not a C++, OpenCV or Boost guru. This project is the first time I use all three of them.
做这样的事情(带线程的完整代码，所以在 repo 中搜索 WebcamService)
Do something like this (full code with threading and so search for WebcamService in the repo)
cv::VideoCapture capture();
cv::Mat frame;
while (1) {
if (!capture.isOpened()) {
break; //do some logging here or something else - webcam not available
}
//Create image frames from capture
capture >> frame;
if (!frame.empty()) {
//do something with your image (e.g. provide it)
lastImage = frame.clone();
}
}
通过 HTTP 提供您的图片
好吧，我不会详细介绍如何使用 C++ 创建 HTTP 服务器.Boost for C+ 提供了一个很好的例子+11.我已经复制了这段代码并根据我的需要进行了调整.你可以在上面提到的 repo 中找到我的实现的源代码.代码目前位于基础设施/网络/视频流.
Provide your image via HTTP
Well I don't go into detail how you create a HTTP server with C++. There is a nice example provided by Boost for C++11. I have copied this code and adapted it to my needs. You can find the source code of my implementation in the repo mentioned above. The code is currently located at infrastructure / networking / videostreaming.
没有必要使用 FFMPEG、GStreamer 或类似的东西.您可以像这样使用 OpenCV 创建内存中的 JPEG(请参阅 StreamResponseHandler 的代码):
There is no need to use FFMPEG, GStreamer or something simular. You can create a in-memory JPEG using OpenCV like this (see code of StreamResponseHandler):
cv::Mat image = webcamService->GetLastImage();
// encode mat to jpg and copy it to content
std::vector<uchar> buf;
cv::imencode(".jpg", image, buf, std::vector<int>());
std::string content(buf.begin(), buf.end()); //this must be sent to the client
感谢@codeDr 的在这里发帖.
Thanks to @codeDr for his post here.
content 变量表示以字节为单位的图像，它将被发送到客户端.您必须遵循 MJPEG 协议.
The content variable represents the image in bytes, which will be sent to the client. You have to follow the protocol of MJPEG.
这样就足够了(如此处所述)
Something like this is enough (as mentioned here)
<html>
<body>
<h1> Test for simple Webcam Live streaming </h1>
<img src="http://127.0.0.1:4711/videostream">
</body>
</html>
您必须更改服务器连接的 IP、端口等.
You have to change the IP, port and so on to your server connection.
我希望这会有所帮助.
这篇关于c ++ opencv网络摄像头流到html的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
