python从套接字流接收图像 [英] python receiving image from socket stream

问题描述

我正在尝试使用opencv通过python发送帧并抛出套接字.图像作为字符串流发送，但是当我在服务器上重新制作图像时，字节的顺序不正确.

I'm trying to send a frame and throw socket by python, using opencv . The image is sent as string stream but when i remake the image on the server,the bytes are not in the right order.

客户代码:

import socket
import numpy as np
import cv2


UDP_IP = "127.0.0.1"
UDP_PORT = 5005

cap = cv2.VideoCapture(1)
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
while 1:
    ret, frame = cap.read()
    cv2.imshow('client', frame)

    d = frame.flatten()
    s = d.tostring()
    #sock.sendall(s)
    #print len(s)
    for i in range(20):
        sock.sendto(s[i*46080:(i+1)*46080], (UDP_IP, UDP_PORT))
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

服务器端:

import socket
import numpy as np
import cv2


UDP_IP = "127.0.0.1"
UDP_PORT = 5005

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((UDP_IP, UDP_PORT))
s = ""
while True:
    for i in range(20):
        data,addr = sock.recvfrom(46080)
        s = s + data
    #data, addr = sock.recvfrom(46080)
    #s = s+data
    frame = np.fromstring(s, dtype='uint8')
    frame = frame.reshape(480, 640, 3)
    #cv2.imwrite('frame.png',frame)
    cv2.imshow('server', frame)
    s=""
    '''
    if len(data) == 921600:
        frame = np.fromstring(s, dtype='uint8')
        frame = frame.reshape(480, 640, 3)
        cv2.imshow('frame', frame)
        s = ""
    '''
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

框架客户端

框架服务器端

推荐答案

UDP数据报不保证以任何特定顺序显示.在许多平台上，无论如何，localhost UDP最终还是有序的，或者至少在机器负载不大但在任何地方都不能保证时，最终UDP是有序的.

UDP datagrams are not guaranteed to show up in any particular order. On many platforms, localhost UDP will end up ordered anyway, or at least will end up ordered when the machine isn't heavily loaded, but that isn't guaranteed anywhere.

因此，如果要通过UDP发送数据，则需要自行处理订单.

So, if you want to send data over UDP, you need to handle the ordering itself.

这意味着想出一个协议，该协议在每个帧中都包含某种索引或偏移量.例如，在发送方:

This means coming up with a protocol that includes some kind of index or offset within each frame. For example, on the sending side:

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
for framenum in itertools.count():
    # etc.
    #print len(s)
    for i in range(20):
        buf = struct.pack('>II', (framenum, i)) + s[i*46080:(i+1)*46080])
        sock.sendto(buf, (UDP_IP, UDP_PORT))

在接收方:

s = [None]*20
for framenum in itertools.count():
    for i in range(20):
        data,addr = sock.recvfrom(46080)
        frame, index = struct.unpack('>II', data[:8])
        if frame == framenum:
            s[idx] = data[8:]
            if all(s):
                framedata = ''.join(s)
                # now we have a full frame to display

我还没有编写代码来处理从第2帧获取数据甚至还没有完成第1帧的情况，因为您必须考虑在这种情况下要做什么.扔掉框架1并开始收集框架2?显示第1帧的内容，用第0帧剩余的片段填充缺失的间隙?保留框架列表的列表，并在完成时立即显示每个列表吗?保留框架列表的列表以及已经完成的列表，以便确保按顺序显示它们?使用时间戳记，以便如果第1帧的时间超过X ms并且尚未完成，则将其丢弃?

I haven't written code to handle the case where you get data from frame #2 before you've even finished frame #1, because you have to think about what you want to do in that case. Throw out frame 1 and start collecting frame 2? Display what you have of frame 1, filling in the missing gaps with pieces left over from frame 0? Keep a list of frame lists and show each one as soon as it's completed? Keep a list of frame lists and which ones you've already finished so you can make sure to display them in order? Use timestamps so if frame 1 is more than X ms old and you haven't finished it yet, you just throw it out?

如果您是通过Internet而不是localhost进行此操作，则还有另一个问题:第1帧的最后一个块可能永远不会到达.显然，这会影响您的选择-如果要在显示第1帧之前等待所有第1帧，则可能会永远挂起.您可以添加一种方法，让客户端要求服务器重新发送它认为应该已经到达的(frame，i)，或者为客户端确认它已经看到的服务器重新发送方法.服务器可以告知某些内容可能已经丢失，或者是客户端告诉服务器很多东西出现故障或丢失，变慢的方式"，...

If you're doing this over the internet, rather than localhost, there's an additional problem: the last chunk of frame 1 might never arrive. That obviously affects your options—if you're going to wait for all of frame 1 before displaying any later frames, you may just hang forever. You could add a way for the client to ask the server to resend a (frame, i) that it thinks should have already arrived, or a way for the client to acknowledge which ones it has seen so the server can tell that something might have been lost, or maybe a way for the client to tell the server "way to much stuff is coming out of order or getting lost, slow down", …

所有这些以及更多这些都是针对视频流客户端的可行设计.因此，您必须选择所需的一个并实现它.

All of these, and more, are viable designs for a video streaming client. So, you have to pick the one you want, and implement it.

或者，如果您希望始终按顺序显示帧，则可能要使用TCP而不是UDP.在幕后，TCP只是发送一堆带有附加索引的数据包，但是它具有所有逻辑，可以按顺序将数据包放回原处，确认并重新发送丢失的数据包，当数据包丢失过高时减慢速度(并再次加快速度)(如果那是暂时性的问题)等等，那么您不必编写任何此类内容.您得到的只是字节流，其发送顺序完全相同.(当然，这意味着您必须知道如何从下一条消息中划出一条消息，但是对于您而言，这是微不足道的:每条消息的长度恰好是921600字节，因此，您将一直循环读取直到拥有921600字节.)

Alternatively, if what you want is to always display the frames in order no matter what, you might want to use TCP instead of UDP. Under the covers, TCP is just sending a bunch of packets with indexes attached, but it has all the logic to put the packets back in order, acknowledge and resend missing packets, slow down when packet loss gets too high (and speed back up again if that was a transient problem), etc., so you don't have to write any of that. What you get is just a stream of bytes, in the exact order they were sent. (Of course that means you have to know how to delimit one message from the next, but in your case, that's trivial: every message is exactly 921600 bytes long, so you'd just keep reading in a loop until you have 921600 bytes.)

这篇关于python从套接字流接收图像的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！