使用python接收高速率的UDP数据包 [英] Receive an high rate of UDP packets with python

问题描述

我正在使用python以便从FPGA接收UDP数据包流，并尝试丢失尽可能少的数据包. 数据包速率从大约5kHz上升到大约MHz，我们希望在特定的时间窗口(代码中的acq_time)获取数据. 我们现在有以下代码:

I'm working with python in order to receive a stream of UDP packets from an FPGA, trying to lose as few packets as possible. The packet rate goes from around 5kHz up to some MHz and we want to take data in a specific time window (acq_time in the code). We have this code now:

BUFSIZE=4096
dataSock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
dataSock.settimeout(0.1)
dataSock.bind((self.HOST_IP, self.HOST_PORT))
time0=time.time()
data_list = []
while time.time()-time0<acq_time:
     fast_acquisition(data_list)

def fast_acquisition(data_list_tmp):
    data, addr = dataSock.recvfrom(self.BUFSIZE)
    data_list_tmp.append(data) 
    return len(data)

在获取之后，我们将data_list保存在磁盘上.

And after the acquisition we save our data_list on disk.

该代码本应尽可能简单和快速，但是它仍然太慢，即使在5kHz时我们也会丢失太多数据包，我们认为发生这种情况的原因是，当我们读取并在列表中存储一个数据包时，检查时间，下一个(一个)到达并丢失. 有什么办法可以保持插座打开吗?我们可以通过并行处理串联"打开多个套接字，以便从第一个保存文件时，第二个可以接收另一个数据包吗? 我们甚至可以考虑只使用另一种语言来接收和存储磁盘上的数据包.

This code is meant to be as simple and fast as possible, but it's still too slow and we lose too many packets even at 5kHz, and we think that this happens because while we read, and store in the list one packet and check the time, the next one (ore ones) arrives and is lost. Is there any way to keep the socket open? Can we open multiple sockets "in series" with parallel processing, so that when we are saving the file from the first the second can receive another packet? We can even think to use another language only to receive and store the packets on disk.

推荐答案

您可以使用 tcpdump (在C中实现的 )捕获UDP流量，因为它比 python :

You could use tcpdump (which is implemented in C) to capture the UDP traffic, since it's faster than python:

#!/bin/bash

iface=$1 # interface, 1st arg
port=$2  # port, 2nd arg

tcpdump -i $iface -G acq_time_in_seconds -n udp port $port -w traffic.pcap

然后您可以使用 scapy 处理该流量

And then you can use e.g. scapy to process that traffic

#!/usr/bin/env python

from scapy.all import *

scapy_cap = rdpcap('traffic.pcap')
for packet in scapy_cap:
    # process packets...

这篇关于使用python接收高速率的UDP数据包的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！