如何对静态图片(视频)流进行即时编码，以将其从C#发送到Python? [英] How to perform an On-the-Fly encoding of a stream of still-pictures (video) for sending these from C# to Python?

问题描述

我同时获得了深度和使用Kinect SDK(C#)来自Kinect 2的彩色框架，我正在使用 ZeroMQ 将它们发送到Python客户端.

I'm getting both Depth & Color frames from the Kinect 2, using the Kinect SDK ( C# ), and I'm sending them to Python clients using ZeroMQ.

this.shorts     = new ushort[ 217088]; //  512 *  424
this.depthBytes = new   Byte[ 434176]; //  512 *  424 * 2
this.colorBytes = new   Byte[4147200]; // 1920 * 1080 * 4

public void SendDepthFrame(DepthFrame depthFrame)
    {
        depthFrame.CopyFrameDataToArray(this.shorts);
        Buffer.BlockCopy(shorts, 0, this.depthBytes, 0, this.depthBytes.Length);
        this.depthPublisher.SendByteArray(this.depthBytes);
    }

public void SendColorFrame(ColorFrame colorFrame, WriteableBitmap map)
    {
        colorFrame.CopyRawFrameDataToArray(this.colorBytes);
        this.colorPublisher.SendByteArray(this.colorBytes);
    }

由于我要发送未压缩的数据，所以我过载了网络，我想压缩这些帧.

Since I'm sending uncompressed data, I'm overloading the network and I'd like to compress these frames.

我知道我可以通过以PNG/JPEG格式进行压缩来做到这一点，但我想保留视频流的概念.

I know that I can do that by compressing in a PNG/JPEG format, but I would like to maintain the notion of video stream.

目标是在C#中发送压缩数据，然后在Python中对其进行解码.

The goal is to send the compressed data in C#, and then decoding them in Python.

有没有允许这样做的库?

Is there any libs that allow to do that ?

推荐答案

暂时可能会忘记压缩，而PoC

的比例会降低

如果您的设计确实有意义，请尝试首先专注于核心 CV 功能，但要以降低(缩小)的FPS，色彩深度，分辨率(按优先级顺序)为代价

May forget about compression for the moment and downscale for PoC

If your design indeed makes sense, try to focus rather on core CV-functionality first, at a cost of reduced ( downscaled ) FPS, colordepth, resolution ( in this order of priority ).

您指示的数据产生大约1 Gbps的出口数据流，即将出现的CV处理将在任何情况下阻塞，具有显着的 CV -处理性能(延迟/延迟)/临时数据表示的内存管理瓶颈.

Your indicated data produces about 1 Gbps exgress data-stream, where the forthcoming CV-processing will choke anyways, having remarkable CV-process performance ( delay / latency ) / interim data-representations' memory-management bottlenecks.

这表示PoC可能会受益于1/4-1/10的FPS采集/流处理速度降低，并且经过微调的解决方案可能会向您显示每帧多少纳秒您的代码具有流处理裕度(最终确定是否是否具有足够的时间和处理能力，可以将任何类型的CODEC处理都包含在本来可以工作的范围内管道)

This said, the PoC may benefit from 1/4 - 1/10 slower FPS acquisition/stream-processing and the finetuned solution may show you, how many nanoseconds-per-frame does your code have in stream-processing margin ( to finally decide if there is time & processing-power enough to include any sort of CODEC-processing into the otherwise working pipeline )

通过右键单击-> [在新标签页中打开]
检查左下窗口在[usec] 中的延迟
以查看放大并实现在实际的处理中，一些实际 openCV 的比例/数量级处理延迟约为您的 FullFD 静态图像的1/4在单线程i7/ 3.33 GHz 设备上的FPS小得多，其中L3高速缓存大小可以承载多达 15 MB 个图像数据，并且小于 13 ns ( core-local access case ) .. 40 ns ( core-remote NUMA access case ) 的最快延迟+编排为CV的图像处理的块性质可以从最小的缓存丢失率(如果不是零)中受益匪浅-但这不是通用的部署硬件方案要依靠:
费用( penalty )每个高速缓存未命中，并且需要在主DDR-RAM中请求并执行对数据访问的权限大约是+100 ns >>> https://stackoverflow. com/a/33065382/3666197

check the lower-left window delays in [usec] by a right-click -> [Open in a New Tab]
to see enlarged and realise a scale / order of magnitude of a few actual openCV procesing latencies of about a 1/4 of your one FullFD still image in a real-world processing with much smaller FPS on a single-threaded i7/3.33 GHz device, where L3 cache sizes can carry as much as 15 MB of imagery-data with fastest latencies of less than 13 ns ( core-local access case ) .. 40 ns ( core-remote NUMA access case ) + block-nature of the CV-orchestrated image-processing benefits a lot from minimal if not zero cache-miss-rate -- but this is not a universal deployment hardware scenario to rely on:
The costs ( penalty ) of each cache-miss and a need to ask for and peform an access to data in the main DDR-RAM is about +100 ns >>> https://stackoverflow.com/a/33065382/3666197

没有工作流水线，没有有关持续流处理的定量数据/决定建议的PoC的CODEC-难题先验的每帧边距-实施.

Without a working pipeline, there are no quantitative data about the sustained stream-processing / it's margin-per-frame to decide the CODEC-dilemma a-priori of the proposed PoC-implementation.

这篇关于如何对静态图片(视频)流进行即时编码，以将其从C#发送到Python?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！