轮询和等待kinect帧以及无法在c ++中获取帧 [英] polling vs waiting for kinect frames and failure to acquire frames in c++

问题描述

1）所有kinect v2 c ++样本都使用轮询模型来处理kinect帧（相比之下v1传感器的样本，等待kinect事件）。是否有任何理由使用轮询而不是从技术角度等待事件？

2）在使用WaitForMultipleSources时我注意到当所有调用都可以进入情境时

p>

 colorFrameRef-> AcquireFrame（& colorFrame）; 

 返回E_FAIL。基本上事件到了，我们得到了多源框架，但我们无法从颜色，正文等帧中提取数据。

具体来说，主循环看起来像这样（所有错误检查为简洁省略，注释显示哪些调用返回S_OK）

而（true）
 
 {
 if（WAIT_OBJECT_0 == WaitForSingleObject（reinterpret_cast< HANDLE>（m_MultiSourceEvent），INFINITE））
 {
 IMultiSourceFrameArrivedEventArgs * pArgs {nullptr}; 
 IMultiSourceFrameReference * frameRef {nullptr}; 
 IMultiSourceFrame * frame {nullptr}; 
 // S_OK 
 hr = m_MultiSourceReader-> GetMultiSourceFrameArrivedEventData（m_MultiSourceEvent，& pArgs）; 
 // S_OK 
 hr = pArgs-> get_FrameReference（& frameRef）; 
 // S_OK 
 hr = frameRef-> AcquireFrame（& frame）; 
 IColorFrameReference * colorFrameRef {nullptr}; 
 // S_OK 
 hr = frame-> get_ColorFrameReference（& colorFrameRef）; 
 IColorFrame * colorFrame = nullptr; 
 //下一个失败的E_FAIL 
 hr = colorFrameRef-> AcquireFrame（& colorFrame）; 
 //将帧复制到例如的std ::矢量< BYTE> my_buffer 
 //但不要到这里
 colorFrame-> CopyConvertedFrameDataToArray（...）
 
 //释放一切如下：
 SafeRelease（frameRef）; 
 SafeRelease（pArgs）; 
} 
 
 //一些计算要求很高的函数
 
 Process（my_buffer）; //在我的例子中，只需用openCV翻转和旋转图像
 
} //结束主循环

我已经通过实验注意到如果激活了Process而不是_all_颜色，则无法获取深度，正文框架（在代码段中用E_FAIL注释表示），尽管事件  仍然是
到达。假设Process（）需要100ms。我希望我们可以获得每秒10帧的性能（1000/100 = 10），每次我们 get一个事件，我们至少可以获得并处理与该事件相关的帧的
。相反，我们得到了事件，但根本无法获得任何框架。但是（！），如果我删除Process（my_buffer）而不是一切都恢复正常，我会得到帧并将它们复制到my_buffer。  为什么
是那个？可以通过民意调查来修复吗？也许，我滥用API？是否有正式支持的示例使用等待模型？ BTW，究竟是API调用 重置MultiSourceEvent？

解决方案

1）K4W团队在C ++中提供的样本大多使用游戏玩家的循环，主要是因为那是使用DirectX时的典型编程模型。如果您注意到这样做的样本都使用DirectX设备显示内容到屏幕上，
和组件。

使用Kinect 4W v2轮询的想法是你可以绘制并且与其他框架（如OpenFrameworks，Cinder和OpenCV）的其他交互速度比Kinect设备为您提供的速度更快。

无论你多少次投票，kinect只会使用api产生30 fps，所以从技术的角度来看，你获得的唯一真正的好处是能够绘制使用轮询时获得类似性能的游戏。

事件机制运行正常。

就.Net来说，大多数.Net框架都使用事件，这更容易。

2）这是设计的。如果您没有释放与Kinect相关的所有资源，或者您在完成资源释放之前运行了计算成本高昂的流程，那么您获取的下一帧很可能不再有效，
因此E_FAIL 。所以你必须检查以确保你得到的帧在处理之前是有效的。

虽然这不是官方样本，你可以在这里找到Kinect的Cingct事件示例：

https://k4wv2eventsample.codeplex.com/

1) All kinect v2 c++ samples use polling model to process kinect frames (in contrast v1 sensor's sample, which wait for kinect events). Is there any reason to use polling instead of waiting for events from technical perspective?

2) While using WaitForMultipleSources I've noticed that it's possible to get into situation, when all calls like

colorFrameRef->AcquireFrame(&colorFrame);

 return E_FAIL. Basically the event arrives, we get the multisource frame, but we can't extract  data from color, body, etc frames.

Specifically the main loop looks like this (all error checking is omitted for brevity, comments show which calls return S_OK)

while (true)

{
if (WAIT_OBJECT_0 == WaitForSingleObject(reinterpret_cast<HANDLE>(m_MultiSourceEvent), INFINITE))
{
IMultiSourceFrameArrivedEventArgs* pArgs{ nullptr };
IMultiSourceFrameReference* frameRef{ nullptr };
IMultiSourceFrame* frame{ nullptr };
//S_OK
hr = m_MultiSourceReader->GetMultiSourceFrameArrivedEventData(m_MultiSourceEvent, &pArgs);
//S_OK
hr = pArgs->get_FrameReference(&frameRef);
//S_OK
hr = frameRef->AcquireFrame(&frame);
IColorFrameReference *colorFrameRef{ nullptr };
//S_OK
hr = frame->get_ColorFrameReference(&colorFrameRef);
IColorFrame *colorFrame = nullptr;
// Next one fails with E_FAIL
hr = colorFrameRef->AcquireFrame(&colorFrame);
// copy frame to e.g. std::vector<BYTE> my_buffer
// but don't get here
colorFrame->CopyConvertedFrameDataToArray(...)

// RELEASE EVERYTHING like:
SafeRelease(frameRef);
SafeRelease(pArgs);
}

// Some computationally demanding function

 Process(my_buffer); // in my example just flipping and rotating image with openCV

} // end main loop

I've noticed experimentally that if Process is activated than _all_ color, depth, body frames cannot be acquired (indicated with E_FAIL comment in the snippet), although the events still arrive. Let's say Process() takes 100ms. I would expect that we could then get 10 frames per second performance (1000/100 =10), and every time we get an event, we can at least acquire and process the frames associated with that event. Instead, we get the events but cannot acquire any frame at all. But(!), If I remove Process(my_buffer) than everything is back to normal, I get the frames and can copy them into my_buffer. Why is that? Could it be fixed with polling? Perhaps, I misuse the APIs? Is there an officially supported example that uses waiting model? BTW, which exactly API call  resets MultiSourceEvent?

解决方案

1) The samples the K4W team have provided in C++ mostly use a gamer's loop, mostly because that's the typical programming model when working with DirectX. If you notice the samples which do this all display content to the screen using DirectX devices, and components.

The idea around polling with the Kinect 4W v2 is that you can draw and have other interactions with other frameworks such as OpenFrameworks, Cinder, and OpenCV at a faster rate than the 30FPS the Kinect Device gives you.

No matter how many times you poll, the kinect will only produce 30 fps using the api's so from a tech standpoint, the only true benefit you get is the ability to draw and get game like performance when using polling.

The Eventing mechanism works just fine.

As far as .Net it's most of the .Net frameworks use eventing and it's a heck of a lot easier.

2) This is by design. If you don't release all the resources pertaining to the Kinect, or you have a computationally expensive process that runs before you finish releasing resources, it's very likely that the next frame you acquired is no longer valid, thus the E_FAIL. So you have to check to make sure the frames you get are valid before processing them.

While this is not an official sample, you can find a Eventing Sample for Kinect with C++ here:

https://k4wv2eventsample.codeplex.com/

这篇关于轮询和等待kinect帧以及无法在c ++中获取帧的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！