存在相机抖动时的基准标记检测 [英] Fiducial marker detection in the presence of camera shake

问题描述

当用户猛烈移动相机（手机）时，我试图使我的基于OpenCV的基准标记检测更加健壮。标记是ArTag风格，在黑色边框内嵌入一个汉明码。通过对图像进行阈值处理来检测边界，然后根据找到的轮廓寻找四边形，然后检查四边形的内部。

一般来说，标记的解码是相当稳定如果识别出黑色边框。我尝试了最明显的事情，这是对图像进行两次降采样，并且还对这些级别执行四分检测。这有助于在极端近地标记上的相机散焦，以及非常小的图像模糊级别，但不能大大帮助相机运动模糊的一般情况。

有可用于研究如何使检测更加强大？想法我想知道include：

1. 你可以做一些光流跟踪来猜测标记在下一帧，然后在这些猜测的区域中进行某种角点检测，而不是将矩形搜索视为全帧阈值处理？


2. 在PC上，可以导出模糊系数（可能通过注册检测到标记的最近的视频帧），并在处理之前对图像进行去模糊处理？


3. 在智能手机上，可以使用陀螺仪和/系数和预处理图像？ （我假设不是，只是因为如果是的话，市场将充斥着摇头校正相机应用程序。）

链接

解决方案

1. 是的，你可以使用光流来估计标记可能在哪里，并本地化您的搜索，但它只是重新定位，您的跟踪将打破模糊的帧。


2. 我不知道deblurring足够


3. 您可以使用传感器来猜测您所面临的模糊，但我猜想去模糊是太计算的




然后使用其他方法：





这里有一些非常聪明的东西： http：//www.robots .ox.ac.uk /〜gk / publications / KleinDrummond2004IVC.pdf 他们在做边缘检测（可以用来找到你的标记边框，即使你现在正在寻找四边形），建模来自传感器的相机移动，以及使用那些值来估计在给定帧速率的情况下模糊方向上的边缘如何出现，以及搜索。十分优雅。





同样， http://www.eecis.udel.edu/~jye/lab_research/11/BLUT_iccv_11.pdf ，他们只是预先模糊跟踪目标，并尝试匹配适当的模糊目标给定的模糊方向。他们使用高斯滤波器模拟模糊，这是对称的，所以你需要一半的预先模糊的目标，你最初的期望。





如果你尝试实施这些，我真的很想知道你是怎么上的！




I'm trying to make my OpenCV-based fiducial marker detection more robust when the user moves the camera (phone) violently. Markers are ArTag-style with a Hamming code embedded within a black border. Borders are detected by thresholding the image, then looking for quads based on the found contours, then checking the internals of the quads.

In general, decoding of the marker is fairly robust if the black border is recognized. I've tried the most obvious thing, which is downsampling the image twice, and also performing quad-detection on those levels. This helps with camera defocus on extreme nearground markers, and also with very small levels of image blur, but doesn't hugely help the general case of camera motion blur

Is there available research on ways to make detection more robust? Ideas I'm wondering about include:

1. Can you do some sort of optical flow tracking to "guess" the positions of the marker in the next frame, then some sort of corner detection in the region of those guesses, rather than treating the rectangle search as a full-frame thresholding?
2. On PCs, is it possible to derive blur coeffiients (perhaps by registration with recent video frames where the marker was detected) and deblur the image prior to processing?
3. On smartphones, is it possible to use the gyroscope and/or accelerometers to get deblurring coefficients and pre-process the image? (I'm assuming not, simply because if it were, the market would be flooded with shake-correcting camera apps.)

Links to failed ideas would also be appreciated if it saves me trying them.

解决方案

1. Yes, you can use optical flow to estimate where the marker might be and localise your search, but it's just relocalisation, your tracking will have broken for the blurred frames.
2. I don't know enough about deblurring except to say it's very computationally intensive, so real-time might be difficult
3. You can use the sensors to guess the sort of blur you're faced with, but I would guess deblurring is too computational for mobile devices in real time.

Then some other approaches:

There is some really smart stuff in here: http://www.robots.ox.ac.uk/~gk/publications/KleinDrummond2004IVC.pdf where they're doing edge detection (which could be used to find your marker borders, even though you're looking for quads right now), modelling the camera movements from the sensors, and using those values to estimate how an edge in the direction of blur should appear given the frame-rate, and searching for that. Very elegant.

Similarly here http://www.eecis.udel.edu/~jye/lab_research/11/BLUT_iccv_11.pdf they just pre-blur the tracking targets and try to match the blurred targets that are appropriate given the direction of blur. They use Gaussian filters to model blur, which are symmetrical, so you need half as many pre-blurred targets as you might initially expect.

If you do try implementing any of these, I'd be really interested to hear how you get on!

这篇关于存在相机抖动时的基准标记检测的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！