如何使用OpenCV和PCL在2D平面上投影红外图像 [英] How to project IR image on a 2D plane using OpenCV and PCL

问题描述

我有一个 Kinect ，我正在使用 OpenCV 和点云库。我想将 IR 图像投影到 2D 平面上以进行叉车托盘检测。我该怎么办？

我正在尝试检测叉车中的托盘这里是一张图片：

解决方案

RGB 数据在哪里？您可以使用它来帮助检测。您无需将图像投影到任何平面上即可检测到颗粒。基本上有两种用于检测的方法

1. 基于神经网络，模糊逻辑，机器学习的非确定性，等等

   
   
   

   此方法需要训练数据集来识别对象。适当的训练集和分类器架构/拓扑选择需要很多经验。但除此之外你不需要编程...因为通常使用一些随时可用的lib /工具来配置和传递数据。




2. 基于距离或相关系数的确定性

   
   
   

   我会从检测特定功能开始：

   
   
   
   
   
   • 托盘具有特定尺寸
   
   
   • 托盘在深度数据中具有锐边和特定几何形状
   
   
   • 托盘具有特定颜色范围（淡黄色木材+/-灯光和污垢）
   
   
   • 木材具有特定纹理图案
   
   
   
   
   

   因此，为每个特征计算一些系数，该对象与实际托盘的接近程度。然后只是阈值所有系数组合的距离（可能加权，因为一些特征更稳健）。

我做不要使用＃1 方法，所以我会选择＃2 。因此，结合 RGB 和深度数据（它们必须完全匹配）。然后分割图像（基于深度和颜色）。之后，每个找到的对象分类，如果它是托盘......

[Edit1]

您的彩色图像与深度数据不对应。对准的灰度级质量差，深度数据图像也很差。是否以某种方式处理深度数据（失去精度）？如果您从不同方面查看数据：

你可以看出它有多差，所以我怀疑你可以使用深度数据进行检测......

PS。我使用

您应该通过设置获得托盘的颜色范围，以便于检测。然后检查这些对象的尺寸，形状，面积，周长等特征...

[Edit2]

所以我会从Image预处理开始：

1. 转换为HSV



2. 阈值仅接近托盘颜色的像素

   
   
   

   我选择了（H = 40，S = 18，V> 100）作为托盘颜色。我的HSV范围是< 0,255> 每个频道所以 Hue 角度差异只能是 < -180deg，+ 180deg> max，对应于我的范围内的< -128，+ 128> 。




3. 删除太薄的区域

   
   
   

   只扫描所有水平线垂直线计算后续设置像素和如果太小的尺寸将它们重新着色为黑色......

这就是结果：

左边原始图像（缩小尺寸使其适合此页面），在中间是颜色阈值结果，最后是过滤掉小区域。您可以使用阈值和托盘颜色来改变行为以满足您的需求。

此处 C ++ 代码：

  int tr_d = 10; //最小托盘尺寸[像素[
 int h，s，v，x，y，xx; 
 color c; 
 
 pic1 = pic0; 
 pic1.pf = _pf_rgba; 
 pic2.resize（pic1.xs * 3，pic1.ys）; XX = 0; 
 pic2.bmp-> Canvas-> Draw（xx，0，pic0.bmp）; XX + = pic1.xs; 
 
 // [颜色选择] 
 for（y = 0; y< pic1.ys; y ++）
 for（x = 0; x< pic1.xs; x ++） 
 {
 //从图像中获取颜色
c = pic0.p [y] [x]; 
 rgb2hsv（c）; 
 // HSV中白黄色的距离（H = 40，S = 18，V> 100）
 h = c.db [picture :: _ h] -40; 
 s = c.db [picture :: _ s] -18; 
 v = c.db [picture :: _ v]; 
 //色调是循环角度，因此如果（h <-128）h + = 256，则仅使用较短的角度
; 
 if（h> +128）h- = 256; 
 // abs值
 if（h <0）h = -h; 
 if（s <0）s = -s; 
 //阈值关闭颜色
 c.dd = 0; 
 if（h< 25）
 if（s< 25）
 if（v> 100）
 c.dd = 0x00FFFFFF; 
 pic1.p [y] [x] = c; 
} 
 pic2.bmp-> Canvas-> Draw（xx，0，pic1.bmp）; XX + = pic1.xs; 
 
 // [删除太薄的区域] 
 for（y = 0; y< pic1.ys; y ++）
 for（x = 0; x< pic1.xs; ）
 {
 for（; x< pic1.xs; x ++）if（pic1.p [y] [x] .dd）break; //找到设置像素
为（h = x; x< pic1.xs; x ++）if（！pic1.p [y] [x] .dd）break; //找到未设置的像素
 if（x-h< tr_d）for（; h< x; h ++）pic1.p [y] [h] .dd = 0; //如果太小的尺寸重新调整到零
} 
 for（x = 0; x< pic1.xs; x ++）
 for（y = 0; y< pic1.ys;）
 {
 for（; y  for（h = y; y< pic1.ys; y ++）if if（！pic1.p [y] [x] .dd）break; //找到未设置的像素
 if（y-h< tr_d）for（; h< y; h ++）pic1.p [h] [x] .dd = 0; //如果太小的尺寸重新变为零
} 
 pic2.bmp-> Canvas-> Draw（xx，0，pic1.bmp）; XX + = pic1.xs; 
  

请参阅如何提取图像边框（OCT /视网膜扫描图像）以获取图片和颜色。或者查看我的 DIP / CV 标记的任何答案。现在代码得到了很好的评论和简单，但只需要添加：

你可以忽略 pic2 它只是它的东西上面张贴的图像所以我不需要手动打印屏幕并将油漆中的子结果合并...为了提高稳健性，您应该添加动态范围的增强（因此阈值对于任何输入图像都具有相同的条件）。此外，您应该比较单一颜色（如果存在更多木材类型的托盘）。

现在您应该对区域进行细分或标记

1. 遍历整个图像


2. 先查找使用托盘颜色设置像素



3. 使用与设置托盘颜色不同的ID颜色填充区域

   
   
   

   我使用黑色 0x00000000 空格和白色 0x00FFFFFF 作为托盘像素颜色。因此，请使用 ID = {1,2,3,4,5 ...} 。还要记住填充像素的数量（即您的区域），这样您就不需要再次计算它。您还可以在填写时直接计算边界框。




4. 计算和比较功能

   
   
   

   您需要尝试多个图像。找出哪些属性有利于检测。我会考虑周长与面积比。和/或边界框尺寸...通过简单地选择具有与黑色像素相邻的适当ID颜色的所有像素，可以提取圆周。

参见类似的使用图像处理手头的裂缝检测

<祝你好运，玩得开心......

I have a Kinect and I'm using OpenCV and point cloud library. I would like to project the IR Image onto a 2D plane for forklift pallet detection. How would I do that?

I'm trying to detect the pallet in the forklift here is an image:

解决方案

Where are the RGB data? You can use it to help with the detection. You do not need to project the image onto any plane to detect a pellet. There are basically 2 ways used for detection

1. non-deterministic based on neural network, fuzzy logic, machine learning, etc

   This approach need a training dataset to recognize the object. Much experience is needed for proper training set and classifier architecture/topology selection. But other then that you do not need to program it... as usually some readily available lib/tool is used just configure and pass the data.

2. deterministic based on distance or correlation coefficients

   I would start with detecting specific features like:

   • pallet has specific size
   • pallet has sharp edges and specific geometry shape in depth data
   • pallet has specific range of colors (yellowish wood +/- lighting and dirt)
   • Wood has specific texture patterns

   So compute some coefficient for each feature how close the object is to real pallet. And then just treshold the distance of all coefficients combined (possibly weighted as some features are more robust).

I do not use the #1 approach so I would go for #2. So combine the RGB and depth data (they have to be matched exactly). Then segmentate the image (based on depth and color). After that for each found object classify if it is pallet ...

[Edit1]

Your colored image does not correspond to depth data. The aligned gray-scale has poor quality and the depth data image is also very poor. Is the depth data processed somehow (loosing precision)? If you look at your data from different sides:

You can see how poor it is so I doubt you can use depth data for detection at all...

PS. I used my Align already captured rgb and depth images for the visualization.

The only thing left is the colored image and detect areas with matching color only. Then detect the features and classify. The color of your pallet in the image is almost white. Here HSV reduced colors to basic 16 colors (too lazy to segmentate)

You should obtain range of colors of the pallets possible by your setup to ease up the detection. Then check those objects for the features like size, shape,area,circumference...

[Edit2]

So I would start with Image preprocessing:

1. convert to HSV

2. treshold only pixels close to pallet color

   I chose (H=40,S=18,V>100) as a pallet color. My HSV ranges are <0,255> per channel so Hue angle difference can be only <-180deg,+180deg> max which corresponds to <-128,+128> in my ranges.

3. remove too thin areas

   Just scan all Horizontal an Vertical lines count consequent set pixels and if too small size recolor them to black...

This is the result:

On the left the original image (downsized so it fits to this page), In the middle is the color treshold result and last is the filtering out of small areas. You can play with tresholds and pallet color to change behavior to suite your needs.

Here C++ code:

int tr_d=10;    // min size of pallet [pixels[
int h,s,v,x,y,xx;
color c;

pic1=pic0;
pic1.pf=_pf_rgba;
pic2.resize(pic1.xs*3,pic1.ys); xx=0;
pic2.bmp->Canvas->Draw(xx,0,pic0.bmp); xx+=pic1.xs;

// [color selection]
for (y=0;y<pic1.ys;y++)
 for (x=0;x<pic1.xs;x++)
    {
    // get color from image
    c=pic0.p[y][x];
    rgb2hsv(c);
    // distance to white-yellowish color in HSV (H=40,S=18,V>100)
    h=c.db[picture::_h]-40;
    s=c.db[picture::_s]-18;
    v=c.db[picture::_v];
    // hue is cyclic angular so use only shorter angle
    if (h<-128) h+=256;
    if (h>+128) h-=256;
    // abs value
    if (h<   0) h=-h;
    if (s<   0) s=-s;
    // treshold close colors
    c.dd=0;
    if (h<25)
     if (s<25)
      if (v>100)
       c.dd=0x00FFFFFF;
    pic1.p[y][x]=c;
    }
pic2.bmp->Canvas->Draw(xx,0,pic1.bmp); xx+=pic1.xs;

// [remove too thin areas]
for (y=0;y<pic1.ys;y++)
 for (x=0;x<pic1.xs;)
    {
    for (   ;x<pic1.xs;x++) if ( pic1.p[y][x].dd) break;    // find set pixel
    for (h=x;x<pic1.xs;x++) if (!pic1.p[y][x].dd) break;    // find unset pixel
    if (x-h<tr_d) for (;h<x;h++) pic1.p[y][h].dd=0;         // if too small size recolor to zero
    }
for (x=0;x<pic1.xs;x++)
 for (y=0;y<pic1.ys;)
    {
    for (   ;y<pic1.ys;y++) if ( pic1.p[y][x].dd) break;    // find set pixel
    for (h=y;y<pic1.ys;y++) if (!pic1.p[y][x].dd) break;    // find unset pixel
    if (y-h<tr_d) for (;h<y;h++) pic1.p[h][x].dd=0;         // if too small size recolor to zero
    }
pic2.bmp->Canvas->Draw(xx,0,pic1.bmp); xx+=pic1.xs;

See how to extract the borders of an image (OCT/retinal scan image) for the description of picture and color. Or look at any of my DIP/CV tagged answers. Now the code is well commented and straightforward but just need to add:

You can ignore pic2 stuff it is just the image posted above so I do not need to manually print screen and merge the subresult in paint... To improve robustness you should add enhancing of dynamic range (so the tresholds have the same conditions for any input images). Also you should compare to more then just single color (if more wood types of pallet is present).

Now you should segmentate or label the areas

1. loop through entire image
2. find first pixel set with the pallet color

3. flood fill the area with some distinct ID color different from set pallet color

   I use black 0x00000000 space and white 0x00FFFFFF as pallete pixel color. So use ID={1,2,3,4,5...}. Also remember number of filled pixels (that is your area) so you do not need to compute it again. You can also compute bounding box directly while filling.

4. compute and compare features

   You need to experiment with more then one image. To find out what properties are good for detection. I would go for circumference length vs area ratio. and or bounding box size... The circumference can be extracted by simply selecting all pixels with proper ID color neighboring black pixel.

See also similar Fracture detection in hand using image proccessing

Good luck and have fun ...

这篇关于如何使用OpenCV和PCL在2D平面上投影红外图像的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！