Gimp过程中的OpenCV Python脚本-草/硬表面边缘检测 [英] OpenCV Python script from Gimp procedure - Grass/Hard surface edge detection
问题描述
我想开发一个Python OpenCV脚本，以对我开发的Gimp程序进行复制/改进.该过程的目标是提供一个x，y点数组，该数组遵循草和硬表面之间的分界线.该阵列将使我能够完成500磅54英寸宽压力清洗机器人，该机器人具有Raspberry Pi Zero(和摄像头)，因此它可以每秒几英寸的速度跟随该边缘.我将进行监视和/或在沙发上看电视时通过其wifi视频流和iPhone应用程序控制该机器人.
I would like to develop a Python OpenCV script to duplicate/improve on a Gimp procedure I have developed. The goal of the procedure is to provide an x,y point array that follows the dividing line between grass and hard surfaces. This array will allow me to finish my 500 lb 54" wide pressure washing robot, which has a Raspberry Pi Zero (and camera), so that it can follow that edge at a speed of a couple inches per second. I will be monitoring and/or controlling the bot via its wifi video stream and an iPhone app while I watch TV on my couch.
以下是原始图片示例(60x80像素):
Here is a sample original image (60x80 pixels):
Gimp程序为:
- 将图像转换为索引的2种颜色.基本上一侧是草，另一侧是砖或人行道.哎呀，哎呀，就是我:)
- 在两种颜色中，使用下面的笔杆设置将较低的色相"值和魔术棒放在该值的像素上. 色调"设置为23表示我去除阴影的方式，而羽毛设置"15"表示删除岛/锯齿(裂缝中的草:)的方式.
- 使用以下高级设置值(对默认值的更改为黄色)对路径进行高级选择.基本上，我只希望线段，而我的(x，y)点数组将是黄色路径点.
- 接下来，我将路径导出到.xml文件，从中可以解析和隔离上图中的黄点.这是.xml文件:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 20010904//EN"
"http://www.w3.org/TR/2001/REC-SVG-20010904/DTD/svg10.dtd">
<svg xmlns="http://www.w3.org/2000/svg"
width="0.833333in" height="1.11111in"
viewBox="0 0 60 80">
<path id="Selection"
fill="none" stroke="black" stroke-width="1"
d="M 60.00,0.00
C 60.00,0.00 60.00,80.00 60.00,80.00
60.00,80.00 29.04,80.00 29.04,80.00
29.04,80.00 29.04,73.00 29.04,73.00
29.04,73.00 30.00,61.00 30.00,61.00
30.00,61.00 30.00,41.00 30.00,41.00
30.00,41.00 29.00,30.85 29.00,30.85
29.00,30.85 24.00,30.85 24.00,30.85
24.00,30.85 0.00,39.00 0.00,39.00
0.00,39.00 0.00,0.00 0.00,0.00
0.00,0.00 60.00,0.00 60.00,0.00 Z" />
</svg>
我在Pi Zero上执行此OpenCV过程的时间目标大约是1-2秒或更短(目前大约需要0.18秒).
My goal for execution time for this OpenCV procedure on my Pi Zero is about 1-2 seconds or less (currently taking ~0.18 secs).
我整理了一些东西，使Gimp xml文件中的点相同.对于面罩的色相范围，我完全不确定它是否正在做Gimp所做的工作.我还没有弄清楚如何在蒙版上应用最小半径，我很确定当蒙版作为蒙版的一部分在硬表面的边缘上出现草"状结块时，我肯定会需要它.这是到目前为止的所有轮廓点(ptscanvas.bmp):
I have cobbled together something that sortof results in the sameish points that are in the Gimp xml file. I am not sure at all if it is doing what Gimp does with regard to the hue range of the mask. I have not yet figured out how to apply the minimum radius on the mask, I am pretty sure I will need that when the mask gets a 'grass' clump on the edge of the hard surface as part of the mask. Here are all the contour points so far (ptscanvas.bmp):
截至2018年7月6日，美国东部时间，这是杂乱无章"的脚本，其工作正常并找到了这些要点；
As of 7/6/2018 5:08 pm EST, here is the 'still messy' script that sortof works and found those points;
import numpy as np
import time, sys, cv2
img = cv2.imread('2-60.JPG')
cv2.imshow('Original',img)
# get a blank pntscanvas for drawing points on
pntscanvas = np.zeros(img.shape, np.uint8)
print (sys.version)
if sys.version_info[0] < 3:
raise Exception("Python 3 or a more recent version is required.")
def doredo():
start_time = time.time()
# Use kmeans to convert to 2 color image
hsv_img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
Z = hsv_img.reshape((-1,3))
Z = np.float32(Z)
# define criteria, number of clusters(K)
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)
K = 2
ret,label,center=cv2.kmeans(Z,K,None,criteria,10,cv2.KMEANS_RANDOM_CENTERS)
# Create a mask by selecting a hue range around the lowest hue of the 2 colors
if center[0,0] < center[1,0]:
hueofinterest = center[0,0]
else:
hueofinterest = center[1,0]
hsvdelta = 8
lowv = np.array([hueofinterest - hsvdelta, 0, 0])
higv = np.array([hueofinterest + hsvdelta, 255, 255])
mask = cv2.inRange(hsv_img, lowv, higv)
# Extract contours from the mask
ret,thresh = cv2.threshold(mask,250,255,cv2.THRESH_BINARY_INV)
im2,contours,hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
# Find the biggest area contour
cnt = contours[0]
max_area = cv2.contourArea(cnt)
for cont in contours:
if cv2.contourArea(cont) > max_area:
cnt = cont
max_area = cv2.contourArea(cont)
# Make array of all edge points of the largets contour, named allpnts
perimeter = cv2.arcLength(cnt,True)
epsilon = 0.01*cv2.arcLength(cnt,True) # 0.0125*cv2.arcLength(cnt,True) seems to work better
allpnts = cv2.approxPolyDP(cnt,epsilon,True)
end_time = time.time()
print("Elapsed cv2 time was %g seconds" % (end_time - start_time))
# Convert back into uint8, and make 2 color image for saving and showing
center = np.uint8(center)
res = center[label.flatten()]
res2 = res.reshape((hsv_img.shape))
# Save, show and print stuff
cv2.drawContours(pntscanvas, allpnts, -1, (0, 0, 255), 2)
cv2.imwrite("pntscanvas.bmp", pntscanvas)
cv2.imshow("pntscanvas.bmp", pntscanvas)
print('allpnts')
print(allpnts)
print("center")
print(center)
print('lowv',lowv)
print('higv',higv)
cv2.imwrite('mask.bmp',mask)
cv2.imshow('mask.bmp',mask)
cv2.imwrite('CvKmeans2Color.bmp',res2)
cv2.imshow('CvKmeans2Color.bmp',res2)
print ("Waiting for 'Spacebar' to Do/Redo OR 'Esc' to Exit")
while(1):
ch = cv2.waitKey(50)
if ch == 27:
break
if ch == ord(' '):
doredo()
cv2.destroyAllWindows()
要做的事:
- 在非边缘像素上添加蒙版圆角，以照顾像Gimp这样在原始蒙版上运行最小半径之前创建的原始蒙版:
1a.截至2018年7月9日，我一直专注于这个问题，因为这似乎是我最大的问题.我无法像cimp2那样使cv2.findcontours平滑边缘草"，而Gimp则具有其魔杖半径功能.左边是2色问题"蒙版，其上覆盖的结果是红色"点，可直接使用cv2.findcontours找到，而右边则是将Gimp半径蒙版应用于cv2之前的左侧图像问题"蒙版.将findcontours应用于它，得到正确的图像和点:
1a. As of July 9, 2018, I have been concentrating on this issue as it seems to be my biggest problem. I am unable to have cv2.findcontours smooth out the 'edge grass' as well as Gimp does with its magic wand radius feature. Here on the left, is a 2 colour 'problem' mask and the overlaid resultant 'Red' points that are found directly using cv2.findcontours and on the right, the Gimp radiused mask applied to the left images 'problem' mask before cv2.findcontours is applied to it, resulting in the right image and points:
我尝试查看Gimps源代码，但是这超出了我的理解范围，我找不到任何可以做到这一点的OpenCV例程.有没有办法在OpenCV中将最小半径平滑应用于边缘蒙版的非边缘"像素???通过非边缘"，我的意思是，正如您所看到的，Gimp不会使这些角"(在黄色高光内部)产生半径，而只是似乎将半径平滑应用于图像内部"的边缘(注意:Gimps半径算法消除了所有蒙版中的小岛，这意味着您不必在应用cv2.findcontours之后即可找到最大的区域轮廓):
I have tried looking at Gimps source code but it is way beyond my comprehension and I can not find any OpenCV routines that can do this. Is there a way to apply a minimum radius smoothing to the 'non-edge' pixels of an edge mask in OpenCV??? By 'non-edge' I mean that as you can see Gimp does not radius these 'corners' (inside Yellow highlight) but only seems to apply the radius smoothing to edges 'inside' the image (Note: Gimps radiusing algorithm eliminates all the small islands in the mask which means that you don't have to find the largest area contour after cv2.findcontours is applied to get the points of interest):
- 从图像边缘上的所有像素中删除不相关的阵列点.
- 弄清楚为什么发现的阵列点似乎是围绕绿草而不是硬表面边界的，我以为我正在使用硬表面色调.
- 弄清楚为什么CvKmeans2Color.bmp中的硬表面颜色显示为橙色而不是Gimps转换中的米色，为什么这与Gimps转换的像素不匹配?这是CvKmeans2Color.bmp和Gimps:
截至美国东部标准时间2018年7月12日下午5点:我已经诉诸于我可以最轻松地使用VB6，ughh创建代码的语言.无论如何，我已经能够制作出行/边缘平滑例程，该例程可以在像素级别上工作，以实现所需的最小半径蒙版.它就像吃豆人沿边缘的右侧尽可能近地漫游，并在吃豆的左侧留下面包屑痕迹的后面.不确定我可以从该代码中制作python脚本，但至少我有一个开始的地方，因为没有人确认有OpenCV替代方法可以执行此操作.如果有人感兴趣，请此处是一个已编译的.exe文件，该文件应在大多数Windows系统上运行而无需安装(我认为).这是它的屏幕截图(蓝色/GreenyBlue像素是未平滑的边缘，绿色/GreenyBlue像素是半径的边缘):
As of 5pm EST July 12, 2018: I have resorted to the language I can most easily create code with, VB6, ughh, I know. Anyway I have been able to make a line/edge smoothing routine that works on the pixel level to do the min radius mask I want. It works like a PacMan roaming along the right side of an edge as close at it can and leaves behind a breadcrumb trail on the Pac's left side. Not sure I can make a python script from that code but at least I have a place to start as nobody has confirmed that there is an OpenCV alternative way to do it. If anyone is interested here is a compiled .exe file that should run on most windows systems without an install (I think). Here is a screenshot from it (Blue/GreenyBlue pixels are the unsmoothed edge and Green/GreenyBlue pixels are the radiused edge):
您可以通过以下VB6例程了解我的过程逻辑要点:
You can get the gist of my process logic by this VB6 routine:
Sub BeginFollowingEdgePixel()
Dim lastwasend As Integer
wasinside = False
While (1)
If HitFrontBumper Then
GoTo Hit
Else
Call MoveForward
End If
If circr = orgpos(0) And circc = orgpos(1) Then
orgpixr = -1 'resets Start/Next button to begin at first first found blue edge pixel
GoTo outnow 'this condition indicates that you have followed all blue edge pixels
End If
Call PaintUnderFrontBumperWhite
Call PaintGreenOutsideLeftBumper
nomove:
If NoLeftBumperContact Then
Call MoveLeft
Call PaintUnderLeftBumperWhite
Call PaintGreenOutsideLeftBumper
If NoLeftBumperContact Then
If BackBumperContact Then
Call MakeLeftTheNewForward
End If
End If
ElseIf HitFrontBumper Then
Hit:
Call PaintAheadOfForwardBumperGreen
Call PaintGreenOutsideLeftSide
Call MakeRightTheNewForward
GoTo nomove
Else
Call PaintAheadOfForwardBumperGreen
Call PaintGreenOutsideLeftSide
Call PaintUnderFrontBumperWhite
End If
If (circr = 19 + circrad Or circr = -circrad Or circc = 19 + circrad Or circc = -circrad) Then
If lastwasend = 0 And wasinside = True Then
'finished following one edge pixel
lastwasend = 1
GoTo outnow
Call redrawit
End If
Else
If IsCircleInsideImage Then
wasinside = True
End If
lastwasend = 0
End If
Pause (pausev) 'seconds between moves - Pressing Esc advances early
Wend
outnow:
End Sub
推荐答案
好吧，我终于有时间来看一下.我将解决您的每个问题，然后显示代码中的更改.如果您有任何疑问或建议，请告诉我.
Okay, I finally had time to look at this. I will address each point of yours and then show the changes in the code. Let me know if you have any questions, or suggestions.
-
看起来您自己能够做得足够好.
Looks like you were able to do this yourself well enough.
1.a.这可以通过在对图像进行任何处理之前对图像进行模糊处理来解决.为实现此目的，对代码进行了以下更改；
1.a. This can be taken care of by blurring the image before doing any processing to it. The following changes to the code were made to accomplish this;
...
start_time = time.time()
blur_img = cv2.GaussianBlur(img,(5,5),0) #here
# Use kmeans to convert to 2 color image
hsv_img = cv2.cvtColor(blur_img, cv2.COLOR_BGR2HSV)
...
我更改了代码，以删除完全跟随图像侧面的直线上的点.草缘与此基本吻合基本上是不可能的.
I have changed the code to remove points that are on a line that perfectly follows the side of the image. It should be basically impossible for a grass edge to also coincide with this.
...
allpnts = cv2.approxPolyDP(cnt,epsilon,True)
new_allpnts = []
for i in range(len(allpnts)):
a = (i-1) % len(allpnts)
b = (i+1) % len(allpnts)
if ((allpnts[i,0,0] == 0 or allpnts[i,0,0] == (img.shape[1]-1)) and (allpnts[i,0,1] == 0 or allpnts[i,0,1] == (img.shape[0]-1))):
tmp1 = allpnts[a,0] - allpnts[i,0]
tmp2 = allpnts[b,0] - allpnts[i,0]
if not (0 in tmp1 and 0 in tmp2):
new_allpnts.append(allpnts[i])
else:
new_allpnts.append(allpnts[i])
...
cv2.drawContours(pntscanvas, new_allpnts, -1, (0, 0, 255), 2)
...
由于如何在图像中找到轮廓，我们可以简单地翻转阈值功能并在图像的其他部分周围找到轮廓.更改如下；
Due to how the contours are found in the image, we can simply flip the thresholding function and find the contour around the other part of the image. Changes are below;
...
#Extract contours from the mask
ret,thresh = cv2.threshold(mask,250,255,cv2.THRESH_BINARY) #here
im2,contours,hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
...
对于色差，您已将图像转换为HSV格式，并且在保存之前没有将其切换回BGR.对HSV的更改确实可以为您带来更好的效果，因此我会保留它，但这是一个不同的调色板.更改如下；
As for the color differences, you have converted your image into HSV format and before saving you are not switching it back to BGR. This change to HSV does give you better results so I would keep it, but it is a different palette. Changes are below;
...
cv2.imshow('mask.bmp',mask)
res2 = cv2.cvtColor(res2, cv2.COLOR_HSV2BGR)
cv2.imwrite('CvKmeans2Color.bmp',res2)
cv2.imshow('CvKmeans2Color.bmp',res2)
...
免责声明:这些更改基于上面的python代码.不在提供代码中对python代码所做的任何更改都会使我的更改无效.
Disclaimer: These changes are based off of the python code from above. Any changes to the python code that are not in the provide code my render my changes ineffective.
这篇关于Gimp过程中的OpenCV Python脚本-草/硬表面边缘检测的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
