如何根据对象位置旋转图像? [英] How can I rotate an image based on object position?
本文介绍了如何根据对象位置旋转图像?的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
首先，很抱歉文章的长度.
我正在开展一个项目，根据叶子的图像对植物进行分类.为了减少数据的方差，我需要旋转图像，以便茎在图像底部水平对齐(270 度).
目前我在哪里...
到目前为止我所做的是创建一个阈值图像，然后从那里找到轮廓并在对象周围绘制一个椭圆(在许多情况下，它无法涉及整个对象，因此省略了茎......)，之后那，我创建了 4 个区域(带有椭圆的边缘)并尝试计算最小值区域，这是因为假设在任何一点都必须找到茎，因此它将是人口较少的区域(主要是因为它会被 0 包围)，这显然不像我想要的那样工作.
之后我以两种不同的方式计算旋转角度，第一种涉及atan2函数，这只需要我想要移动的点(人口最少的质心区域)，其中 x=image width/2 和 y = height.这种方法在某些情况下有效，但在大多数情况下，我没有得到所需的角度，有时需要负角度，它会产生正角度，最终茎在顶部.在其他一些情况下，它只是以一种可怕的方式失败.
我的第二种方法是尝试基于 3 个点计算角度:图像中心、人口最少区域的质心和 270º 点.然后使用 arccos 函数，并将其结果转换为度数.
这两种方法对我来说都失败了.
问题
- 您认为这是一种正确的方法，还是我只是让事情变得比我应该做的更复杂?
- 如何找到叶子的茎(这不是可选的，必须是茎)?因为我的想法并不奏效...
- 如何以稳健的方式确定角度?由于第二个问题中的相同原因...
这里有一些示例和我得到的结果(二进制掩码).矩形表示我正在比较的区域，椭圆上的红线是椭圆的主轴，粉红色圆圈是最小区域内的质心，红色圆圈表示 270º 参考点(角度)，白点代表图像的中心.
我目前的解决方案
定义亮度失真(I, mu, sigma):返回 np.sum(I*mu/sigma**2,axis=-1)/np.sum((mu/sigma)**2,axis=-1)def chromacity_distortion(I, mu, sigma):阿尔法 = 亮度_失真(我，亩，西格玛)[...，无]返回 np.sqrt(np.sum(((I - alpha * mu)/sigma)**2, axis=-1))def bwareafilt(图像):图像 = image.astype(np.uint8)nb_components, output, stats, centroids = cv2.connectedComponentsWithStats(image,connectivity=4)尺寸 = 统计数据 [:, -1]最大标签 = 1max_size = 尺寸[1]对于范围内的 i (2, nb_components):如果尺寸[i] >最大尺寸:max_label = imax_size = 尺寸[i]img2 = np.zeros(output.shape)img2[输出 == max_label] = 255返回 img2def get_thresholded_rotated(im_path):#读取图像img = cv2.imread(im_path)img = cv2.resize(img, (600, 800), 插值 = Image.BILINEAR)sat = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)[:,:,1]val = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)[:,:,2]sat = cv2.medianBlur(sat, 11)val = cv2.medianBlur(val, 11)#创建阈值thresh_S = cv2.adaptiveThreshold(sat, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 401, 10);thresh_V = cv2.adaptiveThreshold(val, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 401, 10);#mean，标准mean_S, stdev_S = cv2.meanStdDev(img, mask = 255 - thresh_S)mean_S = mean_S.ravel().flatten()stdev_S = stdev_S.ravel()#色度chrom_S = chromacity_distortion(img, mean_S, stdev_S)chrom255_S = cv2.normalize(chrom_S, chrom_S, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX).astype(np.uint8)[:,:,None]mean_V, stdev_V = cv2.meanStdDev(img, mask = 255 - thresh_V)mean_V = mean_V.ravel().flatten()stdev_V = stdev_V.ravel()chrom_V = chromacity_distortion(img, mean_V, stdev_V)chrom255_V = cv2.normalize(chrom_V, chrom_V, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX).astype(np.uint8)[:,:,None]#创建不同的阈值thresh2_S = cv2.adaptiveThreshold(chrom255_S, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 401, 10);thresh2_V = cv2.adaptiveThreshold(chrom255_V, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 401, 10);#阈值图像thresh = cv2.bitwise_and(thresh2_S, cv2.bitwise_not(thresh2_V))#findcountours 并保持最大轮廓 = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)轮廓 = 轮廓 [0] 如果 len(contours) == 2 否则轮廓 [1]big_contour = max(contours, key=cv2.contourArea)# 将椭圆拟合到叶子轮廓椭圆 = cv2.fitEllipse(big_contour)(xc,yc), (d1,d2), 角度 = 椭圆打印('脱粒形状:'，thresh.shape)#print(xc,yc,d1,d2,angle)rmajor = max(d1,d2)/2rminor = min(d1,d2)/2origi_angle = 角度如果角度＞90:角度 = 角度 - 90别的:角度 = 角度 + 90#calc 主轴xtop = xc + math.cos(math.radians(angle))*rmajorytop = yc + math.sin(math.radians(angle))*rmajorxbot = xc + math.cos(math.radians(angle+180))*rmajorybot = yc + math.sin(math.radians(angle+180))*rmajor#calc 短轴线xtop_m = xc + math.cos(math.radians(origi_angle))*rminorytop_m = yc + math.sin(math.radians(origi_angle))*rminorxbot_m = xc + math.cos(math.radians(origi_angle+180))*rminorybot_m = yc + math.sin(math.radians(origi_angle+180))*rminor#确定哪个区域在上，哪个在下如果 max(xtop, xbot) == xtop :x_tij = xtopy_tij = ytopx_b_tij = xboty_b_tij = ybot别的:x_tij = xboty_tij = ybotx_b_tij = xtopy_b_tij = ytop如果 max(xtop_m, xbot_m) == xtop_m :x_tij_m = xtop_my_tij_m = ytop_mx_b_tij_m = xbot_my_b_tij_m = ybot_m别的:x_tij_m = xbot_my_tij_m = ybot_mx_b_tij_m = xtop_my_b_tij_m = ytop_m打印(' - - -')打印(x_tij，y_tij)rect_size = 100"计算椭圆长轴的边缘区域这是通过创建一个 rect_size x rect_size 的平方区域来完成的，边缘是正方形的中心"x_min_tij = int(0 if x_tij - rect_size < 0 else x_tij - rect_size)x_max_tij = int(thresh.shape[1]-1 if x_tij + rect_size > thresh.shape[1] else x_tij + rect_size)y_min_tij = int(0 if y_tij - rect_size < 0 else y_tij - rect_size)y_max_tij = int(thresh.shape[0] - 1 if y_tij + rect_size > thresh.shape[0] else y_tij + rect_size)x_b_min_tij = int(0 if x_b_tij - rect_size < 0 else x_b_tij - rect_size)x_b_max_tij = int(thresh.shape[1] - 1 if x_b_tij + rect_size > thresh.shape[1] else x_b_tij + rect_size)y_b_min_tij = int(0 if y_b_tij - rect_size <0 else y_b_tij - rect_size)y_b_max_tij = int(thresh.shape[0] - 1 if y_b_tij + rect_size > thresh.shape[0] else y_b_tij + rect_size)sum_red_region = np.sum(thresh[y_min_tij:y_max_tij, x_min_tij:x_max_tij])sum_yellow_region = np.sum(thresh[y_b_min_tij:y_b_max_tij, x_b_min_tij:x_b_max_tij])"计算椭圆短轴的边缘区域这是通过创建一个 rect_size x rect_size 的平方区域来完成的，边缘是正方形的中心"x_min_tij_m = int(0 if x_tij_m - rect_size < 0 else x_tij_m - rect_size)x_max_tij_m = int(thresh.shape[1]-1 if x_tij_m + rect_size > thresh.shape[1] else x_tij_m + rect_size)y_min_tij_m = int(0 if y_tij_m - rect_size < 0 else y_tij_m - rect_size)y_max_tij_m = int(thresh.shape[0] - 1 if y_tij_m + rect_size > thresh.shape[0] else y_tij_m + rect_size)x_b_min_tij_m = int(0 if x_b_tij_m - rect_size <0 else x_b_tij_m - rect_size)x_b_max_tij_m = int(thresh.shape[1] - 1 if x_b_tij_m + rect_size > thresh.shape[1] else x_b_tij_m + rect_size)y_b_min_tij_m = int(0 if y_b_tij_m - rect_size < 0 else y_b_tij_m - rect_size)y_b_max_tij_m = int(thresh.shape[0] - 1 if y_b_tij_m + rect_size > thresh.shape[0] else y_b_tij_m + rect_size)#区域的值，变量的名称与函数末尾绘制的矩形的颜色有关sum_red_region_m = np.sum(thresh[y_min_tij_m:y_max_tij_m, x_min_tij_m:x_max_tij_m])sum_yellow_region_m = np.sum(thresh[y_b_min_tij_m:y_b_max_tij_m, x_b_min_tij_m:x_b_max_tij_m])#print(sum_red_region, sum_yellow_region, sum_red_region_m, sum_yellow_region_m)min_arg = np.argmin(np.array([sum_red_region, sum_yellow_region, sum_red_region_m, sum_yellow_region_m]))打印('分钟:'，min_arg)如果 min_arg == 1: #sum_yellow_region <sum_red_region :left_quartile = x_b_tij <阈值形状[0]/2上四分位数 = y_b_tij <阈值形状[1]/2center_x = x_b_min_tij + ((x_b_max_tij - x_b_min_tij)/2)center_y = y_b_min_tij + (y_b_max_tij - y_b_min_tij/2)center_x = x_b_min_tij + np.argmax(thresh[y_b_min_tij:y_b_max_tij, x_b_min_tij:x_b_max_tij].mean(axis=0))center_y = y_b_min_tij + np.argmax(thresh[y_b_min_tij:y_b_max_tij, x_b_min_tij:x_b_max_tij].mean(axis=1))elif min_arg == 0:left_quartile = x_tij <阈值形状[0]/2上四分位数 = y_tij <阈值形状[1]/2center_x = x_min_tij + ((x_b_max_tij - x_b_min_tij)/2)center_y = y_min_tij + ((y_b_max_tij - y_b_min_tij)/2)center_x = x_min_tij + np.argmax(thresh[y_min_tij:y_max_tij, x_min_tij:x_max_tij].mean(axis=0))center_y = y_min_tij + np.argmax(thresh[y_min_tij:y_max_tij, x_min_tij:x_max_tij].mean(axis=1))elif min_arg == 3:left_quartile = x_b_tij_m <阈值形状[0]/2upper_quartile = y_b_tij_m <阈值形状[1]/2center_x = x_b_min_tij_m + ((x_b_max_tij_m - x_b_min_tij_m)/2)center_y = y_b_min_tij_m + (y_b_max_tij_m - y_b_min_tij_m/2)center_x = x_b_min_tij_m + np.argmax(thresh[y_b_min_tij_m:y_b_max_tij_m, x_b_min_tij_m:x_b_max_tij_m].mean(axis=0))center_y = y_b_min_tij_m + np.argmax(thresh[y_b_min_tij_m:y_b_max_tij_m, x_b_min_tij_m:x_b_max_tij_m].mean(axis=1))别的:left_quartile = x_tij_m <阈值形状[0]/2upper_quartile = y_tij_m <阈值形状[1]/2center_x = x_min_tij_m + ((x_b_max_tij_m - x_b_min_tij_m)/2)center_y = y_min_tij_m + ((y_b_max_tij_m - y_b_min_tij_m)/2)center_x = x_min_tij_m + np.argmax(thresh[y_min_tij_m:y_max_tij_m, x_min_tij_m:x_max_tij_m].mean(axis=0))center_y = y_min_tij_m + np.argmax(thresh[y_min_tij_m:y_max_tij_m, x_min_tij_m:x_max_tij_m].mean(axis=1))# 在输入的副本上绘制椭圆结果 = img.copy()cv2.ellipse(结果，椭圆，(0,0,255)，1)cv2.line(result, (int(xtop),int(ytop)), (int(xbot),int(ybot)), (255, 0, 0), 1)cv2.circle(result, (int(xc),int(yc)), 10, (255, 255, 255), -1)cv2.circle(result, (int(center_x),int(center_y)), 10, (255, 0, 255), 5)cv2.circle(result, (int(thresh.shape[1]/2),int(thresh.shape[0] - 1)), 10, (255, 0, 0), 5)cv2.rectangle(结果，(x_min_tij，y_min_tij)，(x_max_tij，y_max_tij)，(255,0,0)，3)cv2.rectangle(结果，(x_b_min_tij，y_b_min_tij)，(x_b_max_tij，y_b_max_tij)，(255,255,0)，3)cv2.rectangle(结果，(x_min_tij_m，y_min_tij_m)，(x_max_tij_m，y_max_tij_m)，(255,0,0)，3)cv2.rectangle(结果,(x_b_min_tij_m,y_b_min_tij_m),(x_b_max_tij_m,y_b_max_tij_m),(255,255,0),3)plt.imshow(结果)plt.figure()#旋转图像rot_img = Image.fromarray(thresh)#180bot_point_x = int(thresh.shape[1]/2)bot_point_y = int(thresh.shape[0] - 1)#poipoi_x = int(center_x)poi_y = int(center_y)#image_centerim_center_x = int(thresh.shape[1]/2)im_center_y = int(thresh.shape[0] - 1)/2#a - adalt, b - abaix, c - dreta#ba = a - b#bc = c - a(b en realitat)ba = np.array([im_center_x, im_center_y]) - np.array([bot_point_x, bot_point_y])bc = np.array([poi_x, poi_y]) - np.array([im_center_x, im_center_y])#angle 3 平底船cosine_angle = np.dot(ba, bc)/(np.linalg.norm(ba) * np.linalg.norm(bc))cos_angle = np.arccos(cosine_angle)cos_angle = np.degrees(cos_angle)打印('cos角度:'，cos_angle)打印('打印:'，abs(poi_x-bot_point_x))m = (int(thresh.shape[1]/2)-int(center_x)/int(thresh.shape[0] - 1)-int(center_y))ttan = math.tan(m)theta = math.atan(ttan)打印('θ:'，θ)结果 = Image.fromarray(result)结果 = result.rotate(cos_angle)plt.imshow(结果)plt.figure()#rot_img = rot_img.rotate(origi_angle)rot_img = rot_img.rotate(cos_angle)返回 rot_imgrot_img = get_thresholded_rotated(im_path)plt.imshow(rot_img)提前致谢--- 编辑---
我按照要求在这里留下了一些原始图像.
解决方案
概念
这适用于大多数叶子，只要它们有茎.所以这里是检测和旋转一个叶子图像的旋转的概念:
找出叶子的近似轮廓.由于茎的尖端点通常属于叶的凸包(外点)，找到轮廓的凸包.
遍历属于叶凸包的轮廓索引.对于每个索引，计算 3 个点之间的角度:索引前轮廓中的点、索引处轮廓中的点和索引后轮廓中的点.
计算出的最小角度是茎尖.每次循环找到一个较小的角度时，将三个点存储在一个元组中，当检测到最小角度时，使用尖端两侧的 2 个坐标的中心计算茎指向的角度茎，茎的尖端.
检测到茎的角度后，我们可以相应地旋转图像.
代码
导入 cv2将 numpy 导入为 np定义进程(img):img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)img_blur = cv2.GaussianBlur(img_gray, (3, 3), 2)img_canny = cv2.Canny(img_blur, 127, 47)内核 = np.ones((5, 5))img_dilate = cv2.dilate(img_canny，内核，迭代=2)img_erode = cv2.erode(img_dilate，内核，迭代=1)返回 img_erodedef get_contours(img):轮廓，_ = cv2.findContours(process(img), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)cnt = max(contours, key=cv2.contourArea)peri = cv2.arcLength(cnt, True)返回 cv2.approxPolyDP(cnt, 0.01 * peri, True)def get_angle(a, b, c):ba, bc = a - b, c - bcos_angle = np.dot(ba, bc)/(np.linalg.norm(ba) * np.linalg.norm(bc))返回 np.degrees(np.arccos(cos_angle))def get_rot_angle(img):轮廓 = get_contours(img)长度 = len(轮廓)最小角度 = 180对于 cv2.convexHull(contours, returnPoints=False).ravel() 中的 i:a, b, c = 轮廓[[i - 1, i, (i + 1) % 长度], 0]角度 = get_angle(a, b, c)如果角度＜最小角度:min_angle = 角度分 = a, b, ca, b, c = pts返回 180 - np.degrees(np.arctan2(*(np.mean((a, c), 0) - b)))定义旋转(img):h, w, _ = img.shaperot_mat = cv2.getRotationMatrix2D((w/2, h/2), get_rot_angle(img), 1)返回 cv2.warpAffine(img, rot_mat, (w, h), flags=cv2.INTER_LINEAR)img = cv2.imread("leaf.jpg")cv2.imshow(图像"，旋转(img))cv2.waitKey(0)输出
您提供的每个示例图像的输出:
说明
分解代码:
- 导入必要的库:
导入 cv2将 numpy 导入为 np- 定义一个函数，
process，将图像处理成二进制图像，使程序能够准确检测叶子的轮廓:
def process(img):img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)img_blur = cv2.GaussianBlur(img_gray, (3, 3), 2)img_canny = cv2.Canny(img_blur, 127, 47)内核 = np.ones((5, 5))img_dilate = cv2.dilate(img_canny，内核，迭代=2)img_erode = cv2.erode(img_dilate，内核，迭代=1)返回 img_erode- 定义一个函数，
get_contours，得到图像中最大轮廓的近似轮廓，使用之前定义的process函数:
def get_contours(img):轮廓，_ = cv2.findContours(process(img), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)cnt = max(contours, key=cv2.contourArea)peri = cv2.arcLength(cnt, True)返回 cv2.approxPolyDP(cnt, 0.01 * peri, True)- 定义一个函数，
get_angle，以获取 3 个点之间的角度:
def get_angle(a, b, c):ba, bc = a - b, c - bcos_angle = np.dot(ba, bc)/(np.linalg.norm(ba) * np.linalg.norm(bc))返回 np.degrees(np.arccos(cos_angle))- 定义一个函数
get_rot_angle，以获取图像需要旋转的度数.它通过使用get_angleget_angle 找到叶子的凸包点与叶子轮廓中的 2 个周围点之间的角度，其中 3 个点之间的角度最小，从而确定这个角度.代码>之前定义的函数:
def get_rot_angle(img):轮廓 = get_contours(img)长度 = len(轮廓)最小角度 = 180对于 cv2.convexHull(contours, returnPoints=False).ravel() 中的 i:a, b, c = 轮廓[[i - 1, i, (i + 1) % 长度], 0]角度 = get_angle(a, b, c)如果角度＜最小角度:min_angle = 角度分 = a, b, ca, b, c = pts返回 180 - np.degrees(np.arctan2(*(np.mean((a, c), 0) - b)))- 定义一个函数，
rotate，使用之前定义的get_rot_angle函数沿其中心旋转图像:
def 旋转(img):h, w, _ = img.shaperot_mat = cv2.getRotationMatrix2D((w/2, h/2), get_rot_angle(img), 1)返回 cv2.warpAffine(img, rot_mat, (w, h), flags=cv2.INTER_LINEAR)- 最后，读入您的图像，应用之前定义的
rotate函数并显示旋转后的图像:
img = cv2.imread("leaf.jpg")cv2.imshow(图像"，旋转(img))cv2.waitKey(0)First off, sorry for the length of the post.
I'm working on a project to classify plants based on an image of the leaf. In order to reduce the variance of the data I need to rotate the image so the stem would be horizontally aligned at the bottom of the Image (at 270 degrees).
Where I am at so far...
What I have done so far is to create a thresholded image and from there find contours and draw an ellipse around the object (in many cases it fails to involve the whole object so the stem is left out...), after that, I create 4 regions (with the edges of the ellipse) and try to calculate the minimum value region, this is due to the assumption that at any of this points the stem must be found and thus it will be the less populated region (mostly because it will be surrounded by 0's), this is obviously not working as I would like to.
After that I calculate the angle to rotate in two different ways, the first one involves the atan2 function, this only requires the point I want to move from (the centre of mass of the least populated region) and where x=image width / 2 and y = height. This method works in some cases, but in most cases, I don't get the desired angle, sometimes a negative angle is required and it yields a positive one, ending up with the stem at the top. In some other cases, it just fails in an awful manner.
My second approach is an attempt to calculate the angle based on 3 points: centre of the image, centre of mass of the least populated region and 270º point. Then using an arccos function, and translating its result to degrees.
Both approaches are failing for me.
Questions
- Do you think this is a proper approach or I'm just making things more complicated than I should?
- How can I find the stem of the leaf (this is not optional, it must be the stem)? because my idea is not working so well...
- How can I determine the angle in a robust way? because of the same reason in the second question...
Here are some samples and the results I'm getting (the binary mask). The rectangles denote the regions I'm comparing, the red line across the ellipse is the major axis of the ellipse, the pink circle is the centre of mass inside the minimum region, the red circle denotes the 270º reference point (for the angle), and the white dot represents the centre of the image.
My current Solution
def brightness_distortion(I, mu, sigma):
return np.sum(I*mu/sigma**2, axis=-1) / np.sum((mu/sigma)**2, axis=-1)
def chromacity_distortion(I, mu, sigma):
alpha = brightness_distortion(I, mu, sigma)[...,None]
return np.sqrt(np.sum(((I - alpha * mu)/sigma)**2, axis=-1))
def bwareafilt ( image ):
image = image.astype(np.uint8)
nb_components, output, stats, centroids = cv2.connectedComponentsWithStats(image, connectivity=4)
sizes = stats[:, -1]
max_label = 1
max_size = sizes[1]
for i in range(2, nb_components):
if sizes[i] > max_size:
max_label = i
max_size = sizes[i]
img2 = np.zeros(output.shape)
img2[output == max_label] = 255
return img2
def get_thresholded_rotated(im_path):
#read image
img = cv2.imread(im_path)
img = cv2.resize(img, (600, 800), interpolation = Image.BILINEAR)
sat = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)[:,:,1]
val = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)[:,:,2]
sat = cv2.medianBlur(sat, 11)
val = cv2.medianBlur(val, 11)
#create threshold
thresh_S = cv2.adaptiveThreshold(sat , 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 401, 10);
thresh_V = cv2.adaptiveThreshold(val , 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 401, 10);
#mean, std
mean_S, stdev_S = cv2.meanStdDev(img, mask = 255 - thresh_S)
mean_S = mean_S.ravel().flatten()
stdev_S = stdev_S.ravel()
#chromacity
chrom_S = chromacity_distortion(img, mean_S, stdev_S)
chrom255_S = cv2.normalize(chrom_S, chrom_S, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX).astype(np.uint8)[:,:,None]
mean_V, stdev_V = cv2.meanStdDev(img, mask = 255 - thresh_V)
mean_V = mean_V.ravel().flatten()
stdev_V = stdev_V.ravel()
chrom_V = chromacity_distortion(img, mean_V, stdev_V)
chrom255_V = cv2.normalize(chrom_V, chrom_V, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX).astype(np.uint8)[:,:,None]
#create different thresholds
thresh2_S = cv2.adaptiveThreshold(chrom255_S , 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 401, 10);
thresh2_V = cv2.adaptiveThreshold(chrom255_V , 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 401, 10);
#thresholded image
thresh = cv2.bitwise_and(thresh2_S, cv2.bitwise_not(thresh2_V))
#find countours and keep max
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)
# fit ellipse to leaf contours
ellipse = cv2.fitEllipse(big_contour)
(xc,yc), (d1,d2), angle = ellipse
print('thresh shape: ', thresh.shape)
#print(xc,yc,d1,d2,angle)
rmajor = max(d1,d2)/2
rminor = min(d1,d2)/2
origi_angle = angle
if angle > 90:
angle = angle - 90
else:
angle = angle + 90
#calc major axis line
xtop = xc + math.cos(math.radians(angle))*rmajor
ytop = yc + math.sin(math.radians(angle))*rmajor
xbot = xc + math.cos(math.radians(angle+180))*rmajor
ybot = yc + math.sin(math.radians(angle+180))*rmajor
#calc minor axis line
xtop_m = xc + math.cos(math.radians(origi_angle))*rminor
ytop_m = yc + math.sin(math.radians(origi_angle))*rminor
xbot_m = xc + math.cos(math.radians(origi_angle+180))*rminor
ybot_m = yc + math.sin(math.radians(origi_angle+180))*rminor
#determine which region is up and which is down
if max(xtop, xbot) == xtop :
x_tij = xtop
y_tij = ytop
x_b_tij = xbot
y_b_tij = ybot
else:
x_tij = xbot
y_tij = ybot
x_b_tij = xtop
y_b_tij = ytop
if max(xtop_m, xbot_m) == xtop_m :
x_tij_m = xtop_m
y_tij_m = ytop_m
x_b_tij_m = xbot_m
y_b_tij_m = ybot_m
else:
x_tij_m = xbot_m
y_tij_m = ybot_m
x_b_tij_m = xtop_m
y_b_tij_m = ytop_m
print('-----')
print(x_tij, y_tij)
rect_size = 100
"""
calculate regions of edges of major axis of ellipse
this is done by creating a squared region of rect_size x rect_size, being the edge the center of the square
"""
x_min_tij = int(0 if x_tij - rect_size < 0 else x_tij - rect_size)
x_max_tij = int(thresh.shape[1]-1 if x_tij + rect_size > thresh.shape[1] else x_tij + rect_size)
y_min_tij = int(0 if y_tij - rect_size < 0 else y_tij - rect_size)
y_max_tij = int(thresh.shape[0] - 1 if y_tij + rect_size > thresh.shape[0] else y_tij + rect_size)
x_b_min_tij = int(0 if x_b_tij - rect_size < 0 else x_b_tij - rect_size)
x_b_max_tij = int(thresh.shape[1] - 1 if x_b_tij + rect_size > thresh.shape[1] else x_b_tij + rect_size)
y_b_min_tij = int(0 if y_b_tij - rect_size < 0 else y_b_tij - rect_size)
y_b_max_tij = int(thresh.shape[0] - 1 if y_b_tij + rect_size > thresh.shape[0] else y_b_tij + rect_size)
sum_red_region = np.sum(thresh[y_min_tij:y_max_tij, x_min_tij:x_max_tij])
sum_yellow_region = np.sum(thresh[y_b_min_tij:y_b_max_tij, x_b_min_tij:x_b_max_tij])
"""
calculate regions of edges of minor axis of ellipse
this is done by creating a squared region of rect_size x rect_size, being the edge the center of the square
"""
x_min_tij_m = int(0 if x_tij_m - rect_size < 0 else x_tij_m - rect_size)
x_max_tij_m = int(thresh.shape[1]-1 if x_tij_m + rect_size > thresh.shape[1] else x_tij_m + rect_size)
y_min_tij_m = int(0 if y_tij_m - rect_size < 0 else y_tij_m - rect_size)
y_max_tij_m = int(thresh.shape[0] - 1 if y_tij_m + rect_size > thresh.shape[0] else y_tij_m + rect_size)
x_b_min_tij_m = int(0 if x_b_tij_m - rect_size < 0 else x_b_tij_m - rect_size)
x_b_max_tij_m = int(thresh.shape[1] - 1 if x_b_tij_m + rect_size > thresh.shape[1] else x_b_tij_m + rect_size)
y_b_min_tij_m = int(0 if y_b_tij_m - rect_size < 0 else y_b_tij_m - rect_size)
y_b_max_tij_m = int(thresh.shape[0] - 1 if y_b_tij_m + rect_size > thresh.shape[0] else y_b_tij_m + rect_size)
#value of the regions, the names of the variables are related to the color of the rectangles drawn at the end of the function
sum_red_region_m = np.sum(thresh[y_min_tij_m:y_max_tij_m, x_min_tij_m:x_max_tij_m])
sum_yellow_region_m = np.sum(thresh[y_b_min_tij_m:y_b_max_tij_m, x_b_min_tij_m:x_b_max_tij_m])
#print(sum_red_region, sum_yellow_region, sum_red_region_m, sum_yellow_region_m)
min_arg = np.argmin(np.array([sum_red_region, sum_yellow_region, sum_red_region_m, sum_yellow_region_m]))
print('min: ', min_arg)
if min_arg == 1: #sum_yellow_region < sum_red_region :
left_quartile = x_b_tij < thresh.shape[0] /2
upper_quartile = y_b_tij < thresh.shape[1] /2
center_x = x_b_min_tij + ((x_b_max_tij - x_b_min_tij) / 2)
center_y = y_b_min_tij + (y_b_max_tij - y_b_min_tij / 2)
center_x = x_b_min_tij + np.argmax(thresh[y_b_min_tij:y_b_max_tij, x_b_min_tij:x_b_max_tij].mean(axis=0))
center_y = y_b_min_tij + np.argmax(thresh[y_b_min_tij:y_b_max_tij, x_b_min_tij:x_b_max_tij].mean(axis=1))
elif min_arg == 0:
left_quartile = x_tij < thresh.shape[0] /2
upper_quartile = y_tij < thresh.shape[1] /2
center_x = x_min_tij + ((x_b_max_tij - x_b_min_tij) / 2)
center_y = y_min_tij + ((y_b_max_tij - y_b_min_tij) / 2)
center_x = x_min_tij + np.argmax(thresh[y_min_tij:y_max_tij, x_min_tij:x_max_tij].mean(axis=0))
center_y = y_min_tij + np.argmax(thresh[y_min_tij:y_max_tij, x_min_tij:x_max_tij].mean(axis=1))
elif min_arg == 3:
left_quartile = x_b_tij_m < thresh.shape[0] /2
upper_quartile = y_b_tij_m < thresh.shape[1] /2
center_x = x_b_min_tij_m + ((x_b_max_tij_m - x_b_min_tij_m) / 2)
center_y = y_b_min_tij_m + (y_b_max_tij_m - y_b_min_tij_m / 2)
center_x = x_b_min_tij_m + np.argmax(thresh[y_b_min_tij_m:y_b_max_tij_m, x_b_min_tij_m:x_b_max_tij_m].mean(axis=0))
center_y = y_b_min_tij_m + np.argmax(thresh[y_b_min_tij_m:y_b_max_tij_m, x_b_min_tij_m:x_b_max_tij_m].mean(axis=1))
else:
left_quartile = x_tij_m < thresh.shape[0] /2
upper_quartile = y_tij_m < thresh.shape[1] /2
center_x = x_min_tij_m + ((x_b_max_tij_m - x_b_min_tij_m) / 2)
center_y = y_min_tij_m + ((y_b_max_tij_m - y_b_min_tij_m) / 2)
center_x = x_min_tij_m + np.argmax(thresh[y_min_tij_m:y_max_tij_m, x_min_tij_m:x_max_tij_m].mean(axis=0))
center_y = y_min_tij_m + np.argmax(thresh[y_min_tij_m:y_max_tij_m, x_min_tij_m:x_max_tij_m].mean(axis=1))
# draw ellipse on copy of input
result = img.copy()
cv2.ellipse(result, ellipse, (0,0,255), 1)
cv2.line(result, (int(xtop),int(ytop)), (int(xbot),int(ybot)), (255, 0, 0), 1)
cv2.circle(result, (int(xc),int(yc)), 10, (255, 255, 255), -1)
cv2.circle(result, (int(center_x),int(center_y)), 10, (255, 0, 255), 5)
cv2.circle(result, (int(thresh.shape[1] / 2),int(thresh.shape[0] - 1)), 10, (255, 0, 0), 5)
cv2.rectangle(result,(x_min_tij,y_min_tij),(x_max_tij,y_max_tij),(255,0,0),3)
cv2.rectangle(result,(x_b_min_tij,y_b_min_tij),(x_b_max_tij,y_b_max_tij),(255,255,0),3)
cv2.rectangle(result,(x_min_tij_m,y_min_tij_m),(x_max_tij_m,y_max_tij_m),(255,0,0),3)
cv2.rectangle(result,(x_b_min_tij_m,y_b_min_tij_m),(x_b_max_tij_m,y_b_max_tij_m),(255,255,0),3)
plt.imshow(result)
plt.figure()
#rotate the image
rot_img = Image.fromarray(thresh)
#180
bot_point_x = int(thresh.shape[1] / 2)
bot_point_y = int(thresh.shape[0] - 1)
#poi
poi_x = int(center_x)
poi_y = int(center_y)
#image_center
im_center_x = int(thresh.shape[1] / 2)
im_center_y = int(thresh.shape[0] - 1) / 2
#a - adalt, b - abaix, c - dreta
#ba = a - b
#bc = c - a(b en realitat)
ba = np.array([im_center_x, im_center_y]) - np.array([bot_point_x, bot_point_y])
bc = np.array([poi_x, poi_y]) - np.array([im_center_x, im_center_y])
#angle 3 punts
cosine_angle = np.dot(ba, bc) / (np.linalg.norm(ba) * np.linalg.norm(bc))
cos_angle = np.arccos(cosine_angle)
cos_angle = np.degrees(cos_angle)
print('cos angle: ', cos_angle)
print('print: ', abs(poi_x- bot_point_x))
m = (int(thresh.shape[1] / 2)-int(center_x) / int(thresh.shape[0] - 1)-int(center_y))
ttan = math.tan(m)
theta = math.atan(ttan)
print('theta: ', theta)
result = Image.fromarray(result)
result = result.rotate(cos_angle)
plt.imshow(result)
plt.figure()
#rot_img = rot_img.rotate(origi_angle)
rot_img = rot_img.rotate(cos_angle)
return rot_img
rot_img = get_thresholded_rotated(im_path)
plt.imshow(rot_img)
Thanks in advance --- EDIT ---
I leave here some raw images as requested.
解决方案
The Concept
This works for most of the leaves, as long as they have a stem. So here is the concept for detecting the rotation of and rotating one leaf image:
Find the approximated contour of the leaf. As the tip point of the stem will most often belong to the convex hull (the outer points) of the leaf, find the convex hull of the contour.
Loop through the indices of the contour that belong to the convex hull of the leaf. For each index, calculate the angle between 3 points: the point in the contours before the index, the point in the contours at the index and the point in the contours after the index.
The smallest angle calculated would be the tip of the stem. Every time the loop finds a smaller angle, store the three points in a tuple, and when the smallest angle gets detected, calculate the angle of which the stem is pointing at using the center of the 2 coordinates on both sides of the tip of the stem, and the tip of the stem.
With the angle of the stem detected we can rotate the image accordingly.
The Code
import cv2
import numpy as np
def process(img):
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_blur = cv2.GaussianBlur(img_gray, (3, 3), 2)
img_canny = cv2.Canny(img_blur, 127, 47)
kernel = np.ones((5, 5))
img_dilate = cv2.dilate(img_canny, kernel, iterations=2)
img_erode = cv2.erode(img_dilate, kernel, iterations=1)
return img_erode
def get_contours(img):
contours, _ = cv2.findContours(process(img), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cnt = max(contours, key=cv2.contourArea)
peri = cv2.arcLength(cnt, True)
return cv2.approxPolyDP(cnt, 0.01 * peri, True)
def get_angle(a, b, c):
ba, bc = a - b, c - b
cos_angle = np.dot(ba, bc) / (np.linalg.norm(ba) * np.linalg.norm(bc))
return np.degrees(np.arccos(cos_angle))
def get_rot_angle(img):
contours = get_contours(img)
length = len(contours)
min_angle = 180
for i in cv2.convexHull(contours, returnPoints=False).ravel():
a, b, c = contours[[i - 1, i, (i + 1) % length], 0]
angle = get_angle(a, b, c)
if angle < min_angle:
min_angle = angle
pts = a, b, c
a, b, c = pts
return 180 - np.degrees(np.arctan2(*(np.mean((a, c), 0) - b)))
def rotate(img):
h, w, _ = img.shape
rot_mat = cv2.getRotationMatrix2D((w / 2, h / 2), get_rot_angle(img), 1)
return cv2.warpAffine(img, rot_mat, (w, h), flags=cv2.INTER_LINEAR)
img = cv2.imread("leaf.jpg")
cv2.imshow("Image", rotate(img))
cv2.waitKey(0)
The Output
Output for every sample image you provided:
The Explanation
Breaking down the code:
- Import the necessary libraries:
import cv2
import numpy as np
- Define a function,
process, to process an image into a binary image that will allow the program to accurately detect the contours of the leaves:
def process(img):
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img_blur = cv2.GaussianBlur(img_gray, (3, 3), 2)
img_canny = cv2.Canny(img_blur, 127, 47)
kernel = np.ones((5, 5))
img_dilate = cv2.dilate(img_canny, kernel, iterations=2)
img_erode = cv2.erode(img_dilate, kernel, iterations=1)
return img_erode
- Define a function,
get_contours, to get the approximate contours of the largest contour in the image, using theprocessfunction defined before:
def get_contours(img):
contours, _ = cv2.findContours(process(img), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cnt = max(contours, key=cv2.contourArea)
peri = cv2.arcLength(cnt, True)
return cv2.approxPolyDP(cnt, 0.01 * peri, True)
- Define a function,
get_angle, to get the angle between 3 points:
def get_angle(a, b, c):
ba, bc = a - b, c - b
cos_angle = np.dot(ba, bc) / (np.linalg.norm(ba) * np.linalg.norm(bc))
return np.degrees(np.arccos(cos_angle))
- Define a function,
get_rot_angle, to get the amount of degrees the image needs to be rotated. It determines this angle by finding the point of the convex hull of the leaf where the angle between the point and the 2 surrounding points in the contour of the leaf where the angle between the 3 points is minimal, using theget_anglefunction defined before:
def get_rot_angle(img):
contours = get_contours(img)
length = len(contours)
min_angle = 180
for i in cv2.convexHull(contours, returnPoints=False).ravel():
a, b, c = contours[[i - 1, i, (i + 1) % length], 0]
angle = get_angle(a, b, c)
if angle < min_angle:
min_angle = angle
pts = a, b, c
a, b, c = pts
return 180 - np.degrees(np.arctan2(*(np.mean((a, c), 0) - b)))
- Define a function,
rotate, to rotate the image along its center, using theget_rot_anglefunction defined before:
def rotate(img):
h, w, _ = img.shape
rot_mat = cv2.getRotationMatrix2D((w / 2, h / 2), get_rot_angle(img), 1)
return cv2.warpAffine(img, rot_mat, (w, h), flags=cv2.INTER_LINEAR)
- Finally, read in your images, apply the
rotatefunction defined before and show the rotated image:
img = cv2.imread("leaf.jpg")
cv2.imshow("Image", rotate(img))
cv2.waitKey(0)
这篇关于如何根据对象位置旋转图像?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
