Python / OpenCV —在两个图像中匹配细菌的质心点 [英] Python/OpenCV — Matching Centroid Points of Bacteria in Two Images
问题描述
我正在研究一种使用计算机视觉匹配细菌质心的算法。
由于我是计算机视觉的本科生和初学者，因此我没有专门的代码问题。为了提供一些背景知识，我在GUI中使用了以下功能。
bact变量是指细菌对象，该对象存储每种细菌的ID，位置等。
defidentify_fluor（img，frame：int）：
darkBlue =（139，0，0）
代表fluor_at_frame（frame）中的细菌：
pos =元组（[int（coord）代表bact.position [frame]]中的坐标）
img = cv2.circle（img ，pos，5，darkBlue，-1）
返回img
defidentify_bright（img，frame：int）：
darkRed =（0，0，139）
对于Bright_at_frame（frame）中的bact：
pos =元组（[int（coord）表示bact.position [frame]中的坐标]））
img = cv2.circle（img，pos，5，darkRed，-1）
返回img
这些质心是使用当前图像处理文献中提供的最佳软件找到的。如您所见，右侧（明场）的处理图像显着欠发达，对于细菌学研究人员来说是一大障碍和麻烦。
我们需要在右侧显示这些图像才能因为它们具有明显更高的图像采样率（1秒[右] vs. 11秒[左]），所以进行了处理。荧光图像（左图）在采样过于频繁时会累积化学损伤，从而失去荧光。
在某些情况下，图像会完美对齐：
在这些情况下，图像
< a href = https://i.stack.imgur.com/CZTix.png rel = nofollow noreferrer>细菌匹配的样本6
明场图像
附加链接
此外，正如您在第三幅图像中看到的，即最左边的细菌，其中心没有精确地标记在中心。之所以发生这种情况，是因为在代码中，我在一处使用了轮廓的凸包。您可以通过跟踪所有轮廓来解决此问题，然后最后以初始轮廓的中心为准。
我确信也可以修改此代码并将其做得更好但这就是我现在能想到的。任何建议都受到欢迎。
I am working on an algorithm to match the centroids of bacteria using computer vision.
As I'm an undergraduate and beginner to computer vision, I do not have code specifically for this problem. Just to provide some background, I'm using the following functions in my GUI.
The 'bact' variable refers to Bacteria objects, which stores each bacteria's ID, position, etc.
def identify_fluor(img, frame: int):
darkBlue = (139, 0, 0)
for bact in fluor_at_frame(frame):
pos = tuple([int(coord) for coord in bact.position[frame]])
img = cv2.circle(img, pos, 5, darkBlue, -1)
return img
def identify_bright(img, frame: int):
darkRed = (0, 0, 139)
for bact in bright_at_frame(frame):
pos = tuple([int(coord) for coord in bact.position[frame]])
img = cv2.circle(img, pos, 5, darkRed, -1)
return img
These centroids are found using the best software available in current image-processing literature. As you can notice, the processing images on the right (bright-field) is significantly underdeveloped and is a significant hurdle and nuisance for bacteriology researchers.
We need these images on the right to be processed because they have a significantly greater sampling rate of images (1 second [Right] vs. 11 seconds [Left]). The fluorescence images (left) accumulate chemical damage when sampled too frequently, losing their florescence.
These are some instances when the images align perfectly:
In these cases, the images on the right are at an intermediate stage before reaching the next aligned image.
Bright-Field Images
Additional Links
Note: This is not homework. I am doing a research project trying to gain information on the temporal dynamics of bacteria. I am trying to achieve a working solution on one of the samples of the images.
Edit #1: For clarification, I am trying to find the centroids of the bacteria on the right using the bacteria on the left.
Edit #2: I am not looking to match the images by applying a linear transformation. A computer vision algorithm is sought.
Edit #3: Additional bright-field images have been added separately for testing purposes.
My approach works directly on the right image.
The code is shared below and explained with comments:
I am creating a function at the beginning which erodes and dilates the image with a circular kernel, specified number of times.
kernel = np.array([[0, 0, 1, 0, 0],
[0, 1, 1, 1, 0],
[1, 1, 1, 1, 1],
[0, 1, 1, 1, 0],
[0, 0, 1, 0, 0]], dtype=np.uint8)
def e_d(image, it):
image = cv2.erode(image, kernel, iterations=it)
image = cv2.dilate(image, kernel, iterations=it)
return image
Note: Image on the right is read in the grayscale format in the variable "img".
# Applying adaptive mean thresholding
th = cv2.adaptiveThreshold(img,255,cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV,11,2)
# Removing small noise
th = e_d(th.copy(), 1)
# Finding contours with RETR_EXTERNAL flag and removing undesired contours and
# drawing them on a new image.
cnt, hie = cv2.findContours(th, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cntImg = th.copy()
for contour in cnt:
x,y,w,h = cv2.boundingRect(contour)
# Elliminating the contour if its width is more than half of image width
# (bacteria will not be that big).
if w > img.shape[1]/2:
continue
cntImg = cv2.drawContours(cntImg, [cv2.convexHull(contour)], -1, 255, -1)
# Removing almost all the remaining noise.
# (Some big circular noise will remain along with bacteria contours)
cntImg = e_d(cntImg, 5)
# Finding new filtered contours again
cnt2, hie2 = cv2.findContours(cntImg, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
# Now eliminating circular type noise contours by comparing each contour's
# extent of overlap with its enclosing circle.
finalContours = [] # This will contain the final bacteria contours
for contour in cnt2:
# Finding minimum enclosing circle
(x,y),radius = cv2.minEnclosingCircle(contour)
center = (int(x),int(y))
radius = int(radius)
# creating a image with only this circle drawn on it(filled with white colour)
circleImg = np.zeros(img.shape, dtype=np.uint8)
circleImg = cv2.circle(circleImg, center, radius, 255, -1)
# creating a image with only the contour drawn on it(filled with white colour)
contourImg = np.zeros(img.shape, dtype=np.uint8)
contourImg = cv2.drawContours(contourImg, [contour], -1, 255, -1)
# White pixels not common in both contour and circle will remain white
# else will become black.
union_inter = cv2.bitwise_xor(circleImg, contourImg)
# Finding ratio of the extent of overlap of contour to its enclosing circle.
# Smaller the ratio, more circular the contour.
ratio = np.sum(union_inter == 255) / np.sum(circleImg == 255)
# Storing only non circular contours(bacteria)
if ratio > 0.55:
finalContours.append(contour)
finalContours = np.asarray(finalContours)
# Finding center of bacteria and showing it.
bacteriaImg = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
for bacteria in finalContours:
M = cv2.moments(bacteria)
cx = int(M['m10']/M['m00'])
cy = int(M['m01']/M['m00'])
bacteriaImg = cv2.circle(bacteriaImg, (cx, cy), 5, (0, 0, 255), -1)
cv2.imshow("bacteriaImg", bacteriaImg)
cv2.waitKey(0)
NOTE: I am only taking the image on the right and my image's size is (221, 828). If your input image is smaller or bigger than this, adjust the value of the number of iterations of erosion and dilation for removing noise accordingly to get good results.
Here are the output images:
Also, as you can see in the third image, the leftmost bacteria, its center is marked not exactly at the center. This is happening because, in the code, I have used convex hull of the contours in one place. You can solve this by keeping a track of all the contours and then at the end, take the center of the initial contour.
I am sure that this code can also be modified and made better but this is what I could think of right now. Any suggestions are most welcomed.
这篇关于Python / OpenCV —在两个图像中匹配细菌的质心点的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
