如何改进我的爪子检测? [英] How can I improve my paw detection?
问题描述
在我上一个关于、示例 2、示例 3).对于任何有兴趣的人，我还建立了一个博客让您了解最新情况:-)
如果您只是想要(半)连续区域，Python 中已经有一个简单的实现:SciPy 的 ndimage.morphology 模块.这是一种相当常见的图像形态学操作.
<小时>基本上，您有 5 个步骤:
def find_paws(data, smooth_radius=5, threshold=0.0001):数据 = sp.ndimage.uniform_filter(数据，smooth_radius)阈值 = 数据 >临界点填充 = sp.ndimage.morphology.binary_fill_holes(thresh)coded_paws，num_paws = sp.ndimage.label(填充)data_slices = sp.ndimage.find_objects(coded_paws)返回 object_slices稍微模糊输入数据以确保爪子具有连续的足迹.(只使用更大的内核(
structurekwarg 到各种scipy.ndimage.morphology函数)会更有效，但这对某些人来说不太正常原因...)对数组设置阈值，以便您有一个布尔数组，其中包含压力超过某个阈值的位置(即
thresh = data > value)填充所有内部孔洞，以便您拥有更干净的区域 (
filled = sp.ndimage.morphology.binary_fill_holes(thresh))找到单独的连续区域(
coded_paws, num_paws = sp.ndimage.label(filled)).这将返回一个数组，其中包含按数字编码的区域(每个区域是一个唯一整数的连续区域(1 到爪子的数量)，其他地方都是零)).使用
data_slices = sp.ndimage.find_objects(coded_paws)隔离连续区域.这将返回slice对象的元组列表，因此您可以使用[data[x] for x in data_slices]获取每个爪子的数据区域.相反，我们将根据这些切片绘制一个矩形，这需要做更多的工作.
<小时>
下面的两个动画显示了您的重叠爪子"和分组爪子"示例数据.这种方法似乎工作得很好.(不管它值多少钱，这比我机器上的下面的 GIF 图像运行得更流畅，所以爪子检测算法相当快......)
<小时>这是一个完整的例子(现在有更详细的解释).其中绝大多数是读取输入并制作动画.实际爪子检测只有 5 行代码.
将 numpy 导入为 np将 scipy 导入为 sp导入 scipy.ndimage导入 matplotlib.pyplot 作为 plt从 matplotlib.patches 导入矩形定义动画(输入文件名):"""检测爪子并动画每帧的位置和原始数据在输入文件"""# 使用 matplotlib，更新属性要快得多# 一个显示对象比创建一个新对象更重要，所以我们只需更新# 整个动画中相同对象的数据和位置...infile = paw_file(input_filename)# 由于我们正在使用 matplotlib 制作动画，我们需要# ion() 而不是 show()...plt.ion()fig = plt.figure()ax = fig.add_subplot(111)fig.suptitle(input_filename)# 根据我们稍后更新的第一帧制作图像#(第一帧从不实际显示)im = ax.imshow(infile.next()[1])# 制作 4 个矩形，我们可以稍后将其移动到每个爪子的位置rects = [Rectangle((0,0), 1,1, fc='none', ec='red') for i in range(4)][ax.add_patch(rect) 用于矩形中的矩形]title = ax.set_title('时间 0.0 毫秒')# 处理并显示每一帧对于时间，文件中的帧:paw_slices = find_paws(frame)# 隐藏任何可能可见的矩形[rect.set_visible(False) 用于 rects 中的 rect]# 为每个爪子设置一个矩形的位置和大小并显示它对于切片，zip 中的矩形(paw_slices，rects):dy, dx = 切片rect.set_xy((dx.start, dy.start))rect.set_width(dx.stop - dx.start + 1)rect.set_height(dy.stop - dy.start + 1)rect.set_visible(真)# 更新图的图像数据和标题title.set_text('时间 %0.2f ms' % 时间)im.set_data(frame)im.set_clim([frame.min(), frame.max()])fig.canvas.draw()def find_paws(数据，smooth_radius=5，阈值=0.0001):"""检测并隔离输入数组中的连续区域"""# 稍微模糊输入数据，使爪子有连续的足迹数据 = sp.ndimage.uniform_filter(数据，smooth_radius)# 模糊数据的阈值(由于模糊，这需要有点> 0)阈值 = 数据 >临界点# 填充爪子上的任何内部孔以获得更清洁的区域...填充 = sp.ndimage.morphology.binary_fill_holes(thresh)# 标记每个连续的爪子coded_paws，num_paws = sp.ndimage.label(填充)# 隔离每个爪子的范围data_slices = sp.ndimage.find_objects(coded_paws)返回数据切片def paw_file(文件名):"""返回一个迭代器，它产生每一帧的时间和数据infile 是一个时间步长的 ascii 文件，格式类似于:帧 0(0.00 毫秒)0.0 0.0 0.00.0 0.0 0.0第 1 帧(0.53 毫秒)0.0 0.0 0.00.0 0.0 0.0..."""使用 open(filename) 作为 infile:而真:尝试:时间，数据 = read_frame(infile)屈服时间，数据除了停止迭代:休息def read_frame(infile):"""从 infile 中读取一个帧."""frame_header = infile.next().strip().split()时间 = 浮动(frame_header[-2][1:])数据 = []而真:行 = infile.next().strip().split()如果行 == []:休息数据附加(行)返回时间，np.array(数据，dtype=np.float)如果 __name__ == '__main__':animate('重叠爪子.bin')animate('分组 paws.bin')animate('正常测量.bin')<小时>
更新:至于确定哪个爪子在什么时间与传感器接触，最简单的解决方案是进行相同的分析，但一次使用所有数据.(即将输入堆叠到一个 3D 数组中，并使用它，而不是单个时间帧.)因为 SciPy 的 ndimage 函数旨在处理 n 维数组，所以我们不必修改原始的爪子查找函数完全没有.
# 这使用了前面代码示例中的函数(和导入)！！def paw_regions(infile):# 读入所有数据并将其堆叠到一个 3D 数组中数据，时间 = [], []对于 t，paw_file(infile) 中的帧:时间.append(t)数据附加(框架)数据 = np.dstack(data)时间 = np.asarray(time)# 找到并标记爪子的影响data_slices, coded_paws = find_paws(data, smooth_radius=4)# 按初始爪子撞击的时间排序...这样我们就可以确定哪个# paws 是相对于第一个 paw 的简单模 4.#(假设一只 4 条腿的狗，所有 4 个爪子都接触传感器)data_slices.sort(key=lambda dat_slice: dat_slice[2].start)# 绘制一个简单的分析fig = plt.figure()ax1 = fig.add_subplot(2,1,1)annotate_paw_prints(时间，数据，data_slices，ax=ax1)ax2 = fig.add_subplot(2,1,2)plot_paw_impacts(时间，data_slices，ax=ax2)fig.suptitle(infile)def plot_paw_impacts(time, data_slices, ax=None):如果 ax 是 None:ax = plt.gca()# 按爪子分组影响...对于我，枚举中的 dat_slice(data_slices):dx, dy, dt = dat_slice爪子 = i%4 + 1# 在每个爪子接触的时间间隔上画一个条ax.barh(底部=爪子，宽度=时间[dt].ptp()，高度=0.2，left=time[dt].min(), align='center', color='red')ax.set_yticks(范围(1, 5))ax.set_yticklabels(['爪子 1', '爪子 2', '爪子 3', '爪子 4'])ax.set_xlabel('自实验开始的时间 (ms)')ax.yaxis.grid(真)ax.set_title('爪子接触时间')def annotate_paw_prints(time, data, data_slices, ax=None):如果 ax 是 None:ax = plt.gca()# 显示所有爪子的影响(随着时间的推移总和)ax.imshow(data.sum(axis=2).T)# 注释每个撞击是哪个爪子#(相对于第一个爪子击中传感器)x, y = [], []对于 i，enumerate(data_slices) 中的区域:dx, dy, dz = 区域# 获取切片的 x,y 中心...x0 = 0.5 * (dx.start + dx.stop)y0 = 0.5 * (dy.start + dy.stop)x.append(x0);y.append(y0)# 注释爪子的影响ax.annotate('爪子 %i' % (i%4 +1), (x0, y0),颜色='红色', ha='center', va='bottom')# 绘制连接爪子影响的线ax.plot(x,y, '-wo')ax.axis('图像')ax.set_title('步骤顺序')After my previous question on finding toes within each paw, I started loading up other measurements to see how it would hold up. Unfortunately, I quickly ran into a problem with one of the preceding steps: recognizing the paws.
You see, my proof of concept basically took the maximal pressure of each sensor over time and would start looking for the sum of each row, until it finds on that != 0.0. Then it does the same for the columns and as soon as it finds more than 2 rows with that are zero again. It stores the minimal and maximal row and column values to some index.
As you can see in the figure, this works quite well in most cases. However, there are a lot of downsides to this approach (other than being very primitive):
Humans can have 'hollow feet' which means there are several empty rows within the footprint itself. Since I feared this could happen with (large) dogs too, I waited for at least 2 or 3 empty rows before cutting off the paw.
This creates a problem if another contact made in a different column before it reaches several empty rows, thus expanding the area. I figure I could compare the columns and see if they exceed a certain value, they must be separate paws.
The problem gets worse when the dog is very small or walks at a higher pace. What happens is that the front paw's toes are still making contact, while the hind paw's toes just start to make contact within the same area as the front paw!
With my simple script, it won't be able to split these two, because it would have to determine which frames of that area belong to which paw, while currently I would only have to look at the maximal values over all frames.
Examples of where it starts going wrong:
So now I'm looking for a better way of recognizing and separating the paws (after which I'll get to the problem of deciding which paw it is!).
Update:
I've been tinkering to get Joe's (awesome!) answer implemented, but I'm having difficulties extracting the actual paw data from my files.
The coded_paws shows me all the different paws, when applied to the maximal pressure image (see above). However, the solution goes over each frame (to separate overlapping paws) and sets the four Rectangle attributes, such as coordinates or height/width.
I can't figure out how to take these attributes and store them in some variable that I can apply to the measurement data. Since I need to know for each paw, what its location is during which frames and couple this to which paw it is (front/hind, left/right).
So how can I use the Rectangles attributes to extract these values for each paw?
I have the measurements I used in the question setup in my public Dropbox folder (example 1, example 2, example 3). For anyone interested I also set up a blog to keep you up to date :-)
If you're just wanting (semi) contiguous regions, there's already an easy implementation in Python: SciPy's ndimage.morphology module. This is a fairly common image morphology operation.
Basically, you have 5 steps:
def find_paws(data, smooth_radius=5, threshold=0.0001):
data = sp.ndimage.uniform_filter(data, smooth_radius)
thresh = data > threshold
filled = sp.ndimage.morphology.binary_fill_holes(thresh)
coded_paws, num_paws = sp.ndimage.label(filled)
data_slices = sp.ndimage.find_objects(coded_paws)
return object_slices
Blur the input data a bit to make sure the paws have a continuous footprint. (It would be more efficient to just use a larger kernel (the
structurekwarg to the variousscipy.ndimage.morphologyfunctions) but this isn't quite working properly for some reason...)Threshold the array so that you have a boolean array of places where the pressure is over some threshold value (i.e.
thresh = data > value)Fill any internal holes, so that you have cleaner regions (
filled = sp.ndimage.morphology.binary_fill_holes(thresh))Find the separate contiguous regions (
coded_paws, num_paws = sp.ndimage.label(filled)). This returns an array with the regions coded by number (each region is a contiguous area of a unique integer (1 up to the number of paws) with zeros everywhere else)).Isolate the contiguous regions using
data_slices = sp.ndimage.find_objects(coded_paws). This returns a list of tuples ofsliceobjects, so you could get the region of the data for each paw with[data[x] for x in data_slices]. Instead, we'll draw a rectangle based on these slices, which takes slightly more work.
The two animations below show your "Overlapping Paws" and "Grouped Paws" example data. This method seems to be working perfectly. (And for whatever it's worth, this runs much more smoothly than the GIF images below on my machine, so the paw detection algorithm is fairly fast...)
Here's a full example (now with much more detailed explanations). The vast majority of this is reading the input and making an animation. The actual paw detection is only 5 lines of code.
import numpy as np
import scipy as sp
import scipy.ndimage
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
def animate(input_filename):
"""Detects paws and animates the position and raw data of each frame
in the input file"""
# With matplotlib, it's much, much faster to just update the properties
# of a display object than it is to create a new one, so we'll just update
# the data and position of the same objects throughout this animation...
infile = paw_file(input_filename)
# Since we're making an animation with matplotlib, we need
# ion() instead of show()...
plt.ion()
fig = plt.figure()
ax = fig.add_subplot(111)
fig.suptitle(input_filename)
# Make an image based on the first frame that we'll update later
# (The first frame is never actually displayed)
im = ax.imshow(infile.next()[1])
# Make 4 rectangles that we can later move to the position of each paw
rects = [Rectangle((0,0), 1,1, fc='none', ec='red') for i in range(4)]
[ax.add_patch(rect) for rect in rects]
title = ax.set_title('Time 0.0 ms')
# Process and display each frame
for time, frame in infile:
paw_slices = find_paws(frame)
# Hide any rectangles that might be visible
[rect.set_visible(False) for rect in rects]
# Set the position and size of a rectangle for each paw and display it
for slice, rect in zip(paw_slices, rects):
dy, dx = slice
rect.set_xy((dx.start, dy.start))
rect.set_width(dx.stop - dx.start + 1)
rect.set_height(dy.stop - dy.start + 1)
rect.set_visible(True)
# Update the image data and title of the plot
title.set_text('Time %0.2f ms' % time)
im.set_data(frame)
im.set_clim([frame.min(), frame.max()])
fig.canvas.draw()
def find_paws(data, smooth_radius=5, threshold=0.0001):
"""Detects and isolates contiguous regions in the input array"""
# Blur the input data a bit so the paws have a continous footprint
data = sp.ndimage.uniform_filter(data, smooth_radius)
# Threshold the blurred data (this needs to be a bit > 0 due to the blur)
thresh = data > threshold
# Fill any interior holes in the paws to get cleaner regions...
filled = sp.ndimage.morphology.binary_fill_holes(thresh)
# Label each contiguous paw
coded_paws, num_paws = sp.ndimage.label(filled)
# Isolate the extent of each paw
data_slices = sp.ndimage.find_objects(coded_paws)
return data_slices
def paw_file(filename):
"""Returns a iterator that yields the time and data in each frame
The infile is an ascii file of timesteps formatted similar to this:
Frame 0 (0.00 ms)
0.0 0.0 0.0
0.0 0.0 0.0
Frame 1 (0.53 ms)
0.0 0.0 0.0
0.0 0.0 0.0
...
"""
with open(filename) as infile:
while True:
try:
time, data = read_frame(infile)
yield time, data
except StopIteration:
break
def read_frame(infile):
"""Reads a frame from the infile."""
frame_header = infile.next().strip().split()
time = float(frame_header[-2][1:])
data = []
while True:
line = infile.next().strip().split()
if line == []:
break
data.append(line)
return time, np.array(data, dtype=np.float)
if __name__ == '__main__':
animate('Overlapping paws.bin')
animate('Grouped up paws.bin')
animate('Normal measurement.bin')
Update: As far as identifying which paw is in contact with the sensor at what times, the simplest solution is to just do the same analysis, but use all of the data at once. (i.e. stack the input into a 3D array, and work with it, instead of the individual time frames.) Because SciPy's ndimage functions are meant to work with n-dimensional arrays, we don't have to modify the original paw-finding function at all.
# This uses functions (and imports) in the previous code example!!
def paw_regions(infile):
# Read in and stack all data together into a 3D array
data, time = [], []
for t, frame in paw_file(infile):
time.append(t)
data.append(frame)
data = np.dstack(data)
time = np.asarray(time)
# Find and label the paw impacts
data_slices, coded_paws = find_paws(data, smooth_radius=4)
# Sort by time of initial paw impact... This way we can determine which
# paws are which relative to the first paw with a simple modulo 4.
# (Assuming a 4-legged dog, where all 4 paws contacted the sensor)
data_slices.sort(key=lambda dat_slice: dat_slice[2].start)
# Plot up a simple analysis
fig = plt.figure()
ax1 = fig.add_subplot(2,1,1)
annotate_paw_prints(time, data, data_slices, ax=ax1)
ax2 = fig.add_subplot(2,1,2)
plot_paw_impacts(time, data_slices, ax=ax2)
fig.suptitle(infile)
def plot_paw_impacts(time, data_slices, ax=None):
if ax is None:
ax = plt.gca()
# Group impacts by paw...
for i, dat_slice in enumerate(data_slices):
dx, dy, dt = dat_slice
paw = i%4 + 1
# Draw a bar over the time interval where each paw is in contact
ax.barh(bottom=paw, width=time[dt].ptp(), height=0.2,
left=time[dt].min(), align='center', color='red')
ax.set_yticks(range(1, 5))
ax.set_yticklabels(['Paw 1', 'Paw 2', 'Paw 3', 'Paw 4'])
ax.set_xlabel('Time (ms) Since Beginning of Experiment')
ax.yaxis.grid(True)
ax.set_title('Periods of Paw Contact')
def annotate_paw_prints(time, data, data_slices, ax=None):
if ax is None:
ax = plt.gca()
# Display all paw impacts (sum over time)
ax.imshow(data.sum(axis=2).T)
# Annotate each impact with which paw it is
# (Relative to the first paw to hit the sensor)
x, y = [], []
for i, region in enumerate(data_slices):
dx, dy, dz = region
# Get x,y center of slice...
x0 = 0.5 * (dx.start + dx.stop)
y0 = 0.5 * (dy.start + dy.stop)
x.append(x0); y.append(y0)
# Annotate the paw impacts
ax.annotate('Paw %i' % (i%4 +1), (x0, y0),
color='red', ha='center', va='bottom')
# Plot line connecting paw impacts
ax.plot(x,y, '-wo')
ax.axis('image')
ax.set_title('Order of Steps')
这篇关于如何改进我的爪子检测?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
