如何改善我的爪子检测？ [英] How can I improve my paw detection?
问题描述
在我之前关于，示例2 ，示例3 ）。 对于任何有兴趣的人我也设置了一个博客，让您及时更新： - ）
如果您只是想要（半）连续区域，那么在Python中已经有了一个简单的实现： SciPy 的 ndimage.morphology 模块。这是一种相当常见的图像形态操作。
基本上，你有5个步骤：
def find_paws（data，smooth_radius = 5，threshold = 0.0001）：
data = sp.ndimage.uniform_filter（data，smooth_radius）
thresh = data> threshold
filled = sp.ndimage.morphology.binary_fill_holes（thresh）
coded_paws，num_paws = sp.ndimage.label（filled）
data_slices = sp.ndimage.find_objects（coded_paws）
return object_slices
-
模糊输入数据确保爪子有足够的足迹。 （使用更大的内核（
结构kwarg到各种scipy.ndimage.morphology会更有效率）函数）但由于某种原因这不是很正常...） -
阈值数组，以便你有一个布尔数组的地方压力超过某个阈值（即
thresh = data> value） -
填写任何内部漏洞，这样你就拥有了更清洁的区域（
filled = sp.ndimage.morphology.binary_fill_holes（thresh）） -
找到单独的连续区域（
coded_paws，num_paws = sp.ndimage.label（已填充））。这将返回一个数组，其中的区域由数字编码（每个区域是一个唯一整数的连续区域（1到爪子的数量），其他地方都是零））。 -
使用
data_slices = sp.ndimage.find_objects（coded_paws）隔离连续区域。这将返回slice对象的元组列表，因此您可以使用[data [x] for x in获取每个爪子的数据区域data_slices]。相反，我们将根据这些切片绘制一个矩形，这需要稍多的工作。
下面的两个动画显示了重叠Paws和Grouped Paws示例数据。这种方法似乎工作得很好。 （无论它值多少，这比我机器下面的GIF图像运行得更顺畅，所以爪子检测算法相当快......）
这是一个完整的例子（现在有更详细的解释）。绝大多数是阅读输入和制作动画。实际的爪子检测只有5行代码。
import numpy as np
import scipy as sp
import scipy.ndimage
从matplotlib.patches导入matplotlib.pyplot为plt
导入矩形
def animate（input_filename）：
检测爪子并设置输入文件中每帧
的位置和原始数据
＃使用matplotlib，只需更新显示对象的属性
＃就快得多而不是创建一个新的，所以我们只需更新
＃整个动画中相同对象的数据和位置...
infile = paw_file（input_filename）
＃由于我们用matplotlib制作动画，我们需要
＃ion（）而不是show（）...
plt.ion（）
fig = plt.figure（）
ax = fig.add_subplot（111）
fig.suptitle（input_filename）
＃根据我们稍后会更新的第一帧制作图像
＃（第一帧从未实际显示）
im = ax.imshow（infile.next（）[1]）$ b
$ b＃制作4个矩形，我们以后可以移动到每个爪子的位置
rects = [矩形（（0,0），1,1，fc ='无'，ec ='红色'）for i in range（4）]
[ ax.add_patch（rect）for rect in rects]
title = ax.set_title（'Time 0.0 ms'）
＃处理并显示每一帧
for时间，框架在infile中：
paw_slices = find_paws（框架）
＃隐藏任何可见的矩形
[rect.set_visible（False）for rect in rects]
＃设置每个爪子的矩形的位置和大小，并显示它为
为slice，rect为zip（paw_slices，rects）：
dy，dx = slice
rect.set_xy（（dx.start，dy.start））
rect.set_width（dx.stop - dx.start + 1）
rect.set_height（dy.stop - dy.start + 1）
rect.set_visible（T rue）
＃更新图表的图像数据和标题
title.set_text（'时间％0.2f ms'％时间）
im.set_data（frame）
im.set_clim（[frame.min（），frame.max（）]）
fig.canvas.draw（）
def find_paws（data，smooth_radius = 5，threshold = 0.0001）：
检测并隔离输入数组中的连续区域
＃稍微模糊输入数据，以便爪子具有连续的足迹
data = sp.ndimage。 uniform_filter（data，smooth_radius）
＃模糊数据的阈值（这需要有点> 0由于模糊）
thresh = data>阈值
＃填充爪子中的任何内部孔以获得更清洁的区域...
filled = sp.ndimage.morphology.binary_fill_holes（thresh）
＃标记每个连续的爪子
coded_paws ，num_paws = sp.ndimage.label（已填充）
＃隔离每个爪子的范围
data_slices = sp.ndimage.find_objects（coded_paws）
返回data_slices
def paw_file（filename）：
返回一个迭代器，它产生每帧中的时间和数据
infile是一个ascii文件，其时间步长的格式与此类似：
第0帧（0.00毫秒）
0.0 0.0 0.0
0.0 0.0 0.0
第1帧（0.53毫秒）
0.0 0.0 0.0
0.0 0.0 0.0
...
，open（文件名）为infile：
而True：
try：
time，data = read_frame（infile）
收益时间，数据
除了StopIteration：
break
def read_f rame（infile）：
从infile读取一个帧。
frame_header = infile.next（）。strip（）。split（）
time = float（frame_header [-2] [1：]）
data = []
而True：
line = infile.next（）。strip（）。split（）
if line = = []：
中断
data.append（行）
返回时间，np.array（data，dtype = np.float）
如果__name__ == '__main__'：
animate（'Overlapping paws.bin'）
animate（'Grouped up paws.bin'）
animate（'Normal measurement.bin'）
更新：至于确定哪个爪子是在什么时候与传感器接触，最简单的解决方案是进行相同的分析，但一次使用所有数据。 （即将输入堆叠成3D数组，并使用它，而不是单独的时间帧。）因为SciPy的ndimage函数用于处理n维数组，所以我们不必修改原始的paw-finding函数完全没有。
＃这在前面的代码示例中使用了函数（和导入）！
def paw_regions（infile）：
＃读入并将所有数据一起堆叠成3D数组
data，time = []，[]
for t，frame in paw_file（ infile）：
time.append（t）
data.append（frame）
data = np.dstack（data）
time = np.asarray（time）
＃查找并标记爪子的影响
data_slices，coded_paws = find_paws（data，smooth_radius = 4）
＃按初始爪子影响的时间排序...这样我们用一个简单的模4可以确定哪一个
＃爪子与第一只爪子有关。
＃（假设一只四条腿的狗，所有4只爪子都联系了传感器）
data_slices.sort （key = lambda dat_slice：dat_slice [2] .start）
＃绘制一个简单的分析
fig = plt.figure（）
ax1 = fig.add_subplot（2， 1,1）
annotate_paw_prints（time，data，data_slices，ax = ax1）
ax2 = fig.add_subplot（2,1,2）
plot_paw_impacts（time，data_slices，ax = ax2）
fig.suptitle（inf ile）
def plot_paw_impacts（time，data_slices，ax = None）：如果ax为None，则为
：
ax = plt.gca（）
＃按爪子分组影响...
代表i，dat_slice代表枚举（data_slices）：
dx，dy，dt = dat_slice
paw = i％4 + 1
#Draw在每个爪子接触的时间间隔内的一个条形
ax.barh（bottom = paw，width = time [dt] .ptp（），height = 0.2，
left = time [dt]。 min（），align ='center'，color ='red'）
ax.set_yticks（range（1,5））
ax.set_yticklabels（['Paw 1'，'Paw 2'， 'Paw 3'，'Paw 4']）
ax.set_xlabel（'实验开始以来的时间（ms）'）
ax.yaxis.grid（True）
ax.set_title（ 'Paw Contact'周期'）
def annotate_paw_prints（时间，数据，data_slices，ax =无）：
如果ax为None：
ax = plt.gca（）
＃显示所有爪子影响（总和随时间变化）
ax.imshow（data.sum（axis = 2）.T）
＃用爪子注释每个影响
＃（相对于第一个击中传感器的爪子）
x，y = []，[]
表示i，区域为枚举（data_slices） ：
dx，dy，dz = region
＃获取x，y切片中心...
x0 = 0.5 *（dx.start + dx.stop）
y0 = 0.5 *（dy.start + dy.stop）
x.append（x0）; y.append（y0）
＃注释爪子影响
ax.annotate（'Paw％i'％（i％4 +1），（x0，y0），
color ='red'，ha ='center'，va ='bottom'）
＃绘制线连接爪撞击
ax.plot（x，y，' - wo'）
ax.axis（'image'）
ax.set_title（'步骤顺序'）
< hr>
After my previous question on finding toes within each paw, I started loading up other measurements to see how it would hold up. Unfortunately, I quickly ran into a problem with one of the preceding steps: recognizing the paws.
You see, my proof of concept basically took the maximal pressure of each sensor over time and would start looking for the sum of each row, until it finds on that != 0.0. Then it does the same for the columns and as soon as it finds more than 2 rows with that are zero again. It stores the minimal and maximal row and column values to some index.
As you can see in the figure, this works quite well in most cases. However, there are a lot of downsides to this approach (other than being very primitive):
Humans can have 'hollow feet' which means there are several empty rows within the footprint itself. Since I feared this could happen with (large) dogs too, I waited for at least 2 or 3 empty rows before cutting off the paw.
This creates a problem if another contact made in a different column before it reaches several empty rows, thus expanding the area. I figure I could compare the columns and see if they exceed a certain value, they must be separate paws.
The problem gets worse when the dog is very small or walks at a higher pace. What happens is that the front paw's toes are still making contact, while the hind paw's toes just start to make contact within the same area as the front paw!
With my simple script, it won't be able to split these two, because it would have to determine which frames of that area belong to which paw, while currently I would only have to look at the maximal values over all frames.
Examples of where it starts going wrong:
So now I'm looking for a better way of recognizing and separating the paws (after which I'll get to the problem of deciding which paw it is!).
Update:
I've been tinkering to get Joe's (awesome!) answer implemented, but I'm having difficulties extracting the actual paw data from my files.
The coded_paws shows me all the different paws, when applied to the maximal pressure image (see above). However, the solution goes over each frame (to separate overlapping paws) and sets the four Rectangle attributes, such as coordinates or height/width.
I can't figure out how to take these attributes and store them in some variable that I can apply to the measurement data. Since I need to know for each paw, what its location is during which frames and couple this to which paw it is (front/hind, left/right).
So how can I use the Rectangles attributes to extract these values for each paw?
I have the measurements I used in the question setup in my public Dropbox folder (example 1, example 2, example 3). For anyone interested I also set up a blog to keep you up to date :-)
If you're just wanting (semi) contiguous regions, there's already an easy implementation in Python: SciPy's ndimage.morphology module. This is a fairly common image morphology operation.
Basically, you have 5 steps:
def find_paws(data, smooth_radius=5, threshold=0.0001):
data = sp.ndimage.uniform_filter(data, smooth_radius)
thresh = data > threshold
filled = sp.ndimage.morphology.binary_fill_holes(thresh)
coded_paws, num_paws = sp.ndimage.label(filled)
data_slices = sp.ndimage.find_objects(coded_paws)
return object_slices
Blur the input data a bit to make sure the paws have a continuous footprint. (It would be more efficient to just use a larger kernel (the
structurekwarg to the variousscipy.ndimage.morphologyfunctions) but this isn't quite working properly for some reason...)Threshold the array so that you have a boolean array of places where the pressure is over some threshold value (i.e.
thresh = data > value)Fill any internal holes, so that you have cleaner regions (
filled = sp.ndimage.morphology.binary_fill_holes(thresh))Find the separate contiguous regions (
coded_paws, num_paws = sp.ndimage.label(filled)). This returns an array with the regions coded by number (each region is a contiguous area of a unique integer (1 up to the number of paws) with zeros everywhere else)).Isolate the contiguous regions using
data_slices = sp.ndimage.find_objects(coded_paws). This returns a list of tuples ofsliceobjects, so you could get the region of the data for each paw with[data[x] for x in data_slices]. Instead, we'll draw a rectangle based on these slices, which takes slightly more work.
The two animations below show your "Overlapping Paws" and "Grouped Paws" example data. This method seems to be working perfectly. (And for whatever it's worth, this runs much more smoothly than the GIF images below on my machine, so the paw detection algorithm is fairly fast...)
Here's a full example (now with much more detailed explanations). The vast majority of this is reading the input and making an animation. The actual paw detection is only 5 lines of code.
import numpy as np
import scipy as sp
import scipy.ndimage
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
def animate(input_filename):
"""Detects paws and animates the position and raw data of each frame
in the input file"""
# With matplotlib, it's much, much faster to just update the properties
# of a display object than it is to create a new one, so we'll just update
# the data and position of the same objects throughout this animation...
infile = paw_file(input_filename)
# Since we're making an animation with matplotlib, we need
# ion() instead of show()...
plt.ion()
fig = plt.figure()
ax = fig.add_subplot(111)
fig.suptitle(input_filename)
# Make an image based on the first frame that we'll update later
# (The first frame is never actually displayed)
im = ax.imshow(infile.next()[1])
# Make 4 rectangles that we can later move to the position of each paw
rects = [Rectangle((0,0), 1,1, fc='none', ec='red') for i in range(4)]
[ax.add_patch(rect) for rect in rects]
title = ax.set_title('Time 0.0 ms')
# Process and display each frame
for time, frame in infile:
paw_slices = find_paws(frame)
# Hide any rectangles that might be visible
[rect.set_visible(False) for rect in rects]
# Set the position and size of a rectangle for each paw and display it
for slice, rect in zip(paw_slices, rects):
dy, dx = slice
rect.set_xy((dx.start, dy.start))
rect.set_width(dx.stop - dx.start + 1)
rect.set_height(dy.stop - dy.start + 1)
rect.set_visible(True)
# Update the image data and title of the plot
title.set_text('Time %0.2f ms' % time)
im.set_data(frame)
im.set_clim([frame.min(), frame.max()])
fig.canvas.draw()
def find_paws(data, smooth_radius=5, threshold=0.0001):
"""Detects and isolates contiguous regions in the input array"""
# Blur the input data a bit so the paws have a continous footprint
data = sp.ndimage.uniform_filter(data, smooth_radius)
# Threshold the blurred data (this needs to be a bit > 0 due to the blur)
thresh = data > threshold
# Fill any interior holes in the paws to get cleaner regions...
filled = sp.ndimage.morphology.binary_fill_holes(thresh)
# Label each contiguous paw
coded_paws, num_paws = sp.ndimage.label(filled)
# Isolate the extent of each paw
data_slices = sp.ndimage.find_objects(coded_paws)
return data_slices
def paw_file(filename):
"""Returns a iterator that yields the time and data in each frame
The infile is an ascii file of timesteps formatted similar to this:
Frame 0 (0.00 ms)
0.0 0.0 0.0
0.0 0.0 0.0
Frame 1 (0.53 ms)
0.0 0.0 0.0
0.0 0.0 0.0
...
"""
with open(filename) as infile:
while True:
try:
time, data = read_frame(infile)
yield time, data
except StopIteration:
break
def read_frame(infile):
"""Reads a frame from the infile."""
frame_header = infile.next().strip().split()
time = float(frame_header[-2][1:])
data = []
while True:
line = infile.next().strip().split()
if line == []:
break
data.append(line)
return time, np.array(data, dtype=np.float)
if __name__ == '__main__':
animate('Overlapping paws.bin')
animate('Grouped up paws.bin')
animate('Normal measurement.bin')
Update: As far as identifying which paw is in contact with the sensor at what times, the simplest solution is to just do the same analysis, but use all of the data at once. (i.e. stack the input into a 3D array, and work with it, instead of the individual time frames.) Because SciPy's ndimage functions are meant to work with n-dimensional arrays, we don't have to modify the original paw-finding function at all.
# This uses functions (and imports) in the previous code example!!
def paw_regions(infile):
# Read in and stack all data together into a 3D array
data, time = [], []
for t, frame in paw_file(infile):
time.append(t)
data.append(frame)
data = np.dstack(data)
time = np.asarray(time)
# Find and label the paw impacts
data_slices, coded_paws = find_paws(data, smooth_radius=4)
# Sort by time of initial paw impact... This way we can determine which
# paws are which relative to the first paw with a simple modulo 4.
# (Assuming a 4-legged dog, where all 4 paws contacted the sensor)
data_slices.sort(key=lambda dat_slice: dat_slice[2].start)
# Plot up a simple analysis
fig = plt.figure()
ax1 = fig.add_subplot(2,1,1)
annotate_paw_prints(time, data, data_slices, ax=ax1)
ax2 = fig.add_subplot(2,1,2)
plot_paw_impacts(time, data_slices, ax=ax2)
fig.suptitle(infile)
def plot_paw_impacts(time, data_slices, ax=None):
if ax is None:
ax = plt.gca()
# Group impacts by paw...
for i, dat_slice in enumerate(data_slices):
dx, dy, dt = dat_slice
paw = i%4 + 1
# Draw a bar over the time interval where each paw is in contact
ax.barh(bottom=paw, width=time[dt].ptp(), height=0.2,
left=time[dt].min(), align='center', color='red')
ax.set_yticks(range(1, 5))
ax.set_yticklabels(['Paw 1', 'Paw 2', 'Paw 3', 'Paw 4'])
ax.set_xlabel('Time (ms) Since Beginning of Experiment')
ax.yaxis.grid(True)
ax.set_title('Periods of Paw Contact')
def annotate_paw_prints(time, data, data_slices, ax=None):
if ax is None:
ax = plt.gca()
# Display all paw impacts (sum over time)
ax.imshow(data.sum(axis=2).T)
# Annotate each impact with which paw it is
# (Relative to the first paw to hit the sensor)
x, y = [], []
for i, region in enumerate(data_slices):
dx, dy, dz = region
# Get x,y center of slice...
x0 = 0.5 * (dx.start + dx.stop)
y0 = 0.5 * (dy.start + dy.stop)
x.append(x0); y.append(y0)
# Annotate the paw impacts
ax.annotate('Paw %i' % (i%4 +1), (x0, y0),
color='red', ha='center', va='bottom')
# Plot line connecting paw impacts
ax.plot(x,y, '-wo')
ax.axis('image')
ax.set_title('Order of Steps')
这篇关于如何改善我的爪子检测？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
