如何整理我的爪子？ [英] How to sort my paws?
问题描述
在
我最初的想法是使用启发式方法进行排序，如：
- 前爪和后爪的重量比约为60-40％;
- 后爪是通常表面较小;
- 爪子（通常）在左右两侧进行空间划分。
<但是，我对我的启发式方法持怀疑态度，因为一旦遇到一个我没想过的变化，他们就会对我失败。他们也无法应对可能有自己规则的跛脚犬的测量结果。
此外，Joe建议的注释有时会搞砸，并没有考虑到爪子实际上是什么样的。
基于我收到的 （通过测量捆绑）和描述其位置的切片（位于板上的位置）并及时）。
澄清：walk_sliced_data是一个包含['ser_3'，'ser_2'，'sel_1'，'sel_2'，'ser_1'，'sel_3']的字典，这些是测量的名称。每个测量包含另一个字典，[0,1,2,3,4,5,6,7,8,9,10]（例如来自'sel_1'），表示提取的影响。
另请注意，可以忽略假影响，例如部分测量爪子的位置（空间或时间）。它们只是有用，因为它们可以帮助识别模式，但不会分析
。
对于任何有兴趣的人，我正在保持博客关于项目的所有更新！
好吧！我终于设法让事情始终如一！这个问题让我陷入了好几天......好玩的东西！很抱歉这个答案的长度，但我需要详细说明一些事情...（虽然我可能会设置有史以来最长的非垃圾邮件堆栈溢出答案！）
作为旁注，我正在使用Ivo 在的链接.com / questions / 3684484 / peak-detection-in-a-2d-array>原始问题。它是一系列rar文件（每个狗一个），每个文件包含存储为ascii数组的几个不同的实验运行。而不是尝试将独立代码示例复制粘贴到此问题中，这里是 bitbucket mercurial repository 使用完整的独立代码。您可以使用
hg clone https：//joferkington@bitbucket.org/joferkington/paw-analysis 克隆它
概述
那里正如您在问题中所指出的那样，基本上是解决问题的两种方法。我实际上会以不同的方式使用它们。
- 使用爪子撞击的（时间和空间）顺序来确定哪个爪子是什么。
- 尝试仅根据其形状来识别pawprint。
基本上，第一种方法适用于狗的爪子遵循上面Ivo问题中所示的梯形图案，但只要爪子不遵循该模式就会失败。以编程方式检测它何时不起作用相当容易。
因此，我们可以使用它所做的测量来建立一个训练数据集（来自~30只不同的狗的约2000爪子的影响）来识别哪个爪子是哪个，问题减少到监督分类（有一些额外的皱纹...图像识别比正常监督分类问题有点困难）。
模式分析
详细说明第一种方法，当狗走路时（不跑步！）通常（其中一些狗可能不是），我们希望爪子按以下顺序冲击：前左，后右，前右，后左，前左等。图案可以从左前方或前方开始右爪。
如果情况总是如此，我们可以简单地按照初始接触时间对影响进行排序，并使用模4按爪子对它们进行分组。
然而，即使一切都正常，这也行不通。这是由于图案的梯形形状。后爪在空间上落在前一个前爪后面。
因此，初始前爪撞击后的后爪撞击经常从传感器板上掉下来，并且没有记录。同样，最后的爪子撞击通常不是序列中的下一个爪子，因为爪子撞击传感器板之前没有记录。
尽管如此，我们可以使用的形状爪子撞击模式确定何时发生这种情况，以及我们是从左前爪还是右前爪开始。 （我实际上忽略了最后一次影响的问题。但是添加它并不太难。）
def group_paws （data_slices，time）：
＃按初始接触时间排序切片
data_slices.sort（key = lambda s：s [-1] .start）
＃获取质心对于每个爪子的影响......
paw_coords = []
for x，y，z in data_slices：
paw_coords.append（[（item.stop + item.start）/ 2.0 for item in（x，y）]）
paw_coords = np.array（paw_coords）
＃在每次成功影响之间制作一个向量...
dx，dy = np.diff （paw_coords，axis = 0）.T
＃ - 组爪子---------------------------- ---------------
paw_code = {0：'LF'，1：'RH'，2：'RF'，3：'LH'}
paw_number = np.arange（len（paw_coords））
＃在第一个
＃前爪影响后我们是否错过了后爪影响？如果是这样，首先dx将是正数...
如果dx [0]> 0：
paw_number [1：] + = 1
＃我们是从左前爪还是右前爪开始...
＃我们假设我们从左边开始，并检查dy [0]。
＃如果dy [0]> 0（即下一个爪子撞到左边），然后
＃它实际上是右前爪，而不是左边。
如果dy [0]> 0：＃右前爪撞击...
paw_number + = 2
＃现在我们可以用简单的模4确定爪子..
paw_codes = paw_number％4
paw_labels = paw_codes中代码的[paw_code [code]]
返回paw_labels
尽管如此，它经常无法正常工作。完整数据集中的许多狗似乎正在运行，并且爪子撞击不遵循与狗行走时相同的时间顺序。 （或许狗只有严重的髋关节问题......）
幸运的是，我们仍然可以编程方式检测爪子影响是否符合我们预期的空间格局：
def paw_pattern_problems（paw_labels，dx，dy）：
检查标签序列paw_labels是否符合
爪子撞击的预期空间模式。paw_labels应该是一个序列
的字符串：LH，RH，LF，RF对应不同的爪子
＃检查问题...（这可以写成_lot_更干净......）
问题=错误
last = paw_labels [0]
用于paw，dy，dx in zip（paw_labels [1：]，dy，dx）：
＃从左爪到右边，如果last.startswith（'L'）和paw.startswith（'R'），dy应为负
和（dy> 0）：
问题=真
break
＃从右爪到左边，dy应为正
如果last.startswith（'R'）和paw.startswith（'L'）和（dy< 0）：
问题=真
打破
＃从前爪到后爪，如果last.endswith（'F'）和爪子，dx应为负
。 endswith（'H'）和（dx> 0）：
问题=真
打破
＃从后爪到前爪，dx应为正
如果last.endswith（'H'）和paw.endswith（'F'）和（dx< 0）：
problems = True
break
last = paw
返回问题
因此，即使简单的空间分类不能一直运行，我们也可以确定何时它确实有合理的信心。
培训数据集
从正确运行的基于模式的分类中，我们可以建立一个非常大的正确分类爪子的训练数据集（来自32只不同的狗的〜2400爪子撞击！）。
我们现在可以开始看一下平均左前方等等，爪子看起来像。
要做到这一点，我们需要某种爪子度量，这对任何狗来说都是相同的维度。 （在完整的数据集中，有非常大的和非常小的狗！）来自爱尔兰elkhound的爪印将比玩具贵宾犬的爪印更宽，更重。我们需要重新缩放每个爪印，以便a）它们具有相同数量的像素，并且b）压力值是标准化的。为此，我将每个爪印刷重新采样到20x20网格上，并根据爪子撞击的最大，最小和平均压力值重新调整压力值。
def paw_image（paw）：来自scipy.ndimage的
导入map_coordinates
ny，nx = paw。形状
＃修剪爪子周围的任何空白边缘...
mask = paw> 0.01 * paw.max（）
y，x = np.mgrid [：ny，：nx]
ymin，ymax = y [mask] .min（），y [mask] .max（）
xmin，xmax = x [mask] .min（），x [mask] .max（）
＃制作一个20x20网格，将爪子压力值重新取样到
numx， numy = 20,20
xi = np.linspace（xmin，xmax，numx）
yi = np.linspace（ymin，ymax，numy）
xi，yi = np.meshgrid（xi ，yi）
＃将值重新取样到20x20网格上
coords = np.vstack（[yi.flatten（），xi.flatten（）]）
zi = map_coordinates （爪子，坐标）
zi = zi.reshape（（numy，numx））
＃重新调整压力值
zi - = zi.min（）
zi / = zi.max（）
zi - = zi.mean（）＃< - 帮助区分前爪和后爪......
返回zi
在完成所有这些之后，我们终于可以看看平均左前方，后方右侧等爪子的样子。请注意，这是大约30只大小不同的狗的平均值，我们似乎得到了一致的结果！
然而，在我们对这些进行任何分析之前，我们需要减去平均值（平均值）所有狗的所有腿的爪子。）
现在我们可以分析平均值的差异，这有点容易识别：
基于图像的爪子识别
好的...我们终于有了一套模式，我们可以开始尝试匹配爪子了。每个爪子可以被视为400维向量（由 paw_image 函数返回），可以与这四个400维向量进行比较。
不幸的是，如果我们只使用普通监督分类算法（即使用简单的距离找到4种模式中的哪一种最接近特定的爪印），不能始终如一地工作。实际上，它在训练数据集上的效果并不比随机机会好得多。
这是图像识别中的常见问题。由于输入数据的高维度，以及图像的某种模糊性质（即相邻像素具有高协方差），仅仅从模板图像中查看图像的差异并不能很好地衡量它们的形状相似。
Eigenpaws
为了解决这个问题，我们需要建立一套 eigenpaws（就像面部识别中的特征脸一样），并将每个爪印描述为这些特征爪的组合。这与主成分分析相同，基本上提供了一种减少数据维数的方法，因此距离是形状的一个很好的衡量标准。
因为我们有更多训练图像比尺寸（2400 vs 400），没有必要为速度做花式线性代数。我们可以直接使用训练数据集的协方差矩阵：
def make_eigenpaws（paw_data）：
根据paw_data创建一组特征爪。
paw_data是所有观察值的numdimensions矩阵的numdata。
average_paw = paw_data.mean（axis = 0）
paw_data - = average_paw
＃确定数据的协方差矩阵的特征向量
cov = np.cov（paw_data.T）
eigvals，eigvecs = np.linalg.eig（cov ）
＃按特征值递增对特征向量进行排序（最大值为最后）
eig_idx = np.argsort（eigvals）
sorted_eigvecs = eigvecs [：，eig_idx]
sorted_eigvals = eigvals [：，eig_idx]
＃现在选择一个截止数量的特征向量来使用
＃（50似乎运作良好，但它是arbirtrary ...
num_basis_vecs = 50
basis_vecs = sorted_eigvecs [：， - num_basis_vecs：]
返回basis_vecs
这些 basis_vecs 是eigenpaws。
要使用这些，我们只需点（即矩阵乘法）每个爪子图像（作为400维向量，而不是20x20图像）与基础向量。这为我们提供了一个50维向量（每个基础向量一个元素），我们可以用它来对图像进行分类。我们不是将20x20图像与每个模板爪子的20x20图像进行比较，而是将50维变换图像与每个50维变换模板爪进行比较。这对于每个脚趾的确切位置等的微小变化都不太敏感，并且基本上将问题的维度降低到相关尺寸。
基于Eigenpaw的爪子分类
现在我们可以简单地使用50维向量和每条腿的模板向量之间的距离来分类哪个爪子是：
codebook = np.load（'codebook.npy'）＃每个爪子的模板向量
average_paw = np.load（'average_paw.npy'）
basis_stds = np.load（'basis_stds.npy'）＃需要白化数据集...
basis_vecs = np.load（'basis_vecs .npy'）
paw_code = {0：'LF'，1：'RH'，2：'RF'，3：'LH'}
def classify（爪子）：
paw = paw.flatten（）
paw - = average_paw
得分= paw.dot（basis_vecs）/ basis_stds
diff = codebook - 得分
diff * = diff
diff = np.sqrt（diff.sum（axis = 1））
返回paw_code [diff.argmin（）]
以下是一些结果：
剩余问题
仍然存在一些问题，特别是对于太小而无法制作明确的pawprint的狗...（它最适合大型犬，因为在传感器的分辨率下，脚趾更清晰分离。）此外，该系统无法识别部分爪印，而它们可以使用基于梯形图案的系统。
然而，由于特征爪分析固有地使用了距离度量，我们可以将两种方式分类为爪子，并在特征爪分析时回归到基于梯形图案的系统。与码本的最小距离超过某个阈值。不过我还没有实现这个。
Phew ......那很长！我的帽子是因为有这样一个有趣的问题而离开伊沃！
In my previous question I got an excellent answer that helped me detect where a paw hit a pressure plate, but now I'm struggling to link these results to their corresponding paws:
I manually annotated the paws (RF=right front, RH= right hind, LF=left front, LH=left hind).
As you can see there's clearly a repeating pattern and it comes back in almost every measurement. Here's a link to a presentation of 6 trials that were manually annotated.
My initial thought was to use heuristics to do the sorting, like:
- There's a ~60-40% ratio in weight bearing between the front and hind paws;
- The hind paws are generally smaller in surface;
- The paws are (often) spatially divided in left and right.
However, I’m a bit skeptical about my heuristics, as they would fail on me as soon as I encounter a variation I hadn’t thought off. They also won’t be able to cope with measurements from lame dogs, whom probably have rules of their own.
Furthermore, the annotation suggested by Joe sometimes get's messed up and doesn't take into account what the paw actually looks like.
Based on the answers I received on my question about peak detection within the paw, I’m hoping there are more advanced solutions to sort the paws. Especially because the pressure distribution and the progression thereof are different for each separate paw, almost like a fingerprint. I hope there's a method that can use this to cluster my paws, rather than just sorting them in order of occurrence.
So I'm looking for a better way to sort the results with their corresponding paw.
For anyone up to the challenge, I have pickled a dictionary with all the sliced arrays that contain the pressure data of each paw (bundled by measurement) and the slice that describes their location (location on the plate and in time).
To clarfiy: walk_sliced_data is a dictionary that contains ['ser_3', 'ser_2', 'sel_1', 'sel_2', 'ser_1', 'sel_3'], which are the names of the measurements. Each measurement contains another dictionary, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] (example from 'sel_1') which represent the impacts that were extracted.
Also note that 'false' impacts, such as where the paw is partially measured (in space or time) can be ignored. They are only useful because they can help recognizing a pattern, but won't be analyzed.
And for anyone interested, I’m keeping a blog with all the updates regarding the project!
Alright! I've finally managed to get something working consistently! This problem pulled me in for several days... Fun stuff! Sorry for the length of this answer, but I need to elaborate a bit on some things... (Though I may set a record for the longest non-spam stackoverflow answer ever!)
As a side note, I'm using the full dataset that Ivo provided a link to in his original question. It's a series of rar files (one-per-dog) each containing several different experiment runs stored as ascii arrays. Rather than try to copy-paste stand-alone code examples into this question, here's a bitbucket mercurial repository with full, stand-alone code. You can clone it with
hg clone https://joferkington@bitbucket.org/joferkington/paw-analysis
Overview
There are essentially two ways to approach the problem, as you noted in your question. I'm actually going to use both in different ways.
- Use the (temporal and spatial) order of the paw impacts to determine which paw is which.
- Try to identify the "pawprint" based purely on its shape.
Basically, the first method works with the dog's paws follow the trapezoidal-like pattern shown in Ivo's question above, but fails whenever the paws don't follow that pattern. It's fairly easy to programatically detect when it doesn't work.
Therefore, we can use the measurements where it did work to build up a training dataset (of ~2000 paw impacts from ~30 different dogs) to recognize which paw is which, and the problem reduces to a supervised classification (With some additional wrinkles... Image recognition is a bit harder than a "normal" supervised classification problem).
Pattern Analysis
To elaborate on the first method, when a dog is walking (not running!) normally (which some of these dogs may not be), we expect paws to impact in the order of: Front Left, Hind Right, Front Right, Hind Left, Front Left, etc. The pattern may start with either the front left or front right paw.
If this were always the case, we could simply sort the impacts by initial contact time and use a modulo 4 to group them by paw.
However, even when everything is "normal", this doesn't work. This is due to the trapezoid-like shape of the pattern. A hind paw spatially falls behind the previous front paw.
Therefore, the hind paw impact after the initial front paw impact often falls off the sensor plate, and isn't recorded. Similarly, the last paw impact is often not the next paw in the sequence, as the paw impact before it occured off the sensor plate and wasn't recorded.
Nonetheless, we can use the shape of the paw impact pattern to determine when this has happened, and whether we've started with a left or right front paw. (I'm actually ignoring problems with the last impact here. It's not too hard to add it, though.)
def group_paws(data_slices, time):
# Sort slices by initial contact time
data_slices.sort(key=lambda s: s[-1].start)
# Get the centroid for each paw impact...
paw_coords = []
for x,y,z in data_slices:
paw_coords.append([(item.stop + item.start) / 2.0 for item in (x,y)])
paw_coords = np.array(paw_coords)
# Make a vector between each sucessive impact...
dx, dy = np.diff(paw_coords, axis=0).T
#-- Group paws -------------------------------------------
paw_code = {0:'LF', 1:'RH', 2:'RF', 3:'LH'}
paw_number = np.arange(len(paw_coords))
# Did we miss the hind paw impact after the first
# front paw impact? If so, first dx will be positive...
if dx[0] > 0:
paw_number[1:] += 1
# Are we starting with the left or right front paw...
# We assume we're starting with the left, and check dy[0].
# If dy[0] > 0 (i.e. the next paw impacts to the left), then
# it's actually the right front paw, instead of the left.
if dy[0] > 0: # Right front paw impact...
paw_number += 2
# Now we can determine the paw with a simple modulo 4..
paw_codes = paw_number % 4
paw_labels = [paw_code[code] for code in paw_codes]
return paw_labels
In spite of all of this, it frequently doesn't work correctly. Many of the dogs in the full dataset appear to be running, and the paw impacts don't follow the same temporal order as when the dog is walking. (Or perhaps the dog just has severe hip problems...)
Fortunately, we can still programatically detect whether or not the paw impacts follow our expected spatial pattern:
def paw_pattern_problems(paw_labels, dx, dy):
"""Check whether or not the label sequence "paw_labels" conforms to our
expected spatial pattern of paw impacts. "paw_labels" should be a sequence
of the strings: "LH", "RH", "LF", "RF" corresponding to the different paws"""
# Check for problems... (This could be written a _lot_ more cleanly...)
problems = False
last = paw_labels[0]
for paw, dy, dx in zip(paw_labels[1:], dy, dx):
# Going from a left paw to a right, dy should be negative
if last.startswith('L') and paw.startswith('R') and (dy > 0):
problems = True
break
# Going from a right paw to a left, dy should be positive
if last.startswith('R') and paw.startswith('L') and (dy < 0):
problems = True
break
# Going from a front paw to a hind paw, dx should be negative
if last.endswith('F') and paw.endswith('H') and (dx > 0):
problems = True
break
# Going from a hind paw to a front paw, dx should be positive
if last.endswith('H') and paw.endswith('F') and (dx < 0):
problems = True
break
last = paw
return problems
Therefore, even though the simple spatial classification doesn't work all of the time, we can determine when it does work with reasonable confidence.
Training Dataset
From the pattern-based classifications where it worked correctly, we can build up a very large training dataset of correctly classified paws (~2400 paw impacts from 32 different dogs!).
We can now start to look at what an "average" front left, etc, paw looks like.
To do this, we need some sort of "paw metric" that is the same dimensionality for any dog. (In the full dataset, there are both very large and very small dogs!) A paw print from an Irish elkhound will be both much wider and much "heavier" than a paw print from a toy poodle. We need to rescale each paw print so that a) they have the same number of pixels, and b) the pressure values are standardized. To do this, I resampled each paw print onto a 20x20 grid and rescaled the pressure values based on the maximum, mininum, and mean pressure value for the paw impact.
def paw_image(paw):
from scipy.ndimage import map_coordinates
ny, nx = paw.shape
# Trim off any "blank" edges around the paw...
mask = paw > 0.01 * paw.max()
y, x = np.mgrid[:ny, :nx]
ymin, ymax = y[mask].min(), y[mask].max()
xmin, xmax = x[mask].min(), x[mask].max()
# Make a 20x20 grid to resample the paw pressure values onto
numx, numy = 20, 20
xi = np.linspace(xmin, xmax, numx)
yi = np.linspace(ymin, ymax, numy)
xi, yi = np.meshgrid(xi, yi)
# Resample the values onto the 20x20 grid
coords = np.vstack([yi.flatten(), xi.flatten()])
zi = map_coordinates(paw, coords)
zi = zi.reshape((numy, numx))
# Rescale the pressure values
zi -= zi.min()
zi /= zi.max()
zi -= zi.mean() #<- Helps distinguish front from hind paws...
return zi
After all of this, we can finally take a look at what an average left front, hind right, etc paw looks like. Note that this is averaged across >30 dogs of greatly different sizes, and we seem to be getting consistent results!
However, before we do any analysis on these, we need to subtract the mean (the average paw for all legs of all dogs).
Now we can analyize the differences from the mean, which are a bit easier to recognize:
Image-based Paw Recognition
Ok... We finally have a set of patterns that we can begin to try to match the paws against. Each paw can be treated as a 400-dimensional vector (returned by the paw_image function) that can be compared to these four 400-dimensional vectors.
Unfortunately, if we just use a "normal" supervised classification algorithm (i.e. find which of the 4 patterns is closest to a particular paw print using a simple distance), it doesn't work consistently. In fact, it doesn't do much better than random chance on the training dataset.
This is a common problem in image recognition. Due to the high dimensionality of the input data, and the somewhat "fuzzy" nature of images (i.e. adjacent pixels have a high covariance), simply looking at the difference of an image from a template image does not give a very good measure of the similarity of their shapes.
Eigenpaws
To get around this we need to build a set of "eigenpaws" (just like "eigenfaces" in facial recognition), and describe each paw print as a combination of these eigenpaws. This is identical to principal components analysis, and basically provides a way to reduce the dimensionality of our data, so that distance is a good measure of shape.
Because we have more training images than dimensions (2400 vs 400), there's no need to do "fancy" linear algebra for speed. We can work directly with the covariance matrix of the training data set:
def make_eigenpaws(paw_data):
"""Creates a set of eigenpaws based on paw_data.
paw_data is a numdata by numdimensions matrix of all of the observations."""
average_paw = paw_data.mean(axis=0)
paw_data -= average_paw
# Determine the eigenvectors of the covariance matrix of the data
cov = np.cov(paw_data.T)
eigvals, eigvecs = np.linalg.eig(cov)
# Sort the eigenvectors by ascending eigenvalue (largest is last)
eig_idx = np.argsort(eigvals)
sorted_eigvecs = eigvecs[:,eig_idx]
sorted_eigvals = eigvals[:,eig_idx]
# Now choose a cutoff number of eigenvectors to use
# (50 seems to work well, but it's arbirtrary...
num_basis_vecs = 50
basis_vecs = sorted_eigvecs[:,-num_basis_vecs:]
return basis_vecs
These basis_vecs are the "eigenpaws".
To use these, we simply dot (i.e. matrix multiplication) each paw image (as a 400-dimensional vector, rather than a 20x20 image) with the basis vectors. This gives us a 50-dimensional vector (one element per basis vector) that we can use to classify the image. Instead of comparing a 20x20 image to the 20x20 image of each "template" paw, we compare the 50-dimensional, transformed image to each 50-dimensional transformed template paw. This is much less sensitive to small variations in exactly how each toe is positioned, etc, and basically reduces the dimensionality of the problem to just the relevant dimensions.
Eigenpaw-based Paw Classification
Now we can simply use the distance between the 50-dimensional vectors and the "template" vectors for each leg to classify which paw is which:
codebook = np.load('codebook.npy') # Template vectors for each paw
average_paw = np.load('average_paw.npy')
basis_stds = np.load('basis_stds.npy') # Needed to "whiten" the dataset...
basis_vecs = np.load('basis_vecs.npy')
paw_code = {0:'LF', 1:'RH', 2:'RF', 3:'LH'}
def classify(paw):
paw = paw.flatten()
paw -= average_paw
scores = paw.dot(basis_vecs) / basis_stds
diff = codebook - scores
diff *= diff
diff = np.sqrt(diff.sum(axis=1))
return paw_code[diff.argmin()]
Here are some of the results:
Remaining Problems
There are still some problems, particularly with dogs too small to make a clear pawprint... (It works best with large dogs, as the toes are more clearly seperated at the sensor's resolution.) Also, partial pawprints aren't recognized with this system, while they can be with the trapezoidal-pattern-based system.
However, because the eigenpaw analysis inherently uses a distance metric, we can classify the paws both ways, and fall back to the trapezoidal-pattern-based system when the eigenpaw analysis's smallest distance from the "codebook" is over some threshold. I haven't implemented this yet, though.
Phew... That was long! My hat is off to Ivo for having such a fun question!
这篇关于如何整理我的爪子？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
