用于LSTM-RNN训练的填充时间序列子序列 [英] Padding time-series subsequences for LSTM-RNN training

问题描述

我有一个时间序列数据集，可用作LSTM-RNN的动作预期输入.该时间序列包含30 fps的5秒时间(即150个数据点)，该数据代表了面部特征的位置/运动.

I have a dataset of time series that I use as input to an LSTM-RNN for action anticipation. The time series comprises a time of 5 seconds at 30 fps (i.e. 150 data points), and the data represents the position/movement of facial features.

我从我的数据集中采样了其他长度较小的子序列，以便在数据集中增加冗余并减少过度拟合.在这种情况下，我知道了子序列的开始和结束帧.

I sample additional sub-sequences of smaller length from my dataset in order to add redundancy in the dataset and reduce overfitting. In this case I know the starting and ending frame of the sub-sequences.

为了批量训练模型，所有时间序列都必须具有相同的长度，并且根据文献中的许多论文，填充不应影响网络的性能.

In order to train the model in batches, all time series need to have the same length, and according to many papers in the literature padding should not affect the performance of the network.

示例:

原始顺序:

 1 2 3 4 5 6 7 8 9 10

子序列:

4 5 6 7
8 9 10
2 3 4 5 6

考虑到我的网络正在尝试预期一个动作(这意味着只要P(action)>阈值从t = 0到T = tmax，它就会预测该动作)填充会去哪里有关系吗?

considering that my network is trying to anticipate an action (meaning that as soon as P(action) > threshold as it goes from t = 0 to T = tmax, it will predict that action) will it matter where the padding goes?

选项1 :用零代替原始值

0 0 0 4 5 6 7 0 0 0
0 0 0 0 0 0 0 8 9 10
0 2 3 4 5 6 0 0 0 0

选项2 :末尾全为零

4 5 6 7 0 0 0 0 0 0 
8 9 10 0 0 0 0 0 0 0
2 3 4 5 0 0 0 0 0 0

此外，某些时间序列缺少一些帧，但尚不知道它们是-意味着如果我们只有60帧，我们不知道它们是否取自0到2秒，从1到3秒，等等.甚至在提取子序列之前都需要填充这些内容.在这种情况下，填充的最佳做法是什么?

Moreover, some of the time series are missing a number of frames, but it is not known which ones they are - meaning that if we only have 60 frames, we don't know whether they are taken from 0 to 2 seconds, from 1 to 3s, etc. These need to be padded before the subsequences are even taken. What is the best practice for padding in this case?

谢谢.

推荐答案

LSTM和RNN最强大的属性通常是，它们的参数在时间范围内共享(参数 recur 在时间范围内) )，但参数共享取决于以下假设:相同的参数可以用于不同的时间步长，即上一个时间步长和下一个时间步长之间的关系不取决于t，如此处在第388页第二段..

The most powerful attribute of LSTMs and RNNs in general is that their parameters are shared along the time frames(Parameters recur over time frames) but the parameter sharing relies upon the assumption that the same parameters can be used for different time steps i.e. the relationship between the previous time step and the next time step does not depend on t as explained here in page 388, 2nd paragraph.

简而言之，从理论上讲在末尾填充零不会改变模型的准确性.我使用副词 theoretically 是因为LSTM在每个时间步骤上的决定都取决于它的单元状态以及其他因素，而这种单元状态只是对过去框架的简短总结.据我了解，在您的情况下，过去的框架可能会丢失.我认为您在这里需要权衡一些.

In short, padding zeros at the end, theoretically should not change the accuracy of the model. I used the adverb theoretically because at each time step LSTM's decision depends on its cell state among other factors and this cell state is kind of a short summary of the past frames. As far as I understood, that past frames may be missing in your case. I think what you have here is a little trade-off.

我宁愿最后填充零，因为它与RNN的基本假设并不完全冲突，并且更易于实现和跟踪.

I would rather pad zeros at the end because it doesn't completely conflict with the underlying assumption of RNNs and it's more convenient to implement and keep track of.

在实现方面，我知道张量流会在给定每个样本的序列和实际序列大小后计算损失函数(例如对于4 5 6 7 0 0 0 0 0 0您还需要给它实际的大小为4)，假设您正在实现选项2.不过，我不知道选项1是否有实现.

On the implementation side, I know tensorflow calculates the loss function once you give it the sequences and the actual sequence size of each sample(e.g. for 4 5 6 7 0 0 0 0 0 0 you also need to give it the actual size which is 4 here) assuming you're implementing the option 2. I don't know whether there is an implementation for option 1, though.

这篇关于用于LSTM-RNN训练的填充时间序列子序列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！