如何将fit_generator与分割成批的顺序数据一起使用? [英] How to use fit_generator with sequential data that is split into batches?
问题描述
我正在尝试为我的Keras lstm模型编写一个生成器.配合fit_generator方法使用. 我的第一个问题是发电机应该返回什么?一批吗顺序? Keras文档中的示例为每个数据条目返回x,y,但是如果我的数据是连续的怎么办?我想将其分成几批吗?
I am trying to write a generator for my Keras lstm model. To use it with fit_generator method. My first question is what should my generator return? A batch? A sequence? Example in Keras documentation returns x,y for each data entry, but what if my data is sequential? And I want to split it into batches?
这是为给定输入创建批处理的python方法
Here is the python method that creates a batch for a given input
def get_batch(data, batch_num, batch_size, seq_length):
i_start = batch_num*batch_size;
batch_sequences = []
batch_labels = []
batch_chunk = data.iloc[i_start:(i_start+batch_size)+seq_length].values
for i in range(0, batch_size):
sequence = batch_chunk[(i_start+i):(i_start+i)+seq_length];
label = data.iloc[(i_start+i)+seq_length].values;
batch_labels.append(label)
batch_sequences.append(sequence)
return np.array(batch_sequences), np.array(batch_labels);
此方法的输出用于这样的输入:
The output of this method for an input like this:
get_batch(data, batch_num=0, batch_size=2, seq_length=3):
将是:
x = [
[[1],[2],[3]],
[[2],[3],[4]]
]
这就是我对模型的想象:
Here is how I imagine my model:
model = Sequential()
model.add(LSTM(256, return_sequences=True, input_shape=(seq_length, num_features)))
model.add(Dropout(0.2))
model.add(LSTM(256))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
我的问题是如何将我的方法转换为生成器?
My question is how can I translate my method into a generator?
推荐答案
以下是使用 Sequence 就像Keras中的生成器一样:
Here is a solution that uses Sequence which acts like a generator in Keras:
class MySequence(Sequence):
def __init__(self, num_batches):
self.num_batches = num_batches
def __len__(self):
return self.num_batches # the length is the number of batches
def __getitem__(self, batch_id):
return get_batch(data, batch_id, self.batch_size, seq_length)
我认为这更干净,并且不会修改您的原始功能.现在,您将MySequence
的实例传递给model.fit_generator
.
I think this is cleaner and doesn't modify your original function. Now you pass an instance of MySequence
to model.fit_generator
.
这篇关于如何将fit_generator与分割成批的顺序数据一起使用?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!