验证数据可以成为tensorflow.keras 2.0中的生成器吗? [英] Could validation data be a generator in tensorflow.keras 2.0?

问题描述

在 tensorflow.keras

validation_data可以是:Numpy数组或张量的元组(x_val，y_val) Numpy数组的元组(x_val，y_val，val_sample_weights) 数据集对于前两种情况，必须提供batch_size.对于最后一种情况，可以提供validation_steps.

validation_data could be: tuple (x_val, y_val) of Numpy arrays or tensors tuple (x_val, y_val, val_sample_weights) of Numpy arrays dataset For the first two cases, batch_size must be provided. For the last case, validation_steps could be provided.

它没有提到生成器是否可以充当validation_data.所以我想知道validation_data是否可以作为数据生成器?例如以下代码:

It does not mention if generator could act as validation_data. So I want to know if validation_data could be a datagenerator? like the following codes:

net.fit_generator(train_it.generator(), epoch_iterations * batch_size, nb_epoch=nb_epoch, verbose=1,
                  validation_data=val_it.generator(), nb_val_samples=3,
                  callbacks=[checker, tb, stopper, saver])

更新: 在 keras 的正式文档中，内容相同，但又添加了另一句话:

Update: In the official documents of keras, the same contents, but another sentense is added:

• 数据集或数据集迭代器

• dataset or a dataset iterator

考虑

数据集对于前两种情况，必须提供batch_size.对于最后一种情况，可以提供validation_steps.

dataset For the first two cases, batch_size must be provided. For the last case, validation_steps could be provided.

我认为应该有3种情况. Keras的文件是正确的.因此，我将在tensorflow.keras中发布一个问题以更新文档.

I think there should be 3 cases. Keras' documents are correct. So I will post an issue in tensorflow.keras to update the documents.

推荐答案

是的，很奇怪，它不在文档中，但其工作原理与x参数完全相同，您也可以使用keras.Sequence或generator.在我的项目中，我经常使用keras.Sequence，其作用类似于生成器

Yes it can, that's strange that it is not in the doc but is it working exactly like the x argument, you can also use a keras.Sequence or a generator. In my project I often use keras.Sequence that acts like a generator

显示其正常工作的最小工作示例:

Minimum working example that shows that it works :

import numpy as np
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, Flatten

def generator(batch_size): # Create empty arrays to contain batch of features and labels
    batch_features = np.zeros((batch_size, 1000))
    batch_labels = np.zeros((batch_size,1))
    while True:
        for i in range(batch_size):
            yield batch_features, batch_labels

model = Sequential()
model.add(Dense(125, input_shape=(1000,), activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

train_generator = generator(64)
validation_generator = generator(64)

model.fit(train_generator, validation_data=validation_generator, validation_steps=100, epochs=100, steps_per_epoch=100)

100/100 [==============================]-1s 13ms/step-损耗:0.6689-精度: 1.0000-val_loss:0.6448-val_accuracy:1.0000 时代2/100 100/100 [=============================]-0s 4ms/step-损耗:0.6223-精度:1.0000-val_loss :0.6000-val_accuracy:1.0000 时代3/100 100/100 [=============================]-0s 4ms/step-损耗:0.5792-精度:1.0000-val_loss :0.5586-val_accuracy:1.0000 时代4/100 100/100 [=============================]-0s 4ms/step-损耗:0.5393-精度:1.0000-val_loss :0.5203​​-val_accuracy:1.0000

100/100 [==============================] - 1s 13ms/step - loss: 0.6689 - accuracy: 1.0000 - val_loss: 0.6448 - val_accuracy: 1.0000 Epoch 2/100 100/100 [==============================] - 0s 4ms/step - loss: 0.6223 - accuracy: 1.0000 - val_loss: 0.6000 - val_accuracy: 1.0000 Epoch 3/100 100/100 [==============================] - 0s 4ms/step - loss: 0.5792 - accuracy: 1.0000 - val_loss: 0.5586 - val_accuracy: 1.0000 Epoch 4/100 100/100 [==============================] - 0s 4ms/step - loss: 0.5393 - accuracy: 1.0000 - val_loss: 0.5203 - val_accuracy: 1.0000

这篇关于验证数据可以成为tensorflow.keras 2.0中的生成器吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！