预测期间，数据标准化如何在keras中起作用? [英] How does data normalization work in keras during prediction?

问题描述

我看到imageDataGenerator允许我指定不同样式的数据规范化，例如featurewise_center，samplewise_center等.

I see that the imageDataGenerator allows me to specify different styles of data normalization, e.g. featurewise_center, samplewise_center, etc.

从示例中可以看到，如果我指定了这些选项之一，则需要在生成器上调用fit方法，以允许生成器计算统计信息，例如生成器上的均值图像.

I see from the examples that if I specify one of these options, then I need to call the fit method on the generator in order to allow the generator to compute statistics like the mean image on the generator.

(X_train, y_train), (X_test, y_test) = cifar10.load_data()
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)

datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True)

# compute quantities required for featurewise normalization
# (std, mean, and principal components if ZCA whitening is applied)
datagen.fit(X_train)

# fits the model on batches with real-time data augmentation:
model.fit_generator(datagen.flow(X_train, Y_train, batch_size=32),
                samples_per_epoch=len(X_train), nb_epoch=nb_epoch)

我的问题是，如果我在训练期间指定了数据归一化，预测将如何工作?我看不到如何在框架中传递训练集均值/标准差的知识来预测如何使我自己对测试数据进行规范化，但是我也看不到训练代码中该信息在哪里存储.

My question is, how does prediction work if I have specified data normalization during training? I can't see how in the framework I would even pass knowledge of the training set mean/std deviation along to predict to allow me to normalize my test data myself, but I also don't see in the training code where this information is stored.

模型中是否存储了归一化所需的图像统计信息，以便可以在预测期间使用它们?

Are the image statistics needed for normalization stored in the model so that they can be used during prediction?

推荐答案

是的-这是Keras.ImageDataGenerator的巨大缺点，您无法自行提供标准化统计信息.但是-有一个简单的方法可以解决这个问题.

Yes - this is a really huge downside of Keras.ImageDataGenerator that you couldn't provide the standarization statistics on your own. But - there is an easy method on how to overcome this issue.

假设您具有函数normalize(x)，正在对图像批处理进行规范化(请记住，生成器不是在提供简单的图像，而是提供一组图像-批处理形状为(nr_of_examples_in_batch, image_dims ..)的，您可以使用以下方法制作具有规范化的自己的生成器:

Assuming that you have a function normalize(x) which is normalizing an image batch (remember that generator is not providing a simple image but an array of images - a batch with shape (nr_of_examples_in_batch, image_dims ..) you could make your own generator with normalization by using:

def gen_with_norm(gen, normalize):
    for x, y in gen:
        yield normalize(x), y

然后，您可以简单地使用gen_with_norm(datagen.flow, normalize)而不是datagen.flow.

Then you might simply use gen_with_norm(datagen.flow, normalize) instead of datagen.flow.

此外-您可以通过从数据源的适当字段(例如datagen.mean和datagen.std)中获取通过fit方法计算出的mean和std.

Moreover - you might recover the mean and std computed by a fit method by getting it from appropriate fields in datagen (e.g. datagen.mean and datagen.std).

这篇关于预测期间，数据标准化如何在keras中起作用?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！