在预测过程中，数据规范化在 keras 中是如何工作的? [英] How does data normalization work in keras during prediction?

问题描述

我看到 imageDataGenerator 允许我指定不同风格的数据规范化，例如featurewise_center、samplewise_center 等

I see that the imageDataGenerator allows me to specify different styles of data normalization, e.g. featurewise_center, samplewise_center, etc.

我从示例中看到，如果我指定这些选项之一，那么我需要调用生成器上的 fit 方法，以便允许生成器计算统计数据，例如生成器上的平均图像.

I see from the examples that if I specify one of these options, then I need to call the fit method on the generator in order to allow the generator to compute statistics like the mean image on the generator.

(X_train, y_train), (X_test, y_test) = cifar10.load_data()
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)

datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True)

# compute quantities required for featurewise normalization
# (std, mean, and principal components if ZCA whitening is applied)
datagen.fit(X_train)

# fits the model on batches with real-time data augmentation:
model.fit_generator(datagen.flow(X_train, Y_train, batch_size=32),
                samples_per_epoch=len(X_train), nb_epoch=nb_epoch)

我的问题是，如果我在训练期间指定了数据归一化，预测如何工作?我看不出在框架中我什至会传递训练集均值/标准偏差的知识来预测以允许我自己规范化我的测试数据，但我也没有在训练代码中看到这些信息的位置存储.

My question is, how does prediction work if I have specified data normalization during training? I can't see how in the framework I would even pass knowledge of the training set mean/std deviation along to predict to allow me to normalize my test data myself, but I also don't see in the training code where this information is stored.

归一化所需的图像统计信息是否存储在模型中，以便在预测过程中使用?

Are the image statistics needed for normalization stored in the model so that they can be used during prediction?

推荐答案

是的 - 这是 Keras.ImageDataGenerator 的一个非常大的缺点，您无法自己提供标准化统计数据.但是 - 有一个简单的方法可以解决这个问题.

Yes - this is a really huge downside of Keras.ImageDataGenerator that you couldn't provide the standarization statistics on your own. But - there is an easy method on how to overcome this issue.

假设您有一个函数 normalize(x) 对图像进行标准化batch(请记住，生成器提供的不是简单的图像，而是一组图像 -batch with shape (nr_of_examples_in_batch, image_dims ..) 您可以使用以下方法制作自己的标准化生成器:

Assuming that you have a function normalize(x) which is normalizing an image batch (remember that generator is not providing a simple image but an array of images - a batch with shape (nr_of_examples_in_batch, image_dims ..) you could make your own generator with normalization by using:

def gen_with_norm(gen, normalize):
    for x, y in gen:
        yield normalize(x), y

然后你可以简单地使用 gen_with_norm(datagen.flow, normalize) 而不是 datagen.flow.

Then you might simply use gen_with_norm(datagen.flow, normalize) instead of datagen.flow.

此外 - 您可以恢复由 fit 方法计算的 mean 和 std，方法是从 datagen(例如 datagen.mean 和 datagen.std).

Moreover - you might recover the mean and std computed by a fit method by getting it from appropriate fields in datagen (e.g. datagen.mean and datagen.std).

这篇关于在预测过程中，数据规范化在 keras 中是如何工作的?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！