我在哪里调用 Keras 中的 BatchNormalization 函数? [英] Where do I call the BatchNormalization function in Keras?

问题描述

如果我想在 Keras 中使用 BatchNormalization 函数，那么我需要在开始时只调用一次吗?

If I want to use the BatchNormalization function in Keras, then do I need to call it once only at the beginning?

我为此阅读了此文档:http://keras.io/layers/normalization/

我不知道我应该在哪里称呼它.以下是我尝试使用它的代码:

I don't see where I'm supposed to call it. Below is my code attempting to use it:

model = Sequential()
keras.layers.normalization.BatchNormalization(epsilon=1e-06, mode=0, momentum=0.9, weights=None)
model.add(Dense(64, input_dim=14, init='uniform'))
model.add(Activation('tanh'))
model.add(Dropout(0.5))
model.add(Dense(64, init='uniform'))
model.add(Activation('tanh'))
model.add(Dropout(0.5))
model.add(Dense(2, init='uniform'))
model.add(Activation('softmax'))

sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='binary_crossentropy', optimizer=sgd)
model.fit(X_train, y_train, nb_epoch=20, batch_size=16, show_accuracy=True, validation_split=0.2, verbose = 2)

我问是因为如果我用第二行(包括批量标准化)运行代码，如果我在没有第二行的情况下运行代码，我会得到类似的输出.所以要么我没有在正确的地方调用这个函数，要么我想它没有太大区别.

I ask because if I run the code with the second line including the batch normalization and if I run the code without the second line I get similar outputs. So either I'm not calling the function in the right place, or I guess it doesn't make that much of a difference.

推荐答案

只是为了更详细地回答这个问题，正如 Pavel 所说，Batch Normalization 只是另一个层，因此您可以使用它来创建您的所需的网络架构.

Just to answer this question in a little more detail, and as Pavel said, Batch Normalization is just another layer, so you can use it as such to create your desired network architecture.

一般用例是在网络中的线性层和非线性层之间使用 BN，因为它将输入归一化为激活函数，这样您就可以在激活函数的线性部分居中(例如乙状结肠).在这里有一个小讨论

The general use case is to use BN between the linear and non-linear layers in your network, because it normalizes the input to your activation function, so that you're centered in the linear section of the activation function (such as Sigmoid). There's a small discussion of it here

在你上面的例子中，这可能看起来像:

In your case above, this might look like:

# import BatchNormalization
from keras.layers.normalization import BatchNormalization

# instantiate model
model = Sequential()

# we can think of this chunk as the input layer
model.add(Dense(64, input_dim=14, init='uniform'))
model.add(BatchNormalization())
model.add(Activation('tanh'))
model.add(Dropout(0.5))

# we can think of this chunk as the hidden layer    
model.add(Dense(64, init='uniform'))
model.add(BatchNormalization())
model.add(Activation('tanh'))
model.add(Dropout(0.5))

# we can think of this chunk as the output layer
model.add(Dense(2, init='uniform'))
model.add(BatchNormalization())
model.add(Activation('softmax'))

# setting up the optimization of our weights 
sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='binary_crossentropy', optimizer=sgd)

# running the fitting
model.fit(X_train, y_train, nb_epoch=20, batch_size=16, show_accuracy=True, validation_split=0.2, verbose = 2)

<小时>

希望这能进一步说明问题.

---

Hope this clarifies things a bit more.

这篇关于我在哪里调用 Keras 中的 BatchNormalization 函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！