Keras预测给出的误差与评估值不同，损失与指标不同 [英] Keras predict gives different error than evaluate, loss different from metrics

问题描述

我有以下问题:

我在Keras中有一个自动编码器，并对其进行了一些培训.培训概述显示验证MAE为0.0422，MSE为0.0024. 但是，如果我随后调用network.predict并手动计算验证错误，则会得到0.035和0.0024.

I have an autoencoder in Keras, and train it for a few epochs. The training overview shows a validation MAE of 0.0422 and an MSE of 0.0024. However, if I then call network.predict and manually calculate the validation errors, I get 0.035 and 0.0024.

有人会认为我对MAE的手动计算完全不正确，但是很奇怪的是，如果我使用一个身份模型(简单地输出您输入的内容)并将其用于评估预测值，则相同的误差值为返回我的手动计算结果.代码如下:

One would assume that my manual calculation of the MAE is simply incorrect, but the weird thing is that if I use an identity model (simply outputs what you input) and use that to evaluate the predicted values, the same error value is returned as for my manual calculation. The code looks as follows:

input = Input(shape=(X_train.shape[1], ))
encoded = Dense(50, activation='relu', activity_regularizer=regularizers.l1(10e-5))(input)
encoded = Dense(50, activation='relu', activity_regularizer=regularizers.l1(10e-5))(encoded)
encoded = Dense(50, activation='relu', activity_regularizer=regularizers.l1(10e-5))(encoded)
decoded = Dense(50, activation='relu', activity_regularizer=regularizers.l1(10e-5))(encoded)
decoded = Dense(50, activation='relu', activity_regularizer=regularizers.l1(10e-5))(decoded)
decoded = Dense(X_train.shape[1], activation='sigmoid')(decoded)
network = Model(input, decoded)

# sgd = SGD(lr=8, decay=1e-6)
# network.compile(loss='mean_squared_error', optimizer='adam')
network.compile(loss='mean_absolute_error', optimizer='adam', metrics=['mse'])

# Fitting the data
network.fit(X_train, X_train, epochs=2, batch_size=1, shuffle=True, validation_data=(X_valid, X_valid),
            callbacks=[EarlyStopping(monitor='val_loss', min_delta=0.00001, patience=20, verbose=0, mode='auto')])


# Results
recon_valid = network.predict(X_valid, batch_size=1)
score2 = network.evaluate(X_valid, X_valid, batch_size=1, verbose=0)
print('Network evaluate result: mae={}, mse={}'.format(*score2))

x = Input((X_train.shape[1],))
m = Model(x, x)
m.compile(loss='mean_absolute_error', optimizer='adam', metrics=['mse'])
score1 = m.evaluate(recon_valid, X_valid, batch_size=1, verbose=0)
print('Identity evaluate result: mae={}, mse={}'.format(*score1))

errors_test = np.absolute(X_valid - recon_valid)
print("Manual MAE: {}".format(np.average(errors_test)))
errors_test = np.square(X_valid - recon_valid)
print("Manual MSE: {}".format(np.average(errors_test)))

输出以下内容:

Train on 282 samples, validate on 94 samples
Epoch 1/2
2018-04-18 17:24:01.464947: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
282/282 [==============================] - 0s - loss: 0.0861 - mean_squared_error: 0.0187 - val_loss: 0.0451 - val_mean_squared_error: 0.0025
Epoch 2/2
282/282 [==============================] - 0s - loss: 0.0440 - mean_squared_error: 0.0025 - val_loss: 0.0422 - val_mean_squared_error: 0.0024
Network evaluate result: mae=0.04216482736011769, mse=0.0024067993242382767
Identity evaluate result: mae=0.03506102238563781, mse=0.0024067993242382767
Manual MAE: 0.03506102412939072
Manual MSE: 0.002406799467280507

我知道我的手动计算是正确的，因为身份模型(m)返回相同的值. MAE值差异的唯一可能解释是，network.evaluate(X_valid，X_valid)是否以某种方式使用了与network.predict(X_valid)返回的值不同的值，但是MSE也将有所不同.

I know that my manual calculation is correct, since the identity model (m) returns the same value. The only possible explanation for the difference in MAE values would then be if network.evaluate(X_valid, X_valid) somehow uses different values than those returned by network.predict(X_valid), but then the MSE would also be different.

这让我完全困惑，认为Keras MAE计算中可能存在错误.有没有人曾经遇到过这个问题，或者对如何解决这个问题有任何想法?我正在使用Tensorflow后端. 任何帮助将不胜感激！

This leaves me completely confused, thinking there might be a bug in the Keras MAE calculation. Has anyone had this issue before or have any ideas how it might be fixed? I am using the Tensorflow backend. Any help would be much appreciated!

我几乎可以肯定这是一个错误.如果我保持loss ='mae'但也添加metrics = ['mse'，'mae']，则度量标准返回的MAE与我的手动计算和身份模型相同.对于MSE，情况也是如此:如果我设置loss ='mse'，则指标返回的MSE与损失不同.

推荐答案

事实证明，由于正则化，损失应该与度量不同.使用正则化，损失会更高(在我的情况下)，因为当节点不如指定的活动时，正则化会增加损失.指标没有考虑到这一点，因此返回一个不同的值，该值等于手动计算错误时将得到的值.

It turns out that the loss is supposed to be different than the metric, because of the regularization. Using regularization, the loss is higher (in my case), because the regularization increases loss when the nodes are not as active as specified. The metrics don't take this into account, and therefore return a different value, which equals what one would get when manually computing the error.

这篇关于Keras预测给出的误差与评估值不同，损失与指标不同的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！