模型不学习 [英] Model not learning

问题描述

我有一个非常简单的脚本，它创建了一个keras模型，旨在像XOR门一样工作.

I have a pretty simple script that creates a keras model designed to act like an XOR gate.

我在get_data函数中生成了40000个数据点.它创建两个数组；输入数组按一定顺序包含1和0，输出为1或0.

I generate 40000 datapoints in the get_data function. It creates two arrays; an input array containing 1s and 0s in some order, and an output which is either a 1 or a 0.

当我运行代码时，似乎无法学习，并且每次训练时得到的结果都大不相同.

When I run the code it does not appear to learn and the results I get vary dramatically every time I train it.

from keras import models
from keras import layers

import numpy as np

from random import randint


def get_output(a, b): return 0 if a == b else 1


def get_data ():
    data = []
    targets = []

    for _ in range(40010):
        a, b = randint(0, 1), randint(0, 1)

        targets.append(get_output(a, b))
        data.append([a, b])

    return data, targets


data, targets = get_data()

data = np.array(data).astype("float32")
targets = np.array(targets).astype("float32")

test_x = data[40000:]
test_y = targets[40000:]

train_x = data[:40000]
train_y = targets[:40000]

model = models.Sequential()

# input
model.add(layers.Dense(2, activation='relu', input_shape=(2,)))

# hidden
# model.add(layers.Dropout(0.3, noise_shape=None, seed=None))
model.add(layers.Dense(2, activation='relu'))
# model.add(layers.Dropout(0.2, noise_shape=None, seed=None))
model.add(layers.Dense(2, activation='relu'))

# output
model.add(layers.Dense(1, activation='sigmoid')) # sigmoid puts between 0 and 1

model.summary() # print out summary of model

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

res = model.fit(train_x, train_y, epochs=2000, batch_size=200, validation_data=(test_x, test_y)) # train
    
print 'predict: \n', test_x
print model.predict(test_x)

输出

[[0. 1.]
 [1. 1.]
 [1. 1.]
 [0. 0.]
 [1. 0.]
 [0. 0.]
 [0. 0.]
 [0. 1.]
 [1. 1.]
 [1. 0.]]
[[0.6629775 ]
 [0.00603844]
 [0.00603844]
 [0.6629775 ]
 [0.6629775 ]
 [0.6629775 ]
 [0.6629775 ]
 [0.6629775 ]
 [0.00603844]
 [0.6629775 ]]

即使没有辍学层，我也得到了非常相似的结果.

Even without the dropout layers, I got very similar results.

推荐答案

您的问题有几个问题.

首先，您的导入是非正统的(与您的问题无关，是的，但这确实有助于遵守某些约定):

To start with, your imports are rather unorthodox (irrelevant to your issue, true, but it helps sticking to some conventions):

from keras.models import Sequential
from keras.layers import Dense
import numpy as np

第二，您不需要成千上万的XOR问题示例；只有四种组合:

Second, you don't need some thousands of examples for the XOR problem; there are only four combinations:

X = np.array([[0,0],[0,1],[1,0],[1,1]])
y = np.array([[0],[1],[1],[0]])

仅此而已.

第三，出于同样的原因，您实际上无法使用XOR获得验证"或测试"数据；在最简单的方法(即您可以尝试在此处尝试做的事情)中，您只能使用这4种组合来测试模型对函数的学习程度(因为没有更多！).

Third, for the very same reason, you can't actually have "validation" or "test" data with XOR; in the simplest approach (i.e. what you are arguably trying to do here), you can only test how well the model has learnt the function, using these 4 combinations (since there are no more!).

第四，您应该从一个简单的单层模型(具有多于2个单位，没有丢失)开始，然后根据需要逐步进行 :

Fourth, you should start with a simple one-hidden layer model (with somewhat more than 2 units and no dropout), and then proceed gradually if needed:

model = Sequential()
model.add(Dense(8, activation="relu", input_dim=2))
model.add(Dense(1, activation="sigmoid"))

model.compile(loss='binary_crossentropy', optimizer='adam')
model.fit(X, y, batch_size=1, epochs=1000)

这应该使您的损失降低至〜0.12；它对该功能的了解程度如何?

This should take your loss down to ~ 0.12; how well has it learnt the function?

model.predict(X)
# result:
array([[0.31054294],
       [0.9702552 ],
       [0.93392825],
       [0.04611744]], dtype=float32)

y
# result:
array([[0],
       [1],
       [1],
       [0]])

这样够好吗?好吧，我不知道-正确的答案永远是取决于"！但是，您现在有了一个起点(可以说是一个学习的网络)，可以从该起点继续进行进一步的实验...

Is this good enough? Well, I don't know - the correct answer is always "it depends"! But you now have a starting point (i.e. a network that arguably learns something), from which you can proceed to further experiments...

这篇关于模型不学习的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！