神经网络总是预测相同的类 [英] Neural network always predicts the same class

问题描述

我正在尝试实现一个将图像分类为两个离散类别之一的神经网络.但是问题是，当前它始终为任何输入预测0，但我不确定为什么.

I'm trying to implement a neural network that classifies images into one of the two discrete categories. The problem is, however, that it currently always predicts 0 for any input and I'm not really sure why.

这是我的特征提取方法:

Here's my feature extraction method:

def extract(file):
    # Resize and subtract mean pixel
    img = cv2.resize(cv2.imread(file), (224, 224)).astype(np.float32)
    img[:, :, 0] -= 103.939
    img[:, :, 1] -= 116.779
    img[:, :, 2] -= 123.68
    # Normalize features
    img = (img.flatten() - np.mean(img)) / np.std(img)

    return np.array([img])

这是我的梯度下降例程:

Here's my gradient descent routine:

def fit(x, y, t1, t2):
    """Training routine"""
    ils = x.shape[1] if len(x.shape) > 1 else 1
    labels = len(set(y))

    if t1 is None or t2 is None:
        t1 = randweights(ils, 10)
        t2 = randweights(10, labels)

    params = np.concatenate([t1.reshape(-1), t2.reshape(-1)])
    res = grad(params, ils, 10, labels, x, y)
    params -= 0.1 * res

    return unpack(params, ils, 10, labels)

这是我的向前和向后(渐变)传播:

Here are my forward and back(gradient) propagations:

def forward(x, theta1, theta2):
    """Forward propagation"""

    m = x.shape[0]

    # Forward prop
    a1 = np.vstack((np.ones([1, m]), x.T))
    z2 = np.dot(theta1, a1)

    a2 = np.vstack((np.ones([1, m]), sigmoid(z2)))
    a3 = sigmoid(np.dot(theta2, a2))

    return (a1, a2, a3, z2, m)

def grad(params, ils, hls, labels, x, Y, lmbda=0.01):
    """Compute gradient for hypothesis Theta"""

    theta1, theta2 = unpack(params, ils, hls, labels)

    a1, a2, a3, z2, m = forward(x, theta1, theta2)
    d3 = a3 - Y.T
    print('Current error: {}'.format(np.mean(np.abs(d3))))

    d2 = np.dot(theta2.T, d3) * (np.vstack([np.ones([1, m]), sigmoid_prime(z2)]))
    d3 = d3.T
    d2 = d2[1:, :].T

    t1_grad = np.dot(d2.T, a1.T)
    t2_grad = np.dot(d3.T, a2.T)

    theta1[0] = np.zeros([1, theta1.shape[1]])
    theta2[0] = np.zeros([1, theta2.shape[1]])

    t1_grad = t1_grad + (lmbda / m) * theta1
    t2_grad = t2_grad + (lmbda / m) * theta2

    return np.concatenate([t1_grad.reshape(-1), t2_grad.reshape(-1)])

这是我的预测函数:

def predict(theta1, theta2, x):
    """Predict output using learned weights"""
    m = x.shape[0]

    h1 = sigmoid(np.hstack((np.ones([m, 1]), x)).dot(theta1.T))
    h2 = sigmoid(np.hstack((np.ones([m, 1]), h1)).dot(theta2.T))

    return h2.argmax(axis=1)

我可以看到，每次迭代的错误率都在逐渐降低，通常会收敛在1.26e-05左右.

I can see that the error rate is gradually decreasing with each iteration, generally converging somewhere around 1.26e-05.

到目前为止我已经尝试过的:

What I've tried so far:

1. PCA
2. 不同的数据集(来自sklearn的虹膜和来自Coursera ML课程的手写数字，两者均达到约95％的准确性).但是，这两个都是分批处理的，因此我可以假定我的一般实现是正确的，但是提取特征或训练分类器的方式有问题.
3. 尝试了sklearn的SGDClassifier，它的性能并没有好很多，给我带来了〜50％的准确度.那么功能有什么问题吗?

修改: h2的平均输出如下所示:

Edit: An average output of h2 looks like the following:

[0.5004899   0.45264441]
[0.50048522  0.47439413]
[0.50049019  0.46557124]
[0.50049261  0.45297816]

因此，所有验证示例的Sigmoid输出都非常相似.

So, very similar sigmoid outputs for all validation examples.

推荐答案

经过一个半星期的研究，我认为我明白了问题所在.代码本身没有错.妨碍我的实现成功分类的唯一两个问题是花时间学习和正确选择学习率/正则化参数.

After a week and a half of research I think I understand what the issue is. There is nothing wrong with the code itself. The only two issues that prevent my implementation from classifying successfully are time spent learning and proper selection of learning rate / regularization parameters.

我已经使学习例程运行了一段时间，尽管仍有大量的改进空间，但它已经将75％的准确性提高了.

I've had the learning routine running for some tome now, and it's pushing 75% accuracy already, though there is still plenty of space for improvement.

这篇关于神经网络总是预测相同的类的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！