使用Softmax和奇怪输出的神经网络 [英] Neural Network using Softmax with strange outputs

问题描述

我正在尝试使用S型激活隐藏层和3类的softmax输出层构建张量流神经网络.输出大部分都是非常糟糕的，我相信这是因为我在模型构建中犯了一个错误，因为我已经使用Matlab构建了类似的模型，并且效果很好.数据被标准化.这些结果看起来像这样:

I'm trying to build a tensorflow neural network using a sigmoid activation hidden layer and a softmax output layer with 3 classes. The outputs are mostly very bad and I believe it is because I am making a mistake in my model construction because I've built a similar model with Matlab and the results have been good. The data is normalized. These results look like this:

[9.2164397e-01 1.6932052e-03 7.6662831e-02]
[3.4100169e-01 2.2419590e-01 4.3480241e-01]
[2.3466848e-06 1.3276369e-04 9.9986482e-01]
[6.5199631e-01 3.4800139e-01 2.3596617e-06]
[9.9879754e-01 9.0103465e-05 1.1123115e-03]
[6.5749985e-01 2.8860433e-02 3.1363973e-01]

我的nn看起来像这样:

My nn looks like this:

def multilayer_perceptron(x, weights, biases, keep_prob):
    layer_1 = tf.add(tf.matmul(x, weights['h1']), biases['b1'])
    layer_1 = tf.nn.sigmoid(layer_1)
    layer_1 = tf.nn.dropout(layer_1, keep_prob)
    out_layer = tf.nn.softmax(tf.add(tf.matmul(layer_1,weights['out']),biases['out']))
    return out_layer

具有以下成本函数:

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=predictions, labels=y))

我越来越相信我的实现是不正确的，并且我做的事情很愚蠢.在Google上花费大量时间并查看其他示例并没有帮助.

I'm growing convinced that my implementation is incorrect and I am doing something very silly. Hours on google and looking at other examples hasn't helped.

更新:当我更改成本函数(如下所示)时，我得到了不错的结果.不过，这感觉不对.

UPDATE: When I changed the cost function (shown below), I get decent results. This feels wrong though.

cost = tf.losses.mean_squared_error(predictions=predictions, labels=y)

推荐答案

您的cost函数在模型输出的顶部实现了softmax，该输出还具有softmax.您应该删除损失函数中的一个.除此之外，您的代码看起来还不错:您确定:两个模型的拓扑结构(辍学率，层数/每层神经元数)是否相同? 您确定没有按班级顺序排列吗?两次培训后损失和验证损失指标如何?

Your cost function implements a softmax atop of your model output which also has a softmax. You should remove the one in the loss function. Besides this your code seems fine: Are you sure: That the topology (dropout rate, number of layers number of neurons per layer) are the same with both of your models? Are you sure you didn't swar the order of your classes. What about loss and validation loss metric after both trainings?

这篇关于使用Softmax和奇怪输出的神经网络的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！