TensorFlow中的二进制分类，损失和准确度出乎意料的大值 [英] Binary classification in TensorFlow, unexpected large values for loss and accuracy

问题描述

我正在尝试使用深度神经网络体系结构对二进制标签值--1和+1进行分类.这是我在tensorflow中执行此操作的代码.

I am trying to use a deep neural network architecture to classify against a binary label value - -1 and +1. Here is my code to do it in tensorflow.

import tensorflow as tf
import numpy as np
from preprocess import create_feature_sets_and_labels

train_x,train_y,test_x,test_y = create_feature_sets_and_labels()

x = tf.placeholder('float', [None, 5])
y = tf.placeholder('float')

n_nodes_hl1 = 500
n_nodes_hl2 = 500
n_nodes_hl3 = 500

n_classes = 1
batch_size = 100

def neural_network_model(data):

    hidden_1_layer = {'weights':tf.Variable(tf.random_normal([5, n_nodes_hl1])),
                      'biases':tf.Variable(tf.random_normal([n_nodes_hl1]))}

    hidden_2_layer = {'weights':tf.Variable(tf.random_normal([n_nodes_hl1, n_nodes_hl2])),
                      'biases':tf.Variable(tf.random_normal([n_nodes_hl2]))}

    hidden_3_layer = {'weights':tf.Variable(tf.random_normal([n_nodes_hl2, n_nodes_hl3])),
                      'biases':tf.Variable(tf.random_normal([n_nodes_hl3]))}

    output_layer = {'weights':tf.Variable(tf.random_normal([n_nodes_hl3, n_classes])),
                      'biases':tf.Variable(tf.random_normal([n_classes]))}


    l1 = tf.add(tf.matmul(data, hidden_1_layer['weights']), hidden_1_layer['biases'])
    l1 = tf.nn.relu(l1)

    l2 = tf.add(tf.matmul(l1, hidden_2_layer['weights']), hidden_2_layer['biases'])
    l2 = tf.nn.relu(l2)

    l3 = tf.add(tf.matmul(l2, hidden_3_layer['weights']), hidden_3_layer['biases'])
    l3 = tf.nn.relu(l3)

    output = tf.transpose(tf.add(tf.matmul(l3, output_layer['weights']), output_layer['biases']))
    return output



def train_neural_network(x):
    prediction = neural_network_model(x)
    cost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(prediction, y))
    optimizer = tf.train.AdamOptimizer().minimize(cost)

    hm_epochs = 10

    with tf.Session() as sess:
        sess.run(tf.initialize_all_variables())

        for epoch in range(hm_epochs):
            epoch_loss = 0
            i = 0
            while i < len(train_x):
                start = i
                end = i + batch_size
                batch_x = np.array(train_x[start:end])
                batch_y = np.array(train_y[start:end])

                _, c = sess.run([optimizer, cost], feed_dict={x: batch_x,
                                              y: batch_y})
                epoch_loss += c
                i+=batch_size

            print('Epoch', epoch, 'completed out of', hm_epochs, 'loss:', epoch_loss)

        # correct = tf.equal(tf.argmax(prediction, 1), tf.argmax(y, 1))
        # accuracy = tf.reduce_mean(tf.cast(correct, 'float'))

        print (test_x.shape)
        accuracy = tf.nn.l2_loss(prediction-y,name="squared_error_test_cost")/test_x.shape[0]
        print('Accuracy:', accuracy.eval({x: test_x, y: test_y}))

train_neural_network(x)

这是我运行此命令时得到的输出:

This is the output I get when I run this:

('Epoch', 0, 'completed out of', 10, 'loss:', -8400.2424869537354)
('Epoch', 1, 'completed out of', 10, 'loss:', -78980.956665039062)
('Epoch', 2, 'completed out of', 10, 'loss:', -152401.86713409424)
('Epoch', 3, 'completed out of', 10, 'loss:', -184913.46441650391)
('Epoch', 4, 'completed out of', 10, 'loss:', -165563.44775390625)
('Epoch', 5, 'completed out of', 10, 'loss:', -360394.44857788086)
('Epoch', 6, 'completed out of', 10, 'loss:', -475697.51550292969)
('Epoch', 7, 'completed out of', 10, 'loss:', -588638.92993164062)
('Epoch', 8, 'completed out of', 10, 'loss:', -745006.15966796875)
('Epoch', 9, 'completed out of', 10, 'loss:', -900172.41955566406)
(805, 5)
('Accuracy:', 5.8077128e+09)

我不知道我得到的值是否正确，因为确实缺乏非MNIST二进制分类示例.准确性与我所期望的不同.我期望的是百分比，而不是那么大的值.

I don't understand if the values I am getting are correct as there is a real dearth of non-MNIST binary classification examples. The accuracy is nothing like what I expected. I was expecting a percentage instead of that large value.

我还不确定机器学习背后的理论，这就是为什么我无法使用张量流来判断我的方法的正确性.

I am also somewhat unsure of the theory behind machine learning which is why I can't tell the correctness of my approach using tensorflow.

有人可以告诉我我对二进制分类的方法是否正确吗? 我的代码的准确性部分也正确吗?

Can someone please tell me if my approach towards binary classification is correct? Also is the accuracy part of my code correct?

推荐答案

来自此:

二进制标签值--1和+1

a binary label value - -1 and +1

. . .我假设您在train_y和test_y中的值实际上是-1.0和+1.0

. . . I am assuming your values in train_y and test_y are actually -1.0 and +1.0

这与您选择的损失函数sigmoid_cross_entropy_with_logits不能很好地配合-假定为0.0和+1.0.负y值引起混乱！然而，损失函数的选择对于二进制分类是有好处的.我建议将您的y值更改为0和1.

This is not going to work very well with your chosen loss function sigmoid_cross_entropy_with_logits - which assumes 0.0 and +1.0. The negative y values are causing mayhem! However, the loss function choice is good for binary classification. I suggest change your y values to 0 and 1.

此外，从技术上讲，网络的输出不是最终的预测.损耗函数sigmoid_cross_entropy_with_logits旨在与在输出层中具有S型传递函数的网络一起使用，尽管您已经正确地在 之前应用了损耗函数.因此您的训练代码看起来正确

In addition, technically the output of your network is not the final prediction. The loss function sigmoid_cross_entropy_with_logits is designed to work with a network with sigmoid transfer function in the output layer, although you have got it right that the loss function is applied before this is done. So your training code appears correct

尽管我不确定100％是否会使用tf.transpose-如果您亲自删除该内容，我会看到会发生什么情况.

I'm not 100% sure about the tf.transpose though - I would see what happens if you remove that, personally I.e.

output = tf.add(tf.matmul(l3, output_layer['weights']), output_layer['biases'])

无论哪种方式，这都是"logit"输出，而不是您的预测.对于非常自信的预测，output的值可能会变高，这可能会在以后由于缺少S形函数而说明您的非常高的值.因此，添加一个预测张量(这表示该示例处于正类中的概率/置信度):

Either way, this is the "logit" output, but not your prediction. The value of output can get high for very confident predictions, which probably explains your very high values later due to missing the sigmoid function. So add a prediction tensor (this represents the probability/confidence that the example is in the positive class):

prediction = tf.sigmoid(output)

您可以使用它来计算准确性.您的准确度计算不应基于L2误差，而应基于正确值的总和-更接近于您注释掉的代码(该代码似乎来自多类分类).要对二进制分类与真/假进行比较，您需要对预测进行阈值处理，并与真标签进行比较.像这样:

You can use that to calculate accuracy. Your accuracy calculation should not be based on L2 error, but sum of correct values - closer to the code you had commented out (which appears to be from a multiclass classification). For a comparison with true/false for binary classification, you need to threshold the predictions, and compare with the true labels. Something like this:

 predicted_class = tf.greater(prediction,0.5)
 correct = tf.equal(predicted_class, tf.equal(y,1.0))
 accuracy = tf.reduce_mean( tf.cast(correct, 'float') )

精度值应在0.0到1.0之间.如果要以百分比表示，请当然乘以100.

The accuracy value should be between 0.0 and 1.0. If you want as a percentage, just multiply by 100 of course.

这篇关于TensorFlow中的二进制分类，损失和准确度出乎意料的大值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！