使用Tensorflow的神经网络不会更新权重/偏差 [英] Neural Network with Tensorflow doesn't update weights/bias

问题描述

我正在尝试将某些64x64图像分类为黑匣子练习.我写的神经网络不会改变我的体重.第一次编写类似这样的代码，但是在 MNIST 字母输入上可以正常工作，但是在此代码上，它不会像应该这样训练:

I'm trying to classify some 64x64 images as a black box exercise. The NN I have written doesn't change my weights. First time writing something like this, the same code, but on MNIST letters input works just fine, but on this code it does not train like it should:

import tensorflow as tf
import numpy as np


path = ""

# x is a holder for the 64x64 image
x = tf.placeholder(tf.float32, shape=[None, 4096])

# y_ is a 1 element vector, containing the predicted probability of the label
y_ = tf.placeholder(tf.float32, [None, 1])

# define weights and balances
W = tf.Variable(tf.zeros([4096, 1]))
b = tf.Variable(tf.zeros([1]))

# define our model
y = tf.nn.softmax(tf.matmul(x, W) + b)

# loss is cross entropy
cross_entropy = tf.reduce_mean(
                tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))

# each training step in gradient decent we want to minimize cross entropy
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)

train_labels = np.reshape(np.genfromtxt(path + "train_labels.csv", delimiter=',', skip_header=1), (14999, 1))
train_data = np.genfromtxt(path + "train_samples.csv", delimiter=',', skip_header=1)

# perform 150 training steps with each taking 100 train data
for i in range(0, 15000, 100):
    sess.run(train_step, feed_dict={x: train_data[i:i+100], y_: train_labels[i:i+100]})
    if i % 500 == 0:
        print(sess.run(cross_entropy, feed_dict={x: train_data[i:i+100], y_: train_labels[i:i+100]}))
        print(sess.run(b), sess.run(W))

correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


sess.close()

我该如何解决这个问题?

How do I solve this problem?

推荐答案

问题的关键是您输出y_和y的类号是1.在以下情况下应采用one-hot模式您在张量流中的分类问题上使用tf.nn.softmax_cross_entropy_with_logits. tf.nn.softmax_cross_entropy_with_logits将首先计算tf.nn.softmax.当您的班级号为1时，您的结果将完全相同.例如:

The key to the problem is that the class number of you output y_ and y is 1.You should adopt one-hot mode when you use tf.nn.softmax_cross_entropy_with_logits on classification problems in tensorflow. tf.nn.softmax_cross_entropy_with_logits will first compute tf.nn.softmax. When your class number is 1, your results are all the same. For example:

import tensorflow as tf

y = tf.constant([[1],[0],[1]],dtype=tf.float32)
y_ = tf.constant([[1],[2],[3]],dtype=tf.float32)

softmax_var = tf.nn.softmax(logits=y_)
cross_entropy = tf.multiply(y, tf.log(softmax_var))

errors = tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y)

with tf.Session() as sess:
    print(sess.run(softmax_var))
    print(sess.run(cross_entropy))
    print(sess.run(errors))

[[1.]
 [1.]
 [1.]]
[[0.]
 [0.]
 [0.]]
[0. 0. 0.]

这意味着无论您输出什么y_，您的损失将为零.因此，您的weights和bias尚未更新.

This means that no matter what your output y_, your loss will be zero. So your weights and bias haven't been updated.

解决方案是修改y_和y的类号.

The solution is to modify the class number of y_ and y.

我想您的课程编号是n.

I suppose your class number is n.

第一次处理:您可以在Feed数据之前将数据更改为one-hot，然后使用以下代码.

First approch:You can change data to one-hot before feed data.Then use the following code.

y_ = tf.placeholder(tf.float32, [None, n])
W = tf.Variable(tf.zeros([4096, n]))
b = tf.Variable(tf.zeros([n]))

第二种方法:在Feed数据之后将数据更改为one-hot.

Second approch：change data to one-hot after feed data.

y_ = tf.placeholder(tf.int32, [None, 1])
y_ = tf.one_hot(y_,n) # your dtype of y_ need to be tf.int32
W = tf.Variable(tf.zeros([4096, n]))
b = tf.Variable(tf.zeros([n]))

这篇关于使用Tensorflow的神经网络不会更新权重/偏差的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！