TensorFlow:不可重复的结果 [英] TensorFlow: Non-repeatable results
问题描述
我有一个Python脚本，该脚本使用TensorFlow来创建多层感知器网络(带有辍学)，以便进行二进制分类.即使我很小心地设置了Python和TensorFlow种子，我仍然得到了不可重复的结果.如果我运行一次，然后再次运行，则会得到不同的结果.我什至可以运行一次，退出Python，重新启动Python，再次运行并获得不同的结果.
I have a Python script that uses TensorFlow to create a multilayer perceptron net (with dropout) in order to do binary classification. Even though I've been careful to set both the Python and TensorFlow seeds, I get non-repeatable results. If I run once and then run again, I get different results. I can even run once, quit Python, restart Python, run again and get different results.
我知道有人发布了有关在TensorFlow中获得不可重复结果的问题(例如，，，"如何在TensorFlow中获得可重现的结果".)，答案通常是对tf.set_random_seed()的错误使用/理解.我已经确定要执行给出的解决方案，但这并不能解决我的问题.
I know some people posted questions about getting non-repeatable results in TensorFlow (e.g., "How to get stable results...", "set_random_seed not working...", "How to get reproducible result in TensorFlow"), and the answers usually turn out to be an incorrect use/understanding of tf.set_random_seed(). I've made sure to implement the solutions given but that has not solved my problem.
一个常见的错误是没有意识到tf.set_random_seed()仅仅是图级别的种子，并且多次运行脚本会改变图，从而解释了不可重复的结果.我使用以下语句打印出整个图形，并(通过diff)验证了即使在结果不同的情况下该图形也是相同的.
A common mistake is not realizing that tf.set_random_seed() is only a graph-level seed and that running the script multiple times will alter the graph, explaining the non-repeatable results. I used the following statement to print out the entire graph and verified (via diff) that the graph is the same even when the results are different.
print [n.name for n in tf.get_default_graph().as_graph_def().node]
我还使用了tf.reset_default_graph()和tf.get_default_graph().finalize()之类的函数调用来避免对图形进行任何更改，即使这可能是过大的.
I've also used function calls like tf.reset_default_graph() and tf.get_default_graph().finalize() to avoid any changes to the graph even though this is probably overkill.
我的脚本长约360行，所以这里是相关的行(指示了已截断的代码). ALL_CAPS中的所有项目都是在下面的Parameters块中定义的常量.
My script is ~360 lines long so here are the relevant lines (with snipped code indicated). Any items that are in ALL_CAPS are constants that are defined in my Parameters block below.
import numpy as np
import tensorflow as tf
from copy import deepcopy
from tqdm import tqdm # Progress bar
# --------------------------------- Parameters ---------------------------------
(snip)
# --------------------------------- Functions ---------------------------------
(snip)
# ------------------------------ Obtain Train Data -----------------------------
(snip)
# ------------------------------ Obtain Test Data -----------------------------
(snip)
random.seed(12345)
tf.set_random_seed(12345)
(snip)
# ------------------------- Build the TensorFlow Graph -------------------------
tf.reset_default_graph()
with tf.Graph().as_default():
x = tf.placeholder("float", shape=[None, N_INPUT])
y_ = tf.placeholder("float", shape=[None, N_CLASSES])
# Store layers weight & bias
weights = {
'h1': tf.Variable(tf.random_normal([N_INPUT, N_HIDDEN_1])),
'h2': tf.Variable(tf.random_normal([N_HIDDEN_1, N_HIDDEN_2])),
'h3': tf.Variable(tf.random_normal([N_HIDDEN_2, N_HIDDEN_3])),
'out': tf.Variable(tf.random_normal([N_HIDDEN_3, N_CLASSES]))
}
biases = {
'b1': tf.Variable(tf.random_normal([N_HIDDEN_1])),
'b2': tf.Variable(tf.random_normal([N_HIDDEN_2])),
'b3': tf.Variable(tf.random_normal([N_HIDDEN_3])),
'out': tf.Variable(tf.random_normal([N_CLASSES]))
}
# Construct model
pred = multilayer_perceptron(x, weights, biases, USE_DROP_LAYERS, DROP_KEEP_PROB)
mean1 = tf.reduce_mean(weights['h1'])
mean2 = tf.reduce_mean(weights['h2'])
mean3 = tf.reduce_mean(weights['h3'])
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y_))
regularizers = (tf.nn.l2_loss(weights['h1']) + tf.nn.l2_loss(biases['b1']) +
tf.nn.l2_loss(weights['h2']) + tf.nn.l2_loss(biases['b2']) +
tf.nn.l2_loss(weights['h3']) + tf.nn.l2_loss(biases['b3']))
cost += COEFF_REGULAR * regularizers
optimizer = tf.train.GradientDescentOptimizer(LEARNING_RATE).minimize(cost)
out_labels = tf.nn.softmax(pred)
sess = tf.InteractiveSession()
sess.run(tf.initialize_all_variables())
tf.get_default_graph().finalize() # Lock the graph as read-only
#Print the default graph in text form
print [n.name for n in tf.get_default_graph().as_graph_def().node]
# --------------------------------- Training ----------------------------------
print "Start Training"
pbar = tqdm(total = TRAINING_EPOCHS)
for epoch in range(TRAINING_EPOCHS):
avg_cost = 0.0
batch_iter = 0
train_outfile.write(str(epoch))
while batch_iter < BATCH_SIZE:
train_features = []
train_labels = []
batch_segments = random.sample(train_segments, 20)
for segment in batch_segments:
train_features.append(segment[0])
train_labels.append(segment[1])
sess.run(optimizer, feed_dict={x: train_features, y_: train_labels})
line_out = "," + str(batch_iter) + "\n"
train_outfile.write(line_out)
line_out = ",," + str(sess.run(mean1, feed_dict={x: train_features, y_: train_labels}))
line_out += "," + str(sess.run(mean2, feed_dict={x: train_features, y_: train_labels}))
line_out += "," + str(sess.run(mean3, feed_dict={x: train_features, y_: train_labels})) + "\n"
train_outfile.write(line_out)
avg_cost += sess.run(cost, feed_dict={x: train_features, y_: train_labels})/BATCH_SIZE
batch_iter += 1
line_out = ",,,,," + str(avg_cost) + "\n"
train_outfile.write(line_out)
pbar.update(1) # Increment the progress bar by one
train_outfile.close()
print "Completed training"
# ------------------------------ Testing & Output ------------------------------
keep_prob = 1.0 # Do not use dropout when testing
print "now reducing mean"
print(sess.run(mean1, feed_dict={x: test_features, y_: test_labels}))
print "TRUE LABELS"
print(test_labels)
print "PREDICTED LABELS"
pred_labels = sess.run(out_labels, feed_dict={x: test_features})
print(pred_labels)
output_accuracy_results(pred_labels, test_labels)
sess.close()
什么是不可重复的
如您所见，我将每个时期的结果输出到文件中，并在最后打印出准确度数字.尽管我相信我已经正确设置了种子，但所有这些都不匹配.我已经用过random.seed(12345)和tf.set_random_seed(12345)
如果需要提供更多信息，请告诉我.在此先感谢您的帮助.
Please let me know if I need to provide more information. And thanks in advance for any help.
-DG
TensorFlow版本0.8.0(仅适用于CPU)
Enthought Canopy版本1.7.2(Python 2.7，而不是3. +)
Mac OS X版本10.11.3
TensorFlow version 0.8.0 (CPU only)
Enthought Canopy version 1.7.2 (Python 2.7, not 3.+)
Mac OS X version 10.11.3
推荐答案
除了图级种子，您还需要设置操作级种子，即
You need to set operation level seed in addition to graph-level seed, ie
tf.reset_default_graph()
a = tf.constant([1, 1, 1, 1, 1], dtype=tf.float32)
graph_level_seed = 1
operation_level_seed = 1
tf.set_random_seed(graph_level_seed)
b = tf.nn.dropout(a, 0.5, seed=operation_level_seed)
这篇关于TensorFlow:不可重复的结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
