如何使用TensorFlow获得稳定的结果，设置随机种子 [英] How to get stable results with TensorFlow, setting random seed

问题描述

我正在尝试使用不同的参数多次运行神经网络，以便校准网络参数(辍学概率，学习率).但是，我遇到的问题是，当我在如下所示的循环中运行网络时，在保持参数不变的情况下运行网络仍然为我提供了不同的解决方案:

I'm trying to run a neural network multiple times with different parameters in order to calibrate the networks parameters (dropout probabilities, learning rate e.d.). However I am having the problem that running the network while keeping the parameters the same still gives me a different solution when I run the network in a loop as follows:

filename = create_results_file()
for i in range(3):
  g = tf.Graph()
  with g.as_default():
    accuracy_result, average_error = network.train_network(
        parameters, inputHeight, inputWidth, inputChannels, outputClasses)
    f, w = get_csv_writer(filename)
    w.writerow([accuracy_result, "did run %d" % i, average_error])
    f.close()

在设置网络的层和错误功能之前，我在train_network函数的开始处使用以下代码:

I am using the following code at the start of my train_network function before setting up the layers and error function of my network:

np.random.seed(1)
tf.set_random_seed(1)

我也曾尝试在TensorFlow图形创建之前添加此代码，但是在结果输出中我一直得到不同的解决方案.

I have also tried adding this code before the TensorFlow graph creation, but I keep getting different solutions in my results output.

我正在使用AdamOptimizer，并正在使用tf.truncated_normal初始化网络权重.另外，我正在使用np.random.permutation随机播放每个时期的传入图像.

I am using an AdamOptimizer and am initializing network weights using tf.truncated_normal. Additionally I am using np.random.permutation to shuffle the incoming images for each epoch.

推荐答案

设置当前的TensorFlow随机种子仅影响当前的默认图形.由于要为训练创建一个新图并将其设置为默认图(with g.as_default():)，因此必须在该with块的范围内设置随机种子.

Setting the current TensorFlow random seed affects the current default graph only. Since you are creating a new graph for your training and setting it as default (with g.as_default():), you must set the random seed within the scope of that with block.

例如，您的循环应如下所示:

For example, your loop should look like the following:

for i in range(3):
  g = tf.Graph()
  with g.as_default():
    tf.set_random_seed(1)
    accuracy_result, average_error = network.train_network(
        parameters, inputHeight, inputWidth, inputChannels, outputClasses)

请注意，这将对外部for循环的每次迭代使用相同的随机种子.如果您希望在每次迭代中使用不同的种子，但仍要确定性，则可以使用tf.set_random_seed(i + 1).

Note that this will use the same random seed for each iteration of the outer for loop. If you want to use a different—but still deterministic—seed in each iteration, you can use tf.set_random_seed(i + 1).

这篇关于如何使用TensorFlow获得稳定的结果，设置随机种子的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！