如何使用 TensorFlow 获得稳定的结果，设置随机种子 [英] How to get stable results with TensorFlow, setting random seed

问题描述

我试图用不同的参数多次运行神经网络，以校准网络参数(辍学概率、学习率 e.d.).但是我遇到的问题是，当我在循环中运行网络时，在保持参数相同的情况下运行网络仍然给我一个不同的解决方案:

I'm trying to run a neural network multiple times with different parameters in order to calibrate the networks parameters (dropout probabilities, learning rate e.d.). However I am having the problem that running the network while keeping the parameters the same still gives me a different solution when I run the network in a loop as follows:

filename = create_results_file()
for i in range(3):
  g = tf.Graph()
  with g.as_default():
    accuracy_result, average_error = network.train_network(
        parameters, inputHeight, inputWidth, inputChannels, outputClasses)
    f, w = get_csv_writer(filename)
    w.writerow([accuracy_result, "did run %d" % i, average_error])
    f.close()

在设置网络的层和误差函数之前，我在 train_network 函数的开头使用以下代码:

I am using the following code at the start of my train_network function before setting up the layers and error function of my network:

np.random.seed(1)
tf.set_random_seed(1)

我也尝试在创建 TensorFlow 图之前添加此代码，但在结果输出中我不断得到不同的解决方案.

I have also tried adding this code before the TensorFlow graph creation, but I keep getting different solutions in my results output.

我正在使用 AdamOptimizer 并使用 tf.truncated_normal 初始化网络权重.此外，我正在使用 np.random.permutation 对每个时期的传入图像进行洗牌.

I am using an AdamOptimizer and am initializing network weights using tf.truncated_normal. Additionally I am using np.random.permutation to shuffle the incoming images for each epoch.

推荐答案

设置当前的 TensorFlow 随机种子只会影响当前的默认图.由于您正在为您的训练创建一个新图并将其设置为默认值(with g.as_default():)，您必须在该 with 范围内设置随机种子> 阻止.

Setting the current TensorFlow random seed affects the current default graph only. Since you are creating a new graph for your training and setting it as default (with g.as_default():), you must set the random seed within the scope of that with block.

例如，您的循环应如下所示:

For example, your loop should look like the following:

for i in range(3):
  g = tf.Graph()
  with g.as_default():
    tf.set_random_seed(1)
    accuracy_result, average_error = network.train_network(
        parameters, inputHeight, inputWidth, inputChannels, outputClasses)

请注意，这将为外部 for 循环的每次迭代使用相同的随机种子.如果您想在每次迭代中使用不同的—但仍然具有确定性的—种子，您可以使用 tf.set_random_seed(i + 1).

Note that this will use the same random seed for each iteration of the outer for loop. If you want to use a different—but still deterministic—seed in each iteration, you can use tf.set_random_seed(i + 1).

这篇关于如何使用 TensorFlow 获得稳定的结果，设置随机种子的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！