Tensorflow:即使在关闭会话时内存泄漏? [英] Tensorflow : Memory leak even while closing Session?

问题描述

当我意识到即使我在 for 循环中关闭当前 Session 时，我只是在为四元神经网络尝试一些东西时，我的程序也会大幅减慢，并且我会因构造操作而导致内存泄漏.这是我的代码:

I was just trying some stuff for a quaternionic neural network when I realized that, even if I close my current Session in a for loop, my program slows down massively and I get a memory leak caused by ops being constructed. This is my code:

for step in xrange(0,200):#num_epochs * train_size // BATCH_SIZE):
338 
339         with tf.Session() as sess:
340 
341             offset = (BATCH_SIZE) % train_size
342             #print "Offset : %d" % offset
343 
344             batch_data = []
345             batch_labels = []
346             batch_data.append(qtrain[0][offset:(offset + BATCH_SIZE)])
347             batch_labels.append(qtrain_labels[0][offset:(offset + BATCH_SIZE)]
352             retour = sess.run(test, feed_dict={x: batch_data})
357 
358             test2 = feedForwardStep(retour, W_to_output,b_output)
367             #sess.close()

问题似乎来自 test2 = feedForward(..).我需要在执行一次 retour 后声明这些操作，因为 retour 不能作为占位符(我需要遍历它).没有这一行，程序运行得非常好，速度快，而且没有内存泄漏.我不明白为什么即使我关闭会话，TensorFlow 似乎也试图保存" test2 ...

The problem seems to come from test2 = feedForward(..). I need to declare these ops after executing retour once, because retour can't be a placeholder (I need to iterate through it). Without this line, the program runs very well, fast and without a memory leak. I can't understand why it seems like TensorFlow is trying to "save" test2 even if I close the session ...

推荐答案

TL;DR: 关闭会话并不能释放 tf.Graph 数据结构在你的 Python 程序中，如果循环的每次迭代向图中添加节点，您将有泄漏.

TL;DR: Closing a session does not free the tf.Graph data structure in your Python program, and if each iteration of the loop adds nodes to the graph, you'll have a leak.

由于您的函数 feedForwardStep 创建了新的 TensorFlow 操作，并且您在 for 循环中调用它，那么您的代码中存在泄漏—虽然很微妙.

Since your function feedForwardStep creates new TensorFlow operations, and you call it within the for loop, then there is a leak in your code—albeit a subtle one.

除非您另有说明(使用 with tf.Graph().as_default(): 块)，所有 TensorFlow 操作都添加到全局默认图中.这意味着每次调用 tf.constant()、tf.matmul()、tf.Variable() 等都会将对象添加到全局数据结构.有两种方法可以避免这种情况:

Unless you specify otherwise (using a with tf.Graph().as_default(): block), all TensorFlow operations are added to a global default graph. This means that every call to tf.constant(), tf.matmul(), tf.Variable() etc. adds objects to a global data structure. There are two ways to avoid this:

1. 构建您的程序，以便您构建一次图形，然后使用 tf.placeholder() 操作在每次迭代中输入不同的值.您在问题中提到这可能是不可能的.

1. Structure your program so that you build the graph once, then use tf.placeholder() ops to feed in different values in each iteration. You mention in your question that this might not be possible.

在每个 for 循环中显式地创建一个新图.如果图的结构取决于当前迭代中可用的数据，则这可能是必要的.您可以按如下方式执行此操作:

Explicitly create a new graph in each for loop. This might be necessary if the structure of the graph depends on the data available in the current iteration. You would do this as follows:

for step in xrange(200):
    with tf.Graph().as_default(), tf.Session() as sess:
        # Remainder of loop body goes here.

请注意，在此版本中，您不能使用之前迭代中的 Tensor 或 Operation 对象.(例如，从您的代码片段中不清楚 test 来自哪里.)

Note that in this version, you cannot use Tensor or Operation objects from a previous iteration. (For example, it's not clear from your code snippet where test comes from.)

这篇关于Tensorflow:即使在关闭会话时内存泄漏?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！