Tensorflow:即使关闭Session也会发生内存泄漏? [英] Tensorflow : Memory leak even while closing Session?

问题描述

当我意识到即使我在for循环中关闭当前Session时，我只是在尝试四元神经网络的某些东西，我的程序也会大量减慢速度，并且由于构建ops而导致内存泄漏.这是我的代码:

I was just trying some stuff for a quaternionic neural network when I realized that, even if I close my current Session in a for loop, my program slows down massively and I get a memory leak caused by ops being constructed. This is my code:

for step in xrange(0,200):#num_epochs * train_size // BATCH_SIZE):
338 
339         with tf.Session() as sess:
340 
341             offset = (BATCH_SIZE) % train_size
342             #print "Offset : %d" % offset
343 
344             batch_data = []
345             batch_labels = []
346             batch_data.append(qtrain[0][offset:(offset + BATCH_SIZE)])
347             batch_labels.append(qtrain_labels[0][offset:(offset + BATCH_SIZE)]
352             retour = sess.run(test, feed_dict={x: batch_data})
357 
358             test2 = feedForwardStep(retour, W_to_output,b_output)
367             #sess.close()

问题似乎来自test2 = feedForward(..).我必须在执行retour后一次声明这些操作，因为retour不能是占位符(我需要对其进行迭代).没有这一行，该程序将运行得非常好，速度快，并且不会出现内存泄漏.我不明白为什么即使关闭会话，TensorFlow仍试图保存" test2的原因……

The problem seems to come from test2 = feedForward(..). I need to declare these ops after executing retour once, because retour can't be a placeholder (I need to iterate through it). Without this line, the program runs very well, fast and without a memory leak. I can't understand why it seems like TensorFlow is trying to "save" test2 even if I close the session ...

推荐答案

TL; DR::关闭会话不会释放

TL;DR: Closing a session does not free the tf.Graph data structure in your Python program, and if each iteration of the loop adds nodes to the graph, you'll have a leak.

由于函数feedForwardStep创建了新的TensorFlow操作，并在for循环中调用了它，所以代码中存在 泄漏-尽管很细微.

Since your function feedForwardStep creates new TensorFlow operations, and you call it within the for loop, then there is a leak in your code—albeit a subtle one.

除非另行指定(否则，请使用 with tf.Graph().as_default(): 块)，所有TensorFlow操作都会添加到全局默认图表中.这意味着对tf.constant()，tf.matmul()，tf.Variable()等的每次调用都会将对象添加到全局数据结构中.有两种方法可以避免这种情况:

Unless you specify otherwise (using a with tf.Graph().as_default(): block), all TensorFlow operations are added to a global default graph. This means that every call to tf.constant(), tf.matmul(), tf.Variable() etc. adds objects to a global data structure. There are two ways to avoid this:

1. 结构化程序，以便一次构建图形，然后使用 tf.placeholder() 选择在每次迭代中输入不同的值.您在问题中提到不可能做到这一点.

1. Structure your program so that you build the graph once, then use tf.placeholder() ops to feed in different values in each iteration. You mention in your question that this might not be possible.

在每个for循环中显式创建一个新图.如果图的结构取决于当前迭代中可用的数据，则可能有必要.您将按照以下步骤进行操作:

Explicitly create a new graph in each for loop. This might be necessary if the structure of the graph depends on the data available in the current iteration. You would do this as follows:

for step in xrange(200):
    with tf.Graph().as_default(), tf.Session() as sess:
        # Remainder of loop body goes here.

请注意，在此版本中，不能使用上一个迭代中的Tensor或Operation对象. (例如，从您的代码片段中不清楚test的来源.)

Note that in this version, you cannot use Tensor or Operation objects from a previous iteration. (For example, it's not clear from your code snippet where test comes from.)

这篇关于Tensorflow:即使关闭Session也会发生内存泄漏?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！