Tensorflow 在使用 tf.device('/cpu:0') 时分配 GPU 内存 [英] Tensorflow allocating GPU memory when using tf.device('/cpu:0')

问题描述

系统信息:1.1.0、GPU、Windows、Python 3.5，代码在 ipython 控制台中运行.

System Info: 1.1.0, GPU, Windows, Python 3.5, code runs in ipython consoles.

我正在尝试运行两个不同的 Tensorflow 会话，一个在 GPU 上(执行一些批处理工作)，另一个在我用于快速测试的 CPU 上运行.

I am trying to run two different Tensorflow sessions, one on the GPU (that does some batch work) and one on the CPU that I use for quick tests while the other works.

问题是，当我使用 tf.device('/cpu:0') 指定 生成第二个会话时，该会话尝试分配 GPU 内存并使我的另一个会话崩溃.

The problem is that when I spawn the second session specifying with tf.device('/cpu:0') the session tries to allocate GPU memory and crashes my other session.

我的代码:

import os
os.environ["CUDA_VISIBLE_DEVICES"] = ""
import time

import tensorflow as tf

with tf.device('/cpu:0'):
  with tf.Session() as sess:
    # Here 6 GBs of GPU RAM are allocated.
    time.sleep(5)

如何强制 Tensorflow 忽略 GPU?

How do I force Tensorflow to ignore the GPU?

正如@Nicolas 在评论中所建议的，我看了一下 在这个答案 并跑了

As suggested in a comment by @Nicolas, I took a look at this answer and ran

import os
os.environ["CUDA_VISIBLE_DEVICES"] = ""
import tensorflow as tf

from tensorflow.python.client import device_lib
print(device_lib.list_local_devices())

打印:

[name: "/cpu:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 2215045474989189346
, name: "/gpu:0"
device_type: "GPU"
memory_limit: 6787871540
locality {
  bus_id: 1
}
incarnation: 13663872143510826785
physical_device_desc: "device: 0, name: GeForce GTX 1080, pci bus id: 0000:02:00.0"
]

在我看来，即使我明确告诉脚本忽略任何 CUDA 设备，它仍然会找到并使用它们.这可能是 TF 1.1 的错误吗?

It seems to me that even if I explicitly tell the script to ignore any CUDA devices, it still finds and uses them. Could this be a bug of TF 1.1?

推荐答案

事实证明，将 CUDA_VISIBLE_DEVICES 设置为空字符串并不会屏蔽对用户可见的 CUDA 设备脚本.

It turns out that setting CUDA_VISIBLE_DEVICES to the empty string does not mask the CUDA devices visible to the script.

来自 CUDA_VISIBLE_DEVICES 的文档(重点是我加的):

From the documentation of CUDA_VISIBLE_DEVICES (emphasis added by me):

只有索引出现在序列中的设备才可见CUDA 应用程序，它们按顺序枚举序列.如果其中一个索引无效，则只有其无效索引之前的索引对 CUDA 应用程序可见.为了例如，将 CUDA_VISIBLE_DEVICES 设置为 2,1 会导致设备 0 为invisible 并且设备 2 在设备 1 之前被枚举.设置CUDA_VISIBLE_DEVICES 到 0,2,-1,1 导致设备 0 和 2 可见并且设备 1 不可见.

Only the devices whose index is present in the sequence are visible to CUDA applications and they are enumerated in the order of the sequence. If one of the indices is invalid, only the devices whose index precedes the invalid index are visible to CUDA applications. For example, setting CUDA_VISIBLE_DEVICES to 2,1 causes device 0 to be invisible and device 2 to be enumerated before device 1. Setting CUDA_VISIBLE_DEVICES to 0,2,-1,1 causes devices 0 and 2 to be visible and device 1 to be invisible.

似乎空字符串过去被处理为不存在有效设备"，但含义已更改，因为文档中未提及.

It seems like the empty string used to be handled as "no valid devices exist" but changed meaning, as it is not mentioned in the documentation.

将代码更改为 os.environ["CUDA_VISIBLE_DEVICES"] = "-1" 解决了问题.运行

Changing the code to os.environ["CUDA_VISIBLE_DEVICES"] = "-1" fixes the problem. Running

import os
os.environ["CUDA_VISIBLE_DEVICES"]="-1"    
import tensorflow as tf

from tensorflow.python.client import device_lib
print(device_lib.list_local_devices())

现在打印

[name: "/cpu:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 14097726166554667970
]

并且实例化 tf.Session 不再占用 GPU 内存.

and instantiating a tf.Session does not hog GPU memory anymore.

这篇关于Tensorflow 在使用 tf.device('/cpu:0') 时分配 GPU 内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！