Keras/Tensorflow是否降低了GPU使用率? [英] Low GPU usage by Keras / Tensorflow?
问题描述
我在配备nvidia Tesla K20c GPU的计算机上使用具有tensorflow后端的keras. (CUDA 8)
I'm using keras with tensorflow backend on a computer with a nvidia Tesla K20c GPU. (CUDA 8)
我正在转换一个相对简单的卷积神经网络，在训练过程中，我运行终端程序nvidia-smi来检查GPU的使用情况.如下面的输出所示，GPU利用率通常显示为7％-13％
I'm tranining a relatively simple Convolutional Neural Network, during training I run the terminal program nvidia-smi to check the GPU use. As you can see in the following output, the GPU utilization commonly shows around 7%-13%
我的问题是:在CNN培训期间，GPU使用率不应该更高吗?这是否表示keras/tensorflow的GPU配置错误或使用不正确?
My question is: during the CNN training shouldn't the GPU usage be higher? is this a sign of a bad GPU configuration or usage by keras/tensorflow?
推荐答案
可能是由于多种原因，但最有可能是您在阅读训练数据时遇到了瓶颈.由于您的GPU已处理了一批，因此需要更多数据.根据您的实现，这可能会导致GPU等待CPU加载更多数据，从而降低GPU使用率并延长训练时间.
Could be due to several reasons but most likely you're having a bottleneck when reading the training data. As your GPU has processed a batch it requires more data. Depending on your implementation this can cause the GPU to wait for the CPU to load more data resulting in a lower GPU usage and also a longer training time.
尝试将所有数据加载到内存中(如果适合)或使用 QueueRunner ，这将使输入管道在后台读取数据.这样可以减少GPU等待更多数据的时间.
Try loading all data into memory if it fits or use a QueueRunner which will make an input pipeline reading data in the background. This will reduce the time that your GPU is waiting for more data.
TensorFlow网站上的阅读数据指南包含更多信息.
The Reading Data Guide on the TensorFlow website contains more information.
这篇关于Keras/Tensorflow是否降低了GPU使用率?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
