如何为TensorFlow估算器充分利用GPU? [英] How to make best use of GPU for TensorFlow Estimators?

问题描述

我在我的深度学习模型中使用了Tensorflow(CPU版本).在给定参数集(网络结构，隐藏层，alpha等)的情况下，特别是使用DNNRegressor Estimator进行训练.虽然我能够减少损失，但是模型花费了很大的学习时间(大约3天)，并且每100步耗时9秒.

I was using Tensorflow(CPU version) for my Deep Learning Model. Specifically using DNNRegressor Estimator for training, with given set of parameters (network structure, hidden layers, alpha etc.) Though I was able to reduce the loss, but model took very large time for learning (approx 3 days.) and time it was taking was 9 sec per 100th step.

我遇到这篇文章:- https://medium.com/towards-data-科学/如何训练tensorflow-models-79426dabd304 并发现GPU的学习速度更快.因此，我从AWS(单核GPU)中获取了p2.xlarge gpu.，其中包含4(vCPU)，12(ECU)和61(MiB).

I came accross this article :- https://medium.com/towards-data-science/how-to-traine-tensorflow-models-79426dabd304 and found that GPU's can be more faster to learn. So i took p2.xlarge gpu from AWS (single core GPU) with 4(vCPU), 12(ECU) and 61 (MiB).

但是学习速度是相同的，每100步为9秒.我正在使用与CPU上的估算器相同的代码，因为我读到估算器自己使用GPU.这是我的"nvidia-smi"命令输出.

But the learning rate is same at 9 sec per 100th step. I m using same code I used for Estimators on CPU, because I read that Estimators use GPU on their own. Here is my "nvidia-smi" command output.

• 这表明正在使用GPU内存，但是我的Volatile GPU-Util为1％.无法弄清楚，我错过了什么.它的设计工作是否相同，还是我遗漏了一些东西，因为每秒的全局步数对于估算器的CPU和GPU实现都是相同的.
• 我是否必须在DNNRegressor估算器代码中明确更改某些内容?

推荐答案

听起来您可能正在从csv读取并转换为pandas DataFrame，然后使用tensorflow的pandas_input_fn.这是pandas_input_fn的实现中的一个已知问题.您可以在 https://github.com/tensorflow/tensorflow/issues/13530 .

It sounds like you might be reading from csv and converting to a pandas DataFrame, and then using tensorflow's pandas_input_fn. This is a known issue with the implementation of pandas_input_fn. You can track the issue at https://github.com/tensorflow/tensorflow/issues/13530.

要解决此问题，可以对I/O使用其他方法(例如，从TFRecords读取).如果您想继续使用熊猫并增加步数/秒，则可以减小batch_size，尽管这可能会对估算者的学习能力产生负面影响.

To combat this, you can use a different method for i/o (reading from TFRecords, for example). If you'd like to continue using pandas and increase your steps/second, you can reduce your batch_size, though this may have negative consequences on your estimator's ability to learn.

这篇关于如何为TensorFlow估算器充分利用GPU?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！