Lasagne Dropoutlayer无法有效利用GPU [英] Lasagne dropoutlayer does not utilize GPU efficiently

问题描述

我正在将theano和烤宽面条用于DNN语音增强项目.我使用的前馈网络与lasagne文档(/github.com/Lasagne/Lasagne/blob/master/examples/mnist.py)中的mnist示例非常相似.该网络使用多个辍学层.我在Nvidia Titan X GPU上训练网络.但是，当我不使用中断时，我的GPU利用率大约为60％，一个时期大约需要60秒，但是当我使用中断时，我的GPU利用率下降到8％，每个时期大约为600s.这与辍学率设置为20％或0.1％无关.

I am using theano and lasagne for a DNN speech enhancement project. I use a feed-forward network very similar to the mnist example in the lasagne documentation (/github.com/Lasagne/Lasagne/blob/master/examples/mnist.py). This network uses several dropout layers. I train my network on an Nvidia Titan X GPU. However, when I do not use dropout my GPU utilization is approximately 60% and one epoch takes around 60s but when I use dropout my GPU utilization drops to 8% and each epoch takes approximately 600s. This is regardless of the dropout rate is set to 20% or 0.1%.

最初，我认为这是由于用于生成辍学掩码的随机数生成器(RNG)未能在GPU上运行.但是，在代码中( https://github.com/Lasagne/Lasagne/blob/master/lasagne/layers/noise.py )似乎已使用rng_mrg，它应基于以下链接在GPU上运行:

Initially I thought it was due to the random number generator (RNG) used to generate the dropout mask, that did not run on the GPU. However, in the code (https://github.com/Lasagne/Lasagne/blob/master/lasagne/layers/noise.py) it seems like rng_mrg is used, which should be running on the GPU based on this link: http://deeplearning.net/software/theano/tutorial/examples.html#other-implementations

运行theano分析器显示"theano.sandbox.rng_mrg.mrg_uniform"占执行时间的86.7％，我不明白.

Running the theano profiler shows that "theano.sandbox.rng_mrg.mrg_uniform" takes up 86.7% of the execution time, which I do not understand.

如果有人对杀死我的GPU利用率有什么想法，我将不胜感激.

If anyone has an idea of what kills my GPU utilization I would appreciate it.

推荐答案

如果您查看 GPU版本相同的生成器，但是您正在运行的代码显然没有使用它.

If you look at the code for mrg_uniform, you can see that it is a pure python CPU implementation of the random generator. You can also see that there is a GPU version of that same generator, but the code you are running apparently does not use it.

因此，答案不是因为您使用的是纯Python随机生成器，而是GPU利用率大幅下降，而CPU利用率却大大提高.解决方案显然是弄清楚如何切换到GPU加速的随机生成器.

So the answer isn't that your GPU utilisation is going down so much as your CPU utilisation is greatly increasing because you are using a pure Python random generator. The solution would obviously be to work out how to switch to a GPU accelerated random generator.

这篇关于Lasagne Dropoutlayer无法有效利用GPU的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！