何时调用 cudaDeviceSynchronize? [英] When to call cudaDeviceSynchronize?

问题描述

何时真正需要调用 cudaDeviceSynchronize 函数?

when is calling to the cudaDeviceSynchronize function really needed?.

据我从 CUDA 文档中了解到，CUDA 内核是异步的，因此我们似乎应该在每次内核启动后调用 cudaDeviceSynchronize.但是，我尝试了相同的代码(训练神经网络)，有和没有任何 cudaDeviceSynchronize，除了时间测量之前的一个.我发现我得到了相同的结果，但速度提高了 7-12 倍(取决于矩阵大小).

As far as I understand from the CUDA documentation, CUDA kernels are asynchronous, so it seems that we should call cudaDeviceSynchronize after each kernel launch. However, I have tried the same code (training neural networks) with and without any cudaDeviceSynchronize, except one before the time measurement. I have found that I get the same result but with a speed up between 7-12x (depending on the matrix sizes).

所以，问题是是否有任何理由使用 cudaDeviceSynchronize 除了时间测量.

So, the question is if there are any reasons to use cudaDeviceSynchronize apart of time measurement.

例如:

• 在使用 cudaMemcpy 将数据从 GPU 复制回主机之前是否需要?

• Is it needed before copying data from the GPU back to the host with cudaMemcpy?

如果我做矩阵乘法

C = A * B
D = C * F

我应该把 cudaDeviceSynchronize 放在两者之间吗?

should I put cudaDeviceSynchronize between both?

从我的实验看来我没有.

From my experiment It seems that I don't.

为什么 cudaDeviceSynchronize 让程序这么慢?

Why does cudaDeviceSynchronize slow the program so much?

推荐答案

虽然 CUDA 内核启动是异步的，但所有与 GPU 相关的任务放在一个流中(这是默认行为)都是按顺序执行的.

Although CUDA kernel launches are asynchronous, all GPU-related tasks placed in one stream (which is the default behavior) are executed sequentially.

例如，

kernel1<<<X,Y>>>(...); // kernel start execution, CPU continues to next statement
kernel2<<<X,Y>>>(...); // kernel is placed in queue and will start after kernel1 finishes, CPU continues to next statement
cudaMemcpy(...); // CPU blocks until memory is copied, memory copy starts only after kernel2 finishes

所以在您的示例中，不需要 cudaDeviceSynchronize.但是，它可能对调试有用，以检测您的哪个内核导致了错误(如果有的话).

So in your example, there is no need for cudaDeviceSynchronize. However, it might be useful for debugging to detect which of your kernel has caused an error (if there is any).

cudaDeviceSynchronize 可能会导致一些减速，但 7-12x 似乎太多了.可能是时间测量有问题，或者内核速度非常快，显式同步的开销相对于实际计算时间来说是巨大的.

cudaDeviceSynchronize may cause some slowdown, but 7-12x seems too much. Might be there is some problem with time measurement, or maybe the kernels are really fast, and the overhead of explicit synchronization is huge relative to actual computation time.

这篇关于何时调用 cudaDeviceSynchronize?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！