CUDA错误:在python中使用并行时发生初始化错误 [英] CUDA ERROR: initialization error when using parallel in python

问题描述

我将CUDA用于我的代码，但运行速度仍然很慢.因此，我将其更改为使用python中的多处理(pool.map)并行运行.但是我有CUDA ERROR: initialization error

I use CUDA for my code, but it still slow run. Therefore I change it to run parallel using multiprocessing (pool.map) in python. But I have CUDA ERROR: initialization error

这是功能:

def step_M(self, iter_training):
    gpe, e_tuple_list = iter_training
    g = gpe[0]
    p = gpe[1]
    em_iters = gpe[2]

    e_tuple_list = sorted(e_tuple_list, key=lambda tup: tup[0])
    data = self.X[e_tuple_list[0][0]:e_tuple_list[0][1]]
    cluster_indices = np.array(range(e_tuple_list[0][0], e_tuple_list[0][1], 1), dtype=np.int32)
    for i in range(1, len(e_tuple_list)):
        d = e_tuple_list[i]
        cluster_indices = np.concatenate((cluster_indices, np.array(range(d[0], d[1], 1), dtype=np.int32)))
        data = np.concatenate((data, self.X[d[0]:d[1]]))

    g.train_on_subset(self.X, cluster_indices, max_em_iters=em_iters)
    return g, cluster_indices, data

这里是代码调用:

pool = Pool()
iter_bic_list = pool.map(self.step_M, iter_training.items())

iter_training相同:

The iter_training same:

这是错误 您能帮我解决吗，谢谢.

And this is errors could you help me to fix.Thanks you.

推荐答案

我意识到这有点老，但遇到芹菜的情况下遇到了同样的问题:

I realize this is a bit old but I ran into the same problem, while running under celery in my case:

syncedmem.cpp:63] Check failed: error == cudaSuccess (3 vs. 0)  initialization error

从prefork切换到基于eventlet的池已解决了该问题.您的代码可以类似于以下内容进行更新:

Switching from prefork to an eventlet based pool has resolved the issue. Your code could be updated similarly to:

from eventlet import GreenPool
pool = GreenPool()
iter_bic_list = list(pool.imap(self.step_M, iter_training.items()))

这篇关于CUDA错误:在python中使用并行时发生初始化错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！