Python ValueError:池未在异步多处理中运行 [英] Python ValueError: Pool not running in Async Multiprocessing

问题描述

我有一个简单的代码:

path = [filepath1, filepath2, filepath3]

def umap_embedding(filepath):
    file = np.genfromtxt(filepath,delimiter=' ')
    if len(file) > 20000:
        file = file[np.random.choice(file.shape[0], 20000, replace=False), :]
    neighbors = len(file)//200

    if neighbors >= 2:
        neighbors = neighbors
    else:
        neighbors = 2

    embedder = umap.UMAP(n_neighbors=neighbors,
                         min_dist=0.1,
                         metric='correlation', n_components=2)
    embedder.fit(file)
    embedded = embedder.transform(file)
    name = 'file'
    np.savetxt(name,embedded,delimiter=",")

if __name__ == '__main__':
    p = Pool(processes = 20)
    start = time.time()
    for filepath in path:
        p.apply_async(umap_embedding, [filepath])
        p.close()
        p.join()

    print("Complete")
    end = time.time()
    print('total time (s)= ' + str(end-start))

执行时，控制台返回错误:

When I execute, the console return the error:

Traceback (most recent call last):
  File "/home/cngc3/CBC/parallel.py", line 77, in <module>
    p.apply_async(umap_embedding, [filepath])
  File "/home/cngc3/anaconda3/envs/CBC/lib/python3.6/multiprocessing/pool.py", line 355, in apply_async
    raise ValueError("Pool not running")
ValueError: Pool not running

我试图在Stackoverflow和Google上找到针对此问题的解决方案，但没有相关问题. 谢谢您的帮助.

I tried to find the solution for this problem on Stackoverflow and Google but there's no related problem. Thank you for your help.

推荐答案

p.close()和p.join()必须放在for循环之后.否则，该池将在循环的第一次迭代中关闭，并且在第二次迭代中不接受新作业.

p.close() and p.join() must be placed after the for-loop. Otherwise the pool is closed in the first iteration of the loop and doesn't accept new jobs in the second.

这篇关于Python ValueError:池未在异步多处理中运行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！