Python 2.7多处理无需使用缓冲池即可获得处理结果 [英] Python 2.7 multiprocessing get process result whithout using pool

问题描述

如何在不使用池的情况下从过程中获取结果?

How can I get the result from my process without using a pool ?

(我愿意关注进展情况:

(I'm willing to conserve an eye on the progression:

(print "\r",float(done)/total,"%",)

据我所知，这是无法使用游泳池完成的事情

which can't be done using a pool as far I know)

def multiprocess(function, argslist, ncpu):
    total = len(argslist)
    done = 0
    jobs = []
    while argslist != []:
        if len(mp.active_children()) < ncpu:
            p = mp.Process(target=function,args=(argslist.pop(),))
            jobs.append(p)
            p.start()
            done+=1
            print "\r",float(done)/total,"%",
    #get results here
    for job in jobs:
        job.get_my_result()???

这些过程确实很短(<0.5秒)，但是我大约有100万.

The processes are really short (<0.5 seconds) but I have around 1 million of them.

我看到了该线程我可以从中获取返回值吗? multiprocessing.Process?我试图重现它，但无法使其正常工作.

I saw this thread Can I get a return value from multiprocessing.Process? I tried to reproduce it but I couldn't make it work properly.

您可以随时获取任何进一步的信息.

At your entire disposal for any further information.

推荐答案

此问题可以被视为重复问题，但无论如何，这是我的问题的解决方案:

This question may be considered as a duplicate but anyway here is the solution to my problem:

def multiprocess(function, argslist, ncpu):
    total = len(argslist)
    done = 0
    result_queue = mp.Queue()
    jobs = []
    while argslist != [] and done<10 :
        if len(mp.active_children()) < ncpu:
            p = mp.Process(target=function,args=(result_queue, argslist.pop(),))
            jobs.append(p)
            p.start()
            done+=1
            print "\r",float(done)/total,"%",
    #get results here
    res = [result_queue.get() for p in jobs]
    print res

我也必须更改

return function_result

进入

result_queue.put(function_result)

这篇关于Python 2.7多处理无需使用缓冲池即可获得处理结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！