我如何在多进程中启动每个工作进程?在新的Shell中合并? [英] How can l launch each worker in a multiprocessing.Pool in a new shell?

问题描述

我正在尝试将工作池中生成的进程的执行耦合到新的系统终端.在以下示例中(改编自@ sylvain-leroux对这个问题的回答问题)，我们构造了一个工作人员池以处理排队的对象.

I'm trying to couple the execution of a spawned process in a worker pool to a new system terminal. In the following example (adapted from @sylvain-leroux's answer to this question) a pool of workers is constructed to do some work with queued objects.

import os
import time
import multiprocessing

# A main function, to be run by our workers.
def worker_main(queue):
    print('The worker at', os.getpid(), 'is initialized.')
    while True:

        # Block until something is in the queue.
        item = queue.get(True)
        print(item)
        time.sleep(0.5)

if __name__ == '__main__':

    # Instantiate a Queue for communication.
    the_queue = multiprocessing.Queue()

    # Build a Pool of workers, each running worker_main.
    the_pool = multiprocessing.Pool(3, worker_main, (the_queue,))

    # Iterate, sending data via the Queue.
    for i in range(5):
        the_queue.put("That's a nice string you got there.")
        the_queue.put("It'd be a shame if something were to... garble it.")

    worker_pool.close()
    worker_pool.join()
    time.sleep(10)

如果从系统终端运行此命令，则会看到一堆乱码，因为每个工作人员都在同一个控制台中写信并在同一控制台中执行.对于我正在处理的项目，生成一个新的Shell/控制台以承载每个工作进程会很有帮助，这样所有打印输出都将显示在该Shell中，而工作进程的执行将在该Shell中进行.我已经看到了几个使用shell关键字使用Popen进行此操作的示例，但是由于兼容性限制，我需要坚持基于池的实现.外面有没有人这样做?指导表示赞赏.

If you run this from a system terminal you'll see a bunch of garbled text, because each of the workers is writing out to, and executing in, the same console. For an project I'm working on, it would be helpful to spawn a new shell/console to host each worker process, such that all printed output is displayed in that shell, and the execution of the worker process is host in that shell. I've seen several examples doing this with Popen using the shell keyword, but I need to stick to a pool-based implementation, due to compatibility constraints. Has anyone out there done this? Guidance is appreciated.

推荐答案

尝试以另一种方式使用Queue.

Try using the Queue the other way around.

让工人put消息放入Queue，并在父进程中将get消息从Queue进行打印.那应该消除混杂的输出.

Let the workers put messages into the Queue, and in the parent process get them from the Queue and print them. That should get rid of intermingled output.

如果您希望将消息从父级传递到工人，然后再传递回，请使用两个队列.一种用于将消息传递给工人，另一种用于将消息传递给父母.

If you want to pass messages both from parent to workers and back, use two queues. One for passing messages to workers, and one to pass messages back to the parent.

这篇关于我如何在多进程中启动每个工作进程?在新的Shell中合并?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！