Python多处理:比请求更多的进程 [英] Python multiprocessing: more processes than requested

问题描述

为什么当我仅使用1个内核时，为什么在同一个脚本中(在RHEL 6的htop中)运行着这么多的python进程?

Why do I see so many python processes running (in htop on RHEL 6) for the same script when I only use 1 core?

对于每个任务，我都会初始化一个用于处理过程的worker类.它会初始化其他类，但不会初始化任何子进程:

For each task, I init a worker class that manages the processing. It does init other classes, but not any subprocesses:

    tasks = multiprocessing.JoinableQueue()
    results = multiprocessing.Queue()

    num_consumers = 1

    consumers = [Consumer(tasks, results) for i in xrange(num_consumers)]
    for i, consumer in enumerate(consumers):
        logger.debug('Starting consumer %s (%i/%i)' % (consumer.name, i + 1, num_consumers))
        consumer.start()

请注意，atop显示了预期的进程数(在这种情况下，为2:父进程为1，子进程为1). ％MEM通常加起来超过100％，所以我误会了多处理或htop的工作原理.

Note, atop shows the expected number of processes (in this case 2: 1 for the parent and 1 for the child). The %MEM often adds up to well over 100% so I gather I'm misunderstanding how multiprocessing or htop works.

推荐答案

我相信您会看到应用程序的主pid中的多处理"模块启动了辅助线程.这些是您明确创建的线程/进程的补充.

I believe you're seeing helper threads spun up by the Multiprocessing module within the main pid from your app. These are in addition to the Threads/Processes you've spun up explicitly.

这篇关于Python多处理:比请求更多的进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！