Python进程未加入 [英] Python Processes not joining

问题描述

我无法粘贴我的代码片段，所以我会解释这个场景1. 我有一份大约 50000 只需要估值的股票清单2. 这些股票在 MultiProcess.Queue 中3.我创建了多个进程，每个进程会从队列中取出一批50个做一些事情.4.在主线程中，我有一个看起来像这样的检查

I'm unable to paste my code snippet, so I'll explain the scenario 1. I have a list of around 50000 stocks that need some valuation 2. These stocks are in a MultiProcess.Queue 3. I create multiple processes, each process will take a batch of 50 from the queue and do something. 4. in the main thread, I have a check which looks like this

 anymore_to_process = True

 while anymore_to_process:
     if (stock_queue.qsize() == 0):
         anymore_to_process = False

for jobs in stock_jobs:
    jobs.join()

1. 但是，当我处理 50000 条记录时，这似乎不起作用.如果我处理 500 只股票，这可以正常工作.

我做错了什么?为什么我处理很多股票时进程没有加入.

what am I doing wrong? Why are the processes not joining when I process a lot of stocks.

我知道如果不看我的代码就很难回答这个问题.....但是如果你能给我一些指点，那将非常有帮助.

I know this is difficult to answer without looking at my code.....but if you can give me some pointers, that would be very helpful.

推荐答案

问题已解决 - 从 @dano 的问题中得到了提示.我确实在写另一个队列，它阻塞了进程.我把它拿出来，问题解决了.

Problem resolved - took a cue from @dano's question. I was indeed writing to another queue, which was blocking the processes. I took it out and problem resolved.

这篇关于Python进程未加入的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！