使用 concurrent.futures.ThreadPoolExecutor() 作为执行器:...不等待 [英] with concurrent.futures.ThreadPoolExecutor() as executor: ... does not wait

问题描述

我试图在一个类的方法中使用 ThreadPoolExecutor() 来创建一个线程池，该线程池将在同一个类中执行另一个方法.我有 with concurrent.futures.ThreadPoolExecutor()... 但是它不会等待，并且抛出一个错误，说我在with..."之后查询的字典中没有键陈述.我明白为什么会抛出错误，因为字典尚未更新，因为池中的线程没有完成执行.我知道线程没有完成执行，因为我在 ThreadPoolExecutor 中调用的方法中有一个打印(完成")，并且完成"没有打印到控制台.

I am trying to use the ThreadPoolExecutor() in a method of a class to create a pool of threads that will execute another method within the same class. I have the with concurrent.futures.ThreadPoolExecutor()... however it does not wait, and an error is thrown saying there was no key in the dictionary I query after the "with..." statement. I understand why the error is thrown because the dictionary has not been updated yet because the threads in the pool did not finish executing. I know the threads did not finish executing because I have a print("done") in the method that is called within the ThreadPoolExecutor, and "done" is not printed to the console.

我是线程的新手，所以如果有任何关于如何更好地做到这一点的建议，我们将不胜感激！

I am new to threads, so if any suggestions on how to do this better are appreciated!

    def tokenizer(self):
        all_tokens = []
        self.token_q = Queue()
        with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
            for num in range(5):
                executor.submit(self.get_tokens, num)
            executor.shutdown(wait=True)

        print("Hi")
        results = {}
        while not self.token_q.empty():
            temp_result = self.token_q.get()
            results[temp_result[1]] = temp_result[0]
            print(temp_result[1])
        for index in range(len(self.zettels)):
            for zettel in results[index]:
                all_tokens.append(zettel)
        return all_tokens

    def get_tokens(self, thread_index):
        print("!!!!!!!")
        switch = {
            0: self.zettels[:(len(self.zettels)/5)],
            1: self.zettels[(len(self.zettels)/5): (len(self.zettels)/5)*2],
            2: self.zettels[(len(self.zettels)/5)*2: (len(self.zettels)/5)*3],
            3: self.zettels[(len(self.zettels)/5)*3: (len(self.zettels)/5)*4],
            4: self.zettels[(len(self.zettels)/5)*4: (len(self.zettels)/5)*5],
        }
        new_tokens = []
        for zettel in switch.get(thread_index):
            tokens = re.split('\W+', str(zettel))
            tokens = list(filter(None, tokens))
            new_tokens.append(tokens)
        print("done")
        self.token_q.put([new_tokens, thread_index])

'''

期望在 print ("Hi") 之前看到所有 print("!!!!!!") 和 print("done") 语句) 语句.实际上显示 !!!!!!!!! 然后是 Hi，然后是结果字典的 KeyError.

Expected to see all print("!!!!!!") and print("done") statements before the print ("Hi") statement. Actually shows the !!!!!!! then the Hi, then the KeyError for the results dictionary.

推荐答案

你需要循环 concurrent.futures.as_completed() 如图 此处.它将在每个线程完成时产生值.

You need to loop over concurrent.futures.as_completed() as shown here. It will yield values as each thread completes.

这篇关于使用 concurrent.futures.ThreadPoolExecutor() 作为执行器:...不等待的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！