我如何“多过程"处理? itertools产品模块? [英] How do I "multi-process" the itertools product module?

问题描述

因此，我尝试计算以下字符串的数百万种不同组合，但我每秒仅计算大约1,750种组合，这甚至还远没有达到我所需要的速度.那么，我将如何重塑这种形状，以使同一件事情的多个过程正在计算不同的零件，而不是计算已经计算出的零件并保持快速的速度呢?下面的代码部分是我一直在使用的代码.任何示例将不胜感激！

So I tried I tried calculating millions and millions of different combinations of the below string but I was only calculating roughly 1,750 combinations a second which isn't even near the speed I need. So how would I reshape this so multiple processes of the same thing are calculating different parts, while not calculating parts that have already been calculated and also maintaining fast speeds? The code below is partially what I've been using. Any examples would be appreciated!

from itertools import product
for chars in product("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ12234567890!@#$%^&*?,()-=+[]/;", repeat = 4):
   print chars

推荐答案

将产品分解成一个部分的一种方法是分解产品的第一个组成部分，以便每个独立的工作都具有从特定元素开始的所有元素.第一组字母.例如:

One way to break the product up into parts is to break up the first component of the product, so that each independent job has all the elements starting with a certain set of first letters. For example:

import string
import multiprocessing as mp
import itertools

alphabet = string.ascii_letters+string.digits+"!@#$%^&*?,()-=+[]/;"
num_parts = 4
part_size = len(alphabet) // num_parts

def do_job(first_bits):
    for x in itertools.product(first_bits, alphabet, alphabet, alphabet):
        print(x)

if __name__ == "__main__":
    pool = mp.Pool()
    results = []
    for i in xrange(num_parts):
        if i == num_parts - 1:
            first_bit = alphabet[part_size * i :]
        else:
            first_bit = alphabet[part_size * i : part_size * (i+1)]
        results.append(pool.apply_async(do_job(first_bit)))

    pool.close()
    pool.join()

(显然，如果do_job实际上返回了某些内容，您只会使用results).

(where obviously you'd only use results if do_job actually returned something).

这篇关于我如何“多过程"处理? itertools产品模块?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！