如何在python中使用线程 [英] How to use threads in python

问题描述

我想用它来读取17770个文件中的值，并将所有这些值最后添加到一个字典对象中.我有一台8核的机器.

I want to use it for reading values in 17770 files and to add all of them at the end to one dictionary object. I have a machine with 8 cores.

这是代码

def run_item_preprocess ():
    scores = {};
    for i in range(1,17771):
        filename1 = "x_" + str(i) + ".txt";
        lines1 = open(filename1).readlines();
        Item1 = {};
        for line in lines1:
            tokens = line.split(',');
            Item1[int(tokens[1])] = int(tokens[2]);
        for j in range(1,17771):
            if j == i:
                continue;
            filename2 = "x_" + str(i) + ".txt";
            lines2 = open(filename2).readlines();
            Item2 = {};
            for line in lines2:
                tokens = line.split(',');
                u = int(tokens[1]);
                r = int(tokens[2]);
                if u in Item1:
                    Item2[u] = r;
            if i not in scores:
                scores[i] = {};
            scores[i]= (s(Item1,Item2),j);

推荐答案

这是很棒的多重处理模块.它使您可以使用进程而不是线程来并行化代码.这将使用所有内核.

Here is the wonderful multiprocessing module. It lets you parallelise code, using processes not threads. This will use all cores.

一个重要的区别是进程不共享内存.队列将有助于减少步骤.

An important difference is that processes don't share memory; a queue will help with the reduce step.

这篇关于如何在python中使用线程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！