如何在Python中并行处理列表? [英] How to process a list in parallel in Python?

问题描述

我写了这样的代码:

def process(data):
   #create file using data

all = ["data1", "data2", "data3"]

我想在所有列表上并行执行处理功能，因为它们正在创建较小的文件，所以我不必担心磁盘写入，但是处理时间很长，所以我想使用我的所有内核.

I want to execute process function on my all list in parallel, because they are creating small files so I am not concerned about disk write but the processing takes long, so I want to use all of my cores.

如何使用python 2.7中的默认模块来做到这一点?

How can I do this using default modules in python 2.7?

推荐答案

您可以尝试以下基本示例:

You may try a basic example like:

from threading import Thread

def process(data):
    print "processing %s" % data

all = ["data1", "data2", "data3"]

for task in all:
    t = Thread(target=process, args=(task,))
    t.start()

这是 repl 和

Here's a repl and an brief tutorial which shows how to let your caller pause for the threads to join if desired.

关于使用所有内核，我没有任何信息，但是这里有一些资源可能会有所帮助:[ 3 ]

In regards to using all your cores, I don't have any information on that, but here are some resources that might be helpful: [1], [2], [3]

这篇关于如何在Python中并行处理列表?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！