多处理for循环? [英] Multiprocessing a for loop?

问题描述

我有一个数组(称为data_inputs)，其中包含数百个天文学图像文件的名称.这些图像然后被处理.我的代码有效，并且需要几秒钟来处理每个图像.但是，一次只能做一张图像，因为我是通过for循环运行数组的:

I have an array (called data_inputs) containing the names of hundreds of astronomy images files. These images are then manipulated. My code works and takes a few seconds to process each image. However, it can only do one image at a time because I'm running the array through a for loop:

for name in data_inputs:
    sci=fits.open(name+'.fits')
    #image is manipulated

没有理由我必须先修改映像，那么是否有可能利用我机器上的所有4个内核，而每个内核都通过for循环在不同的映像上运行?

There is no reason why I have to modify an image before any other, so is it possible to utilise all 4 cores on my machine with each core running through the for loop on a different image?

我已经阅读了有关multiprocessing模块的信息，但是我不确定如何在我的情况下实现它. 我渴望使multiprocessing能够正常工作，因为最终我将不得不在10,000多个图像上运行它.

I've read about the multiprocessing module but I'm unsure how to implement it in my case. I'm keen to get multiprocessing to work because eventually I'll have to run this on 10,000+ images.

推荐答案

您可以简单地使用 multiprocessing.Pool :

You can simply use multiprocessing.Pool:

from multiprocessing import Pool

def process_image(name):
    sci=fits.open('{}.fits'.format(name))
    <process>

if __name__ == '__main__':
    pool = Pool()                         # Create a multiprocessing Pool
    pool.map(process_image, data_inputs)  # process data_inputs iterable with pool

这篇关于多处理for循环?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！