处理具有多个线程的"N"个项目的列表 [英] Process list of 'N' items with multiple threads

问题描述

我有List个N项，我想按顺序将这个List除以固定数量的threads.

I have List of N items and I want to divide this List in a sequential manner between a fixed number of threads.

按顺序，我的意思是，我想将1 to N/4传递给第一个thread，将N/4 + 1 to N/2传递给第二个线程，将N/2+1 to N传递给第三个thread，现在所有threads完成工作后，我想要通知主thread发送一些消息，说明所有处理已完成.

By sequential I mean, I want to pass 1 to N/4to first thread , N/4 + 1 to N/2to second thread and N/2+1 to N to third thread, Now once all the threads have finished their work, I want to notify to main thread to send some message that all the processing has been completed.

到目前为止，我已经实现了ExecutorService

What I have done so far now is that I have implemented ExecutorService

我做了这样的事情

ExecutorService threadPool = Executors.newFixedThreadPool(Number_of_threads); 
                //List of items List
                List <items>itemList = getList(); 
                  for (int i = 0 i < Number_of_threads ;i++ ) { 
                    //how to divide list here sequentially and pass it to some processor while will process those items.
                    Runnable processor = new Processor(Start, End)
                    executor.execute(process);
                   }
                  if(executor.isTerminated()){
                    logger.info("All threads completed");
                  }

• 如何将列表分成连续的块?
• 有没有更好的方法来实现这种功能?

推荐答案

如果要使所有线程尽可能快地完成处理，并且项数不是 huge ，则只需发布每个项目一个Runnable到一个newFixedThreadPool(NUMBER_OF_THREADS):

If what you want is to make all threads finish processing as fast as possible and the number of items is not huge then just post one Runnable per item into a newFixedThreadPool(NUMBER_OF_THREADS):

    ExecutorService exec = Executors.newFixedThreadPool(NUMBER_OF_THREADS);
    List<Future<?>> futures = new ArrayList<Future<?>>(NUMBER_OF_ITEMS);
    for (Item item : getItems()) {
        futures.add(exec.submit(new Processor(item)));
    }
    for (Future<?> f : futures) {
        f.get(); // wait for a processor to complete
    }
    logger.info("all items processed");

如果您真的要给每个线程一个连续的列表部分(但仍然希望它们尽可能快地完成，并且还希望处理每个项目花费的时间大约相同) )，然后尽可能按均匀"方式拆分项目，以使每个线程的最大项目数与最小数量的差异不超过一个(例如:14个项目，4个线程，那么您希望进行拆分)为[4,4,3,3]，而不是[3,3,3,5]).为此，您的代码应为

If you really want to give each thread a continuous portion of the list (but still want them to finish as fast as possible, and also expect that processing each item takes approximately the same amount of time), then split the items as "evenly" as you can so that the maximum number of items per thread differed from the minimum number by no more than one (example: 14 items, 4 threads, then you want the splitting to be [4,4,3,3], not e.g. [3,3,3,5]). For that, your code would be e.g.

    ExecutorService exec = Executors.newFixedThreadPool(NUMBER_OF_THREADS);
    List<Item> items = getItems();
    int minItemsPerThread = NUMBER_OF_ITEMS / NUMBER_OF_THREADS;
    int maxItemsPerThread = minItemsPerThread + 1;
    int threadsWithMaxItems = NUMBER_OF_ITEMS - NUMBER_OF_THREADS * minItemsPerThread;
    int start = 0;
    List<Future<?>> futures = new ArrayList<Future<?>>(NUMBER_OF_ITEMS);
    for (int i = 0; i < NUMBER_OF_THREADS; i++) {
        int itemsCount = (i < threadsWithMaxItems ? maxItemsPerThread : minItemsPerThread);
        int end = start + itemsCount;
        Runnable r = new Processor(items.subList(start, end));
        futures.add(exec.submit(r));
        start = end;
    }
    for (Future<?> f : futures) {
        f.get();
    }
    logger.info("all items processed");

这篇关于处理具有多个线程的"N"个项目的列表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！