我应该使用Java中的哪个ThreadPool？ [英] Which ThreadPool in Java should I use?

问题描述

有大量的任务。
每项任务都属于一个组。要求是每组任务应该像在单个线程中执行一样串行执行，并且吞吐量应该在多核（或多CPU）环境中最大化。注意：还有大量的组与任务数量成正比。

There are a huge amount of tasks. Each task is belong to a single group. The requirement is each group of tasks should executed serially just like executed in a single thread and the throughput should be maximized in a multi-core (or multi-cpu) environment. Note: there are also a huge amount of groups that is proportional to the number of tasks.

天真的解决方案是使用ThreadPoolExecutor并同步（或锁定）。但是，线程会相互阻塞，并且吞吐量不会最大化。

The naive solution is using ThreadPoolExecutor and synchronize (or lock). However, threads would block each other and the throughput is not maximized.

有什么好主意吗？或者是否存在满足要求的第三方库？

Any better idea? Or is there exist a third party library satisfy the requirement?

推荐答案

一种简单的方法是将所有组任务连接到一个超级任务，从而使子任务串行运行。但这可能会导致其他组无法启动的延迟，除非其他组完全完成并在线程池中占用一些空间。

A simple approach would be to "concatenate" all group tasks into one super task, thus making the sub-tasks run serially. But this will probably cause delay in other groups that will not start unless some other group completely finishes and makes some space in the thread pool.

作为替代方案，请考虑链接小组的任务。以下代码说明了这一点：

As an alternative, consider chaining a group's tasks. The following code illustrates it:

public class MultiSerialExecutor {
    private final ExecutorService executor;

    public MultiSerialExecutor(int maxNumThreads) {
        executor = Executors.newFixedThreadPool(maxNumThreads);
    }

    public void addTaskSequence(List<Runnable> tasks) {
        executor.execute(new TaskChain(tasks));
    }

    private void shutdown() {
        executor.shutdown();
    }

    private class TaskChain implements Runnable {
        private List<Runnable> seq;
        private int ind;

        public TaskChain(List<Runnable> seq) {
            this.seq = seq;
        }

        @Override
        public void run() {
            seq.get(ind++).run(); //NOTE: No special error handling
            if (ind < seq.size())
                executor.execute(this);
        }       
    }

优点是没有额外的资源（线程/队列）正在使用，并且任务的粒度优于天真方法中的任务。缺点是所有小组的任务应事先知道。

The advantage is that no extra resource (thread/queue) is being used, and that the granularity of tasks is better than the one in the naive approach. The disadvantage is that all group's tasks should be known in advance.

- 编辑 -

要使此解决方案通用且完整，您可能需要决定错误处理（即，即使发生错误，链是否继续），并且实现ExecutorService并委派所有调用也是一个好主意到底层执行者。

To make this solution generic and complete, you may want to decide on error handling (i.e whether a chain continues even if an error occures), and also it would be a good idea to implement ExecutorService, and delegate all calls to the underlying executor.

这篇关于我应该使用Java中的哪个ThreadPool？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！