Java ExecutorService-有时比顺序处理慢吗? [英] Java ExecutorService - sometimes slower than sequential processing?

问题描述

我正在编写一个简单的实用程序，该实用程序接受一组Callable任务，并并行运行它们.希望总花费的时间很少，而不是最长的任务所花费的时间.该实用程序还添加了一些错误处理逻辑-如果任何任务失败，并且该失败可以被视为可重试"(例如超时或用户指定的异常)，则我们直接运行该任务. /p>

我已经在ExecutorService周围实现了该实用程序.有两个部分:

1. 将所有Callable任务提交给ExecutorService，存储Future对象.
2. 在for循环中，get()每个Future的结果.如果有例外，请执行可重试"逻辑.

我编写了一些单元测试，以确保使用此实用程序比按顺序运行任务更快.对于每个测试，我都会生成一定数量的Callable，每个Callable本质上都在一定范围内随机执行Thread.sleep().我尝试了不同的超时，不同数量的任务等，该实用程序的性能似乎优于顺序执行.

但是，当我将其添加到需要这种实用程序的实际系统中时，我看到的结果变化很大-有时并行执行更快，有时更慢，有时更快，但仍然花费很多比最长的个人任务要花更多的时间.

我只是做错了吗?我知道ExecutorService具有invokeAll()，但是吞没了基础异常.我还尝试使用CompletionService按任务完成的顺序来获取任务结果，但是它表现出的行为或多或少相同.我现在正在阅读有关闩锁和障碍的信息-这是解决此问题的正确方向吗?

解决方案

多线程编程不是免费提供的.它有开销.开销很容易超过性能提升，通常会使您的代码更复杂.

其他线程可以访问更多的cpu功能(假设您有备用的cpus)，但是通常它们不会使HDD旋转得更快，为您提供更多的网络带宽或加速不受cpu约束的速度.

多个线程可以帮助您获得更多的外部资源份额.

I'm writing a simple utility which accepts a collection of Callable tasks, and runs them in parallel. The hope is that the total time taken is little over the time taken by the longest task. The utility also adds some error handling logic - if any task fails, and the failure is something that can be treated as "retry-able" (e.g. a timeout, or a user-specified exception), then we run the task directly.

I've implemented this utility around an ExecutorService. There are two parts:

1. submit() all the Callable tasks to the ExecutorService, storing the Future objects.
2. in a for-loop, get() the result of each Future. In case of exceptions, do the "retry-able" logic.

I wrote some unit tests to ensure that using this utility is faster than running the tasks in sequence. For each test, I'd generate a certain number of Callable's, each essentially performing a Thread.sleep() for a random amount of time within a bound. I experimented with different timeouts, different number of tasks, etc. and the utility seemed to outperform sequential execution.

But when I added it to the actual system which needs this kind of utility, I saw results that were very variable - sometimes the parallel execution was faster, sometimes it was slower, and sometimes it was faster, but still took a lot more time than the longest individual task.

Am I just doing it all wrong? I know ExecutorService has invokeAll() but that swallows the underlying exceptions. I also tried using a CompletionService to fetch task results in the order in which they completed, but it exhibited more or less the same behavior. I'm reading up now on latches and barriers - is this the right direction for solving this problem?

解决方案

Multi-threaded programming doesn't come for free. It has an overhead. The over head can easily exceed and performance gain and usually makes your code more complex.

Additional threads give access to more cpu power (assuming you have spare cpus) but in general they won't make you HDD spin faster , give you more network bandwidth or speed up something which is not cpu bound.

Multiple threads can help give you a greater share of an external resource.

这篇关于Java ExecutorService-有时比顺序处理慢吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！