ForkJoinPool性能Java 8 vs 11 [英] ForkJoinPool performance Java 8 vs 11
问题描述
请考虑以下代码:
package com.sarvagya;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ForkJoinPool;
import java.util.stream.Collectors;
public class Streamer {
private static final int LOOP_COUNT = 2000;
public static void main(String[] args){
try{
for(int i = 0; i < LOOP_COUNT; ++i){
poolRunner();
System.out.println("done loop " + i);
try{
Thread.sleep(50L);
}
catch (Exception e){
System.out.println(e);
}
}
}
catch (ExecutionException | InterruptedException e){
System.out.println(e);
}
// Add a delay outside the loop to make sure all daemon threads are cleared before main exits.
try{
Thread.sleep(10 * 60 * 1000L);
}
catch (Exception e){
System.out.println(e);
}
}
/**
* poolRunner method.
* Assume I don't have any control over this method e.g. done by some library.
* @throws InterruptedException
* @throws ExecutionException
*/
private static void poolRunner() throws InterruptedException, ExecutionException {
ForkJoinPool pool = new ForkJoinPool();
pool.submit(() ->{
List<Integer> numbers = Arrays.asList(1,2,3,4,5,6,7,8,9,10, 11,12,14,15,16);
List<Integer> collect = numbers.stream()
.parallel()
.filter(xx -> xx > 5)
.collect(Collectors.toList());
System.out.println(collect);
}).get();
}
}
在上面的代码中，poolRunner方法正在创建ForkJoinPool并将一些任务提交给它.当使用Java 8并将LOOP_COUNT保持为2000时，我们可以看到创建的最大线程数约为3600，如下所示
无花果:分析
In above code,poolRunner method is creating a ForkJoinPool and submitting some tasks to it. When using Java 8 and keeping LOOP_COUNT as 2000, we could see max threads created was about 3600 as seen below
fig: Profiling
图:线程信息.
所有这些线程在一段时间后下降到将近10个.但是，在OpenJDK 11中，保持相同的LOOP_COUNT将产生以下错误:
All these threads goes down to almost 10 after some period of time. However, in OpenJDK 11 keeping same LOOP_COUNT is going to produce following error:
[28.822s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[28.822s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[28.822s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
Exception in thread "ForkJoinPool-509-worker-5" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:803)
at java.base/java.util.concurrent.ForkJoinPool.createWorker(ForkJoinPool.java:1329)
at java.base/java.util.concurrent.ForkJoinPool.tryAddWorker(ForkJoinPool.java:1352)
at java.base/java.util.concurrent.ForkJoinPool.signalWork(ForkJoinPool.java:1476)
at java.base/java.util.concurrent.ForkJoinPool.deregisterWorker(ForkJoinPool.java:1458)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:187)
它很快达到最大线程限制.将LOOP_COUNT保持为500可以正常工作，但是，这些线程的清除速度非常慢，并且达到了约500个线程的平稳状态.看到下面的图片:
It reaches max thread limit very soon. Keeping LOOP_COUNT to 500, works fine, however, these threads are cleared very very slowly and reaches plateau of about 500 threads. See the images below:
图:OpenJDK 11中的线程信息
fig: Thread info in OpenJDK 11
无花果:在OpenJDK 11中进行分析
fig: Profiling in OpenJDK 11
线程被 PARKED ，但是在JDK 11中是 WAIT .在Java 11中，守护进程线程的数量也应减少，但是，它的速度很慢并且不会不能按预期工作.此外，假设我无法控制poolRunner方法.考虑这种方法是由某些外部库提供的.
Threads were PARKED in JDK 8, but WAIT in JDK 11. Number of daemon threads should be reduced in Java 11 as well , however, it is slow and doesn't work as expected. Moreover, assume I don't have control over poolRunner method. Consider this method is provided by some external library.
是OpenJDK 11的问题，还是代码做错了.谢谢.
Is this issue with OpenJDK 11 or am doing something wrong in code. Thanks.
推荐答案
您的代码正在创建大量的ForkJoinPool实例，并且在使用任何池之后都不会在任何池上调用shutdown().由于在Java 8的情况下，规范中没有任何内容可以保证工作线程将终止，因此该代码甚至可能以2000(> 池数⟩)乘以⟨核数⟩个线程.
Your code is creating a huge amount of ForkJoinPool instances and is never calling shutdown() on any pool after its use. Since in case of Java 8, nothing in the specification guarantees that the worker threads will terminate, this code could even end up with 2000 (⟨number of pools⟩) times ⟨number of cores⟩ threads.
在实践中，观察到的行为源自未记录的空闲超时 两秒钟.请注意，根据评论，超时的结果是尝试减少工作人员的人数，这与终止工作不同.因此，如果 n 线程遇到超时，则并非所有 n 线程都终止，但是线程数减少了一个，其余线程可能会再次等待.此外，短语初始超时值"已经暗示了它，实际超时每次发生时都会增加.因此，由于此(未记录)超时， n 空闲工作线程需要花费n * (n + 1)秒的时间.
In practice, the observed behavior stems from an undocumented idle timeout of two seconds. Note that according to the comment, the consequence of an elapsed timeout is an attempt to shrink the number of workers which is different to just terminating. So if n thread experience the timeout, not all n threads terminate but the number of threads gets reduced by one and the remaining threads may wait again. Further, the phrase "initial timeout value" already hints at it, the actual timeout gets incremented each time it happens. So it takes n * (n + 1) seconds for n idle worker thread to terminate due to this (undocumented) timeout.
从Java 9开始，有一个可配置的 keepAliveTime ，可以在
Starting with Java 9, there is a configurable keepAliveTime which can be specified in a new constructor of ForkJoinPool, which also documents the default value:
keepAliveTime- 自上次使用以来，终止线程之前经过的时间(然后如果需要，可在以后替换).对于默认值，请使用
60, TimeUnit.SECONDS.
该文档可能会误以为现在所有工作线程可能在 keepAliveTime 空闲时可能一起终止，但实际上，仍然存在一次仅将池缩小一个的行为.现在，时间没有增加.因此，现在， n 空闲工作线程终止最多需要60 * n秒.由于以前的行为未指定，因此甚至不兼容.
This documentation may mislead into thinking that now all worker thread may terminate together when being idle for keepAliveTime, but in fact, there’s still a behavior of only shrinking the pool by one at a time, though now, the time is not increasing. So now, it takes up to 60 * n seconds for n idle worker thread to terminate. Since the previous behavior was unspecified, it’s not even an incompatibility.
必须强调的是，即使具有相同的超时行为，最终产生的最大线程数也可能发生变化，因为具有更好代码优化的更新JVM会减少实际操作的执行时间(无需人工插入Thread.sleep(…))这样一来，在终止仍与壁钟时间绑定的同时，它可以更快地创建新线程.
It must be emphasized that even with the same timeout behavior, the resulting maximum number of threads could change, as when a newer JVM with better code optimizations reduces the execution time of the actual operations (without artificial insertions of Thread.sleep(…)) it would create new threads faster while the termination still is bound to wall-clock time.
得出的结论是，当您知道不再需要线程池时，就永远不要依赖于自动辅助线程终止.相反，您应该在完成后致电shutdown().
The takeaway is that you should never rely on the automatic worker thread termination when you know that a thread pool is not needed anymore. Instead, you should call shutdown() when you are done.
您可以使用以下代码验证行为:
You may verify the behavior with the following code:
int threadNumber = 8;
ForkJoinPool pool = new ForkJoinPool(threadNumber);
// force the creation of all worker threads
pool.invokeAll(Collections.nCopies(threadNumber*2, () -> { Thread.sleep(500); return ""; }));
int oldNum = pool.getPoolSize();
System.out.println(oldNum+" threads; waiting for dying threads");
long t0 = System.nanoTime();
while(oldNum > 0) {
while(pool.getPoolSize()==oldNum)
LockSupport.parkNanos(TimeUnit.MILLISECONDS.toNanos(200));
long t1 = System.nanoTime();
oldNum = pool.getPoolSize();
System.out.println(threadNumber-oldNum+" threads terminated after "
+TimeUnit.NANOSECONDS.toSeconds(t1 - t0)+"s");
}
Java 8:
Java 8:
8 threads; waiting for dying threads
1 threads terminated after 2s
2 threads terminated after 6s
3 threads terminated after 12s
4 threads terminated after 20s
5 threads terminated after 30s
6 threads terminated after 42s
7 threads terminated after 56s
8 threads terminated after 72s
Java 11:
Java 11:
8 threads; waiting for dying threads
1 threads terminated after 60s
2 threads terminated after 120s
3 threads terminated after 180s
4 threads terminated after 240s
5 threads terminated after 300s
6 threads terminated after 360s
7 threads terminated after 420s
很显然，至少没有最后一个工作线程保持活动状态
这篇关于ForkJoinPool性能Java 8 vs 11的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
