为什么Java编译上的并行执行会随着时间线性增长 [英] why parallel execution on java compile take linear growth in time

问题描述

time javac Main.java                                      --> 0m1.050s
time javac Main.java & javac Main.java                    --> 0m1.808s
time javac Main.java & javac Main.java & javac Main.java  --> 0m2.690s
time javac Main.java & ... 8 time                         --> 0m8.309s

当我们并行运行javac命令并且每次javac命令增加时，都会为所有javac命令添加~1 sec而添加.

When we run javac command in parallel and with each increase in javac command ~1 sec gets added for all the javac command to complete.

为什么时间线性增长?

在locks上以某种方式涉及所有的javac进程，如果可以的话，如何克服它，以免时间线性增长

Is all javac process while running involved in some kind on locks, if yes how to overcome it so as not to have a linear growth in time

PS:我曾在single core machine，double core machine，4 core machine上尝试过所有这些，它们都表现出相同的行为.

PS: I have tried above on single core machine, double core machine, 4 core machine all showed same behaviour.

PS2:环境RedHat7，javac 1.7.0_79

推荐答案

即使仅编译单个文件，java编译器也已经可以在可用处理器之间进行工作划分.因此，您自己并行运行单独的编译器实例将不会产生预期的性能提升.

The java compiler already handles dividing its work across available processors, even when only compiling a single file. Therefore running separate compiler instances in parallel yourself won't yield the performance gains you are expecting.

为了证明这一点，我在一个名为Main1.java的文件中生成了一个大型程序(一百万行，一万种方法).然后，将其他副本制作为Main2.java至Main8.java.编译时间如下:

To demonstrate this, I generated a large (1 million lines, 10,000 methods) java program in a single file called Main1.java. Then made additional copies as Main2.java through Main8.java. Compile times are as follows:

单个文件编译:

time javac Main1.java &    --> (real) 11.6 sec

观看在top中编译的单个文件，可以发现处理器使用率大多在200-400％的范围内(指示多个CPU使用率，每个CPU 100％)，偶尔出现700％的峰值(这台机器的最大值是800％，因为有8个处理器.

Watching this single file compile in top revealed processor usage mostly in the 200-400% range (indicating multiple CPU usage, 100% per CPU), with occasional spikes in the 700% range (the max on this machine is 800% since there are 8 processors).

接下来，同时显示两个文件:

Next, two files simultaneously:

time javac Main1.java &    --> (real) 14.5 sec
time javac Main2.java &    --> (real) 14.8 sec

所以编译两个只花了14.8秒，而编译一个只花了11.6秒.那绝对是非线性的.通过查看top可以清楚地看到它们正在运行时，每个Java编译器一次最多只能利用四个CPU(偶尔会有更高的峰值).因此，这两个编译器在八个CPU之间运行，这些CPU大多彼此并行.

So it only took 14.8 seconds to compile two, when it took 11.6 seconds to compile one. That's definitely non-linear. It was clear by looking at top while these were running that again each java compiler was only taking advantage of at most four CPUs at once (with occasional spikes higher). Because of this, the two compilers ran across eight CPUs mostly in parallel with each other.

接下来，同时显示四个文件:

Next, four files simultaneously:

time javac Main1.java &    --> (real) 24.2 sec
time javac Main2.java &    --> (real) 24.6 sec
time javac Main3.java &    --> (real) 25.0 sec
time javac Main4.java &    --> (real) 25.0 sec

好的，在这里，我们碰壁了.我们再也无法使编译器并行化.四个文件用时25秒，而两个文件用时14.8.那里有一些优化，但主要是线性的时间增加.

Okay, here we've hit the wall. We can no longer out-parallelize the compiler. Four files took 25 seconds when two took 14.8. There's a little optimization there but it's mostly a linear time increase.

最后，同时八个​​:

time javac Main1.java &    --> (real) 51.9 sec
time javac Main2.java &    --> (real) 52.3 sec
time javac Main3.java &    --> (real) 52.5 sec
time javac Main4.java &    --> (real) 53.0 sec
time javac Main5.java &    --> (real) 53.4 sec
time javac Main6.java &    --> (real) 53.5 sec
time javac Main7.java &    --> (real) 53.6 sec
time javac Main8.java &    --> (real) 54.6 sec

这实际上比线性情况差一点，因为八个花费了54.6秒，而四个仅仅花费了25.0.

This was actually a little worse than linear, as eight took 54.6 seconds while four only took 25.0.

因此，我认为所有这一切的收获是要相信编译器将在优化可用CPU资源上的工作方面做得不错，并且尝试手动添加其他并行化将受到限制(如果有的话).

So I think the takeaway from all this is to have faith that the compiler will do a decent job trying to optimize the work you give it across the available CPU resources, and that trying to add additional parallelization by hand will have limited (if any) benefit.

作为参考，我在Oracle的错误数据库中找到了两个条目，它们与增强javac以利用多个处理器的优势有关:

For reference, there are two entries I found in Oracle's bug database regarding enhancing javac to take advantage of multiple processors:

• 错误ID: JDK-6629150 -原始投诉，最终被标记为以下内容的重复副本:
• 错误ID:JDK-6713663 -根据"已解决的日期"似乎是在2008-06-12上添加了javac中的多处理器支持.

• Bug ID: JDK-6629150 -- The original complaint, this was eventually marked as a duplicate of:
• Bug ID: JDK-6713663 -- Suggests the resolution, and based on the "Resolved Date" it appears that multi-processor support in javac was added on 2008-06-12.

这篇关于为什么Java编译上的并行执行会随着时间线性增长的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！