使用 Scala 并行集合和 setParallelism 的奇怪行为 [英] Strange Behaviour Using Scala Parallel Collections and setParallelism
问题描述
我最近在 Scala 2.9 中发现了 Parallel Collection，并且很高兴看到可以使用 collection.parallel.ForkJoinTasks.defaultForkJoinPool.setParallelism 设置并行度.
I recently found out about Parallel Collection in Scala 2.9 and was excited to see that the degree of parallelism can be set using collection.parallel.ForkJoinTasks.defaultForkJoinPool.setParallelism.
然而，当我尝试将两个大小分别为 100 万的向量相加的实验时，我发现
However when I tried an experiment of adding two vectors of size one million each , I find
- 使用并行度设置为 64 的并行收集与顺序收集一样快(显示在结果中).
- 增加 setParallelism 似乎以非线性方式提高了性能.我至少会有预期的单调行为(也就是说，如果我增加并行度，性能应该不会降低)
有人可以解释为什么会发生这种情况
Can some one explain why this is happening
object examplePar extends App{
val Rnd = new Random()
val numSims = 1
val x = for(j <- 1 to 1000000) yield Rnd.nextDouble()
val y = for(j <- 1 to 1000000) yield Rnd.nextDouble()
val parInt = List(1,2,4,8,16,32,64,128,256)
var avg:Double = 0.0
var currTime:Long = 0
for(j <- parInt){
collection.parallel.ForkJoinTasks.defaultForkJoinPool.setParallelism(j)
avg = 0.0
for (k <- 1 to numSims){
currTime = System.currentTimeMillis()
(x zip y).par.map(x => x._1 + x._2)
avg += (System.currentTimeMillis() - currTime)
}
println("Average Time to execute with Parallelism set to " + j.toString + " = "+ (avg/numSims).toString + "ms")
}
currTime = System.currentTimeMillis()
(x zip y).map(x => x._1 + x._2)
println("Time to execute using Sequential = " + (System.currentTimeMillis() - currTime).toString + "ms")
}
使用 Scala 2.9.1 和四核处理器运行示例的结果是
The results on running the example using Scala 2.9.1 and a four core processor is
Average Time to execute with Parallelism set to 1 = 1047.0ms
Average Time to execute with Parallelism set to 2 = 594.0ms
Average Time to execute with Parallelism set to 4 = 672.0ms
Average Time to execute with Parallelism set to 8 = 343.0ms
Average Time to execute with Parallelism set to 16 = 375.0ms
Average Time to execute with Parallelism set to 32 = 391.0ms
Average Time to execute with Parallelism set to 64 = 406.0ms
Average Time to execute with Parallelism set to 128 = 813.0ms
Average Time to execute with Parallelism set to 256 = 469.0ms
Time to execute using Sequential = 406ms
虽然这些结果是针对一次运行的，但在多次运行中取平均值时它们是一致的
Though these results are for one run, they are consistent when averaged over more runs
推荐答案
并行不是免费的.需要额外的周期才能将问题拆分为更小的块、组织所有内容并同步结果.
Parallelism does not come free. It requires extra cycles to split the problem into smaller chunks, organize everything, and synchronize the result.
你可以把这想象成打电话给你所有的朋友帮你搬家，等他们到那里，帮你装卡车，然后带他们出去吃午饭，最后，继续你的任务.
在您的测试用例中，您添加了两个双打，这是一项微不足道的练习，而且花费的时间很少，以至于并行化的开销比简单地在一个线程中完成任务要大.
In your test case you are adding two doubles, which is a trivial exercise and takes so little time that overhead from parallelization is greater than simply doing the task in a one thread.
再一次，这个比喻是叫你所有的朋友帮你搬 3 个手提箱.摆脱它们需要你半天的时间，而你可以在几分钟内完成.
要从并行化中获得任何好处，您的任务必须足够复杂以保证额外的开销.尝试进行一些昂贵的计算，例如一个包含 5-10 个三角函数和对数函数的公式.
To get any benefit from parallelization your task has to be complicated enough to warrant the extra overhead. Try doing some expensive calculations, for example a formula involving a mix of 5-10 trigonometric and logarithmic functions.
这篇关于使用 Scala 并行集合和 setParallelism 的奇怪行为的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
