Scala并行集合:如何知道和配置线程数 [英] Scala Parallel Collections: How to know and configure the number of threads

问题描述

我正在使用scala并行集合.

I am using scala parallel collections.

val largeList = list.par.map(x => largeComputation(x)).toList

它的速度很快，但是我有一种感觉，如果我们同时运行"largeComputation"，我可能会遇到内存不足的问题.

It is blazing fast, but I have a feeling that I may run into out-of-memory issues if we run too may "largeComputation" in parallel.

因此，在测试时，我想知道并行集合使用了多少个线程(如果需要)，我如何配置并行集合的线程数.

Therefore when testing, I would like to know how many threads is the parallel collection using and if-need-be, how can I configure the number of threads for the parallel collections.

推荐答案

这是一段scaladoc，他们在其中解释如何更改任务支持并将其包装在ForkJoinPool中.实例化ForkJoinPool时，将您希望的并行度级别作为参数传递:

Here is a piece of scaladoc where they explain how to change the task support and wrap inside it the ForkJoinPool. When you instantiate the ForkJoinPool you pass as the parameter desired parallelism level:

Here is a way to change the task support of a parallel collection:

import scala.collection.parallel._
val pc = mutable.ParArray(1, 2, 3)
pc.tasksupport = new ForkJoinTaskSupport(new scala.concurrent.forkjoin.ForkJoinPool(2))

对于您的情况，它将是

val largeList = list.par
largerList.tasksupport = new ForkJoinTaskSupport(
  new scala.concurrent.forkjoin.ForkJoinPool(x)
)
largerList.map(x => largeComputation(x)).toList

这篇关于Scala并行集合:如何知道和配置线程数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！