获取两个列表的元素求和的最快方法 [英] Fastest way to take elementwise sum of two Lists

问题描述

我可以使用 Zipped 功能执行诸如sum之类的元素操作.让我有两个列表，如下所示: L1 和 L2

I can do elementwise operation like sum using Zipped function. Let I have two Lists L1 and L2 as shown below

val L1 = List(1,2,3,4)
val L2 = List(5,6,7,8)

我可以按照以下方式获取元素明智的总和

I can take element wise sum in following way

(L1,L2).zipped.map(_+_)

结果是

List(6, 8, 10, 12) 

符合预期.

我在我的实际代码中使用了 Zipped 函数，但是这花费了太多时间.实际上，我的列表大小超过 1000 ，而我的列表超过 1000 ，并且我的算法是迭代的，迭代次数可以达到十亿 .

I am using Zipped function in my actual code but it takes too much time. In reality My List Size is more than 1000 and I have more than 1000 Lists and my algorithm is iterative where iterations could be up to one billion.

在代码中，我必须做以下事情

In code I have to do following stuff

list =( (L1,L2).zipped.map(_+_).map (_  * math.random) , L3).zipped.map(_+_)

L1 ， L2 和 L3 的大小相同.而且，我必须在群集上执行我的实际代码.

size of L1,L2 and L3 is same. Moreover I have to execute my actual code on a cluster.

在Scala中对列表进行元素求和最快的方法是什么?

What is the fastest way to take elementwise sum of Lists in Scala?

推荐答案

一种选择是使用 Streaming 实现，利用 lazyness 可能会提高性能

One option would be to use a Streaming implementation, taking advantage of the lazyness may increase the performance.

使用 LazyList (在Scala 2.13中引入)的示例.

An example using LazyList (introduced in Scala 2.13).

def usingLazyList(l1: LazyList[Double], l2: LazyList[Double], l3: LazyList[Double]): LazyList[Double] =
  ((l1 zip l2) zip l3).map {
    case ((a, b), c) =>
      ((a + b) * math.random()) + c
  }

以及使用 fs2.Stream 的示例(由 fs2 库).

import fs2.Stream
import cats.effect.IO

def usingFs2Stream(s1: Stream[IO, Double], s2: Stream[IO, Double], s3: Stream[IO, Double]): Stream[IO, Double] =
  s1.zipWith(s2) {
    case (a, b) =>
      (a + b) * math.random()
  }.zipWith(s3) {
    case (acc, c) =>
      acc + c
  }

---

但是，如果仍然太慢，最好的选择是使用纯数组.

---

However, if those are still too slow, the best alternative would be to use plain arrays.

以下是使用 ArraySeq (也在Scala 2.13中引入)的示例，至少可以保留不变性.如果愿意，可以使用原始数组，但要小心.
(如果需要，您也可以使用 collections-parallel module 作为更有表现力)

Here is an example using ArraySeq (introduced in Scala 2.13 too) which at least will preserve immutability. You may use raw arrays if you prefer but take care.
(if you want, you may also use the collections-parallel module to be even more performant)

import scala.collection.immutable.ArraySeq
import scala.collection.parallel.CollectionConverters._

def usingArraySeq(a1: ArraySeq[Double], a2: ArraySeq[Double], a3: ArraySeq[Double]): ArraySeq[Double] = {
  val length = a1.length

  val arr = Array.ofDim[Double](length)

  (0 until length).par.foreach { i =>
    arr(i) = ((a1(i) + a2(i)) * math.random()) + a3(i)
  }

  ArraySeq.unsafeWrapArray(arr)
}

这篇关于获取两个列表的元素求和的最快方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！