Java 8，在流中使用.parallel会导致OOM错误 [英] Java 8, using .parallel in a stream causes OOM error

问题描述

在 Java 8 In Action 一书的第7.1.1节中，作者指出，通过添加函数.parallel()，流可以从并行处理中受益.他们提供了一个称为parallelSum(int)的简单方法来说明这一点.我很想知道它的效果如何，所以我执行了这段代码:

In the book Java 8 In Action, section 7.1.1, the authors state that a stream can benefit from parallel processing by adding the function .parallel(). They provide a simple method called parallelSum(int) to illustrate this. I was curious to see how well it worked so I executed this code:

package lambdasinaction.chap7;

import java.util.stream.Stream;

public class ParallelPlay {

    public static void main(String[] args) {
        System.out.println(parallelSum(100_000_000));
    }

    public static long parallelSum(long n) {
        return Stream.iterate(1L, i -> i + 1)
                .limit(n)
                .parallel()
                .reduce(0L, Long::sum);
    }
}

令我惊讶的是，我收到此错误:

To my surprise, I received this error:

Exception in thread "main" java.lang.OutOfMemoryError
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
    at java.lang.reflect.Constructor.newInstance(Unknown Source)
    at java.util.concurrent.ForkJoinTask.getThrowableException(Unknown Source)
    at java.util.concurrent.ForkJoinTask.reportException(Unknown Source)
    at java.util.concurrent.ForkJoinTask.invoke(Unknown Source)
    at java.util.stream.SliceOps$1.opEvaluateParallelLazy(Unknown Source)
    at java.util.stream.AbstractPipeline.sourceSpliterator(Unknown Source)
    at java.util.stream.AbstractPipeline.evaluate(Unknown Source)
    at java.util.stream.ReferencePipeline.reduce(Unknown Source)
    at lambdasinaction.chap7.ParallelPlay.parallelSum(ParallelPlay.java:15)
    at lambdasinaction.chap7.ParallelPlay.main(ParallelPlay.java:8)
Caused by: java.lang.OutOfMemoryError: Java heap space
    at java.util.stream.SpinedBuffer.ensureCapacity(Unknown Source)
    at java.util.stream.Nodes$SpinedNodeBuilder.begin(Unknown Source)
    at java.util.stream.AbstractPipeline.copyInto(Unknown Source)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
    at java.util.stream.SliceOps$SliceTask.doLeaf(Unknown Source)
    at java.util.stream.SliceOps$SliceTask.doLeaf(Unknown Source)
    at java.util.stream.AbstractShortCircuitTask.compute(Unknown Source)
    at java.util.concurrent.CountedCompleter.exec(Unknown Source)
    at java.util.concurrent.ForkJoinTask.doExec(Unknown Source)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(Unknown Source)
    at java.util.concurrent.ForkJoinPool.runWorker(Unknown Source)
    at java.util.concurrent.ForkJoinWorkerThread.run(Unknown Source)

我正在Windows 7 SP1上使用四核处理器运行Java 1.8.0_45.发生了什么事?

I am running Java 1.8.0_45 on Windows 7, SP1 with a four-core processor. What's going on?

推荐答案

在这里创建无限流，然后限制它.存在关于并行处理无限流的已知问题.特别是没有办法有效地将任务分成相等的部分.在内部，使用了一些启发式方法，这些启发式方法不适用于每个任务.在您的情况下，最好使用LongStream.range:

Here you create an infinite stream and limit it afterwards. There are known problems about processing infinite streams in parallel. In particular there's no way to split the task to equal parts effectively. Internally some heuristics are used which are not well suitable for every task. In your case it's much better to create the finite stream using LongStream.range:

import java.util.stream.LongStream;

public class ParallelPlay {

    public static void main(String[] args) {
        System.out.println(parallelSum(100_000_000));
    }

    public static long parallelSum(long n) {
        return LongStream.rangeClosed(1, n).parallel().sum();
    }
}

在这种情况下，Stream引擎从一开始就知道您有多少个元素，因此它可以有效地拆分任务.另外请注意，使用LongStream更为有效，因为您不会出现不必要的装箱.

In this case Stream engine knows from the very beginning how many elements you have, so it can split the task effectively. Also note that using the LongStream is more effecient as you will have no unnecessary boxing.

如果可以用有限的流解决任务，通常应避免使用无限流.

In general avoid infinite streams if you can solve your task with finite ones.

这篇关于Java 8，在流中使用.parallel会导致OOM错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！