为什么在使用forEachOrdered的静态初始化程序块中使用lambda进行并行流处理会产生死锁，而不是forEach？ [英] Why does a parallel stream processing with lambda in the static initializer block with forEachOrdered produces a deadlock, but not with forEach?

问题描述

在使用Java并行流时，我在静态初始化程序块中完成某些并行操作时遇到了死锁。

While playing with Java parallel streams, I experienced deadlocks when some parallel operations are done within a static initializer block.

使用顺序Stream时，一切正常：

When using a sequential Stream, everything works fine:

import java.util.Arrays;
public class Example1 {
    static {
        // displays the numbers from 1 to 10 ordered => no thread issue
        Arrays.asList(1,2,3,4,5,6,7,8,9,10)
             .forEach(s->System.out.println(s));
    }
    public static final void main(String[] args) {}
}

当并行处理流时，每次工作（数字都没有显示顺序）：

When processing the stream in parallel, everyting work (the numbers are displayed without order):

import java.util.Arrays;
public class Example2 {
    static {
        // displays the numbers from 1 to 10 unordered => no thread issue
        Arrays.asList(1,2,3,4,5,6,7,8,9,10).parallelStream()
             .forEach(s->System.out.println(s));
    }
    public static final void main(String[] args) {}
}

但是，当使用 forEachOrdered（）处理Stream时，会发生死锁（我想这与主线程和ForkJoinPool管理之间的交互有关） ）：

However, when processing the Stream with forEachOrdered(), a deadlock occurs (I suppose this is related to the interaction between the main thread and the ForkJoinPool management):

import java.util.Arrays;
public class Example3 {
    static {
        // hangs forever (deadlock between the main thread which loads the class and the underlying ForkJoinPool which join several tasks)
        Arrays.asList(1,2,3,4,5,6,7,8,9,10).parallelStream()
                .forEachOrdered(s->System.out.println(s));
    }
    public static final void main(String[] args) {}
}

但是当在单独的线程中生成Stream处理时，一切顺利：

But when spawning the Stream processing in a separate Thread, everything goes well:

import java.util.Arrays;
public class Example4 {
    static {
        // displays the numbers from 1 to 10 ordered => no thread issue
        new Thread(()->
            Arrays.asList(1,2,3,4,5,6,7,8,9,10).parallelStream()
                 .forEachOrdered(s->System.out.println(s))
        ).start();
    }
    public static final void main(String[] args) {}
}

从我在Thread Dump中看到的，主线程正在等待 .forEachOrdered（）中使用的ForkJoinPool完成他的工作，但是池中的第一个工作线程被阻塞等待某事（很可能被主线程阻塞）。

From what I've seen from the Thread Dump, the main Thread is waiting on the ForkJoinPool used in the .forEachOrdered() to finish his work, but the first worker Thread in the pool is blocked waiting for something (most probably blocked by the main thread).

我真的很感激理解为什么在某些情况下会发生死锁而在其他情况下不会发生。这显然不仅仅是由于静态初始化程序块，并行流和lambda的使用，因为 Example2 ， Example3 和 Example4 使用这三个概念，但只有 Example3 会导致死锁。

I would really appreciate to understand why the deadlock occurs in some cases and not in other cases. This is obviously not due only to the usage of static initializer block, parallel stream and lambda because Example2, Example3 and Example4 use these three concepts, but only Example3 causes a deadlock.

虽然这个问题看起来像是为什么在静态初始化程序中使用lambda的并行流导致死锁？，事实并非如此。我的问题超出了链接的问题，因为它提供了 Example2 ，我们有静态初始化块，并行流和lambda，但没有死锁。这就是问题标题包含可能导致死锁但不一定的原因。

While this question may look like a duplicate of Why does parallel stream with lambda in static initializer cause a deadlock?, it is not. My question goes beyond the linked one as it provide Example2 for which we have static initializer block, parallel stream and lambda, but no deadlock. This is why the question title contains "may lead to deadlock but not necessarily".

推荐答案

此死锁行为有两个根本原因：

This deadlock behavior has two root causes:

1. main 线程正在等待另一个线程（让我们说 OtherThread ）完成其工作（在Example3中， OtherThread 是 ForkJoinPool的一个线程使用 forEachOrdered（）操作）


2. OtherThread 使用一个Lambda表达式，该表达式将由 main Thread定义但稍后（回想一下：Lambdas是在运行时创建的，而不是在编译时创建的）。在Example3中，此Lambda是 .forEachOrdered（）中的一个。

1. The main Thread is waiting that another Thread (let's say OtherThread) finishes its work (in the Example3, the OtherThread is one of the Thread of the ForkJoinPool used by the forEachOrdered() operation)
2. The OtherThread uses a Lambda expression which will be defined by the main Thread but later (recall: Lambdas are created at runtime, not at compile time). In the Example3, this Lambda is the one in the .forEachOrdered().

<让我们回顾一下这些例子并解释它们为什么会产生死锁。

Let's review the examples and explain why they produce or not a deadlock.

只有一个线程（ main ）执行以下操作：

Only one Thread (main) does the following operations:

1. 处理静态初始化程序块


2. 对每个元素执行foreach


3. 在处理第一个流元素时在运行时创建lambda表达式

由于只有一个线程，因此不会出现死锁。

Since there is only one thread, no deadlock can occur.

为了更好地理解处理，我们可以将其重写为：

In order to have a better understanding of the processing, we can rewrite it as :

import java.util.Arrays;
public class Example2Instrumented {
    static {
        // displays the numbers from 1 to 10 unordered => no thread issue
        System.out.println(Thread.currentThread().getName()+" : "+"static initializer");
        Arrays.asList(1,2,3,4,5,6,7,8,9,10)
             .parallelStream()
             .forEach(s->System.out.println(Thread.currentThread().getName()+" : "+s));
    }
    public static final void main(String[] args) {}
}

这会产生以下结果：

main : static initializer
main : 7
main : 6
ForkJoinPool.commonPool-worker-2 : 9
ForkJoinPool.commonPool-worker-4 : 5
ForkJoinPool.commonPool-worker-9 : 3
ForkJoinPool.commonPool-worker-11 : 2
ForkJoinPool.commonPool-worker-2 : 10
ForkJoinPool.commonPool-worker-4 : 4
ForkJoinPool.commonPool-worker-9 : 1
ForkJoinPool.commonPool-worker-13 : 8

main Thread处理静态初始化程序，然后启动forEach并在处理第一个元素时在运行时构建lambda。其他流元素由 ForkJoinPool 中的工作线程处理。没有死锁，因为 main Thread处理了第一个元素并构建了lambda。

The main Thread processes the static initializer, then starts the forEach and build the lambda at runtime when processing the first element. The other stream elements are processed by the workers Threads from the ForkJoinPool. There is no deadlock because the main Thread processed the first element and built the lambda.

我们可以在没有Lambda的情况下重写Example3来打破僵局：

We can rewrite Example3 without the Lambda to break the deadlock:

import java.util.Arrays;
import java.util.function.Consumer;
public class Example3NoDeadlock {
    static {
        // displays the numbers from 1 to 10 ordered => no thread issue anymore
        Arrays.asList(1,2,3,4,5,6,7,8,9,10).parallelStream()
                .forEachOrdered(
                    new Consumer<Integer>() {
                        @Override
                        public void accept(Integer t) {
                            System.out.println(t);          
                    }});
    }
    public static final void main(String[] args) {}
}

由于 Consumer Class是在编译时构造的（与在运行时构建的lambdas相反），这会打破死锁循环。这证明至少lambda参与了死锁。

Since the Consumer Class is constructed at compile time (contrary to lambdas that are built at runtime), this breaks the deadlock cycle. This prooves that at least the lambda is involved in the deadlock.

为了更好地理解，我们可以按照以下方式设置代码：

To have a better understanding, we could instrument the code as follow:

import java.util.Arrays;
import java.util.function.Consumer;
public class Example3Instrumented {
    static {
        System.out.println("static initializer");
        // hangs forever (deadlock between the main thread which loads the class and the underlying ForkJoinPool which join several tasks)
        Arrays.asList(1,2,3,4,5,6,7,8,9,10).parallelStream()
            .peek(new Consumer<Integer>() {
                @Override
                public void accept(Integer t) {
                        System.out.println(Thread.currentThread().getName()+" "+t);
            }})
                .forEachOrdered(s->System.out.println(s));
    }
    public static final void main(String[] args) {}
}

这会产生以下输出：

main : static initializer
ForkJoinPool.commonPool-worker-6 1
ForkJoinPool.commonPool-worker-9 3
main 7
ForkJoinPool.commonPool-worker-4 2
ForkJoinPool.commonPool-worker-13 6
ForkJoinPool.commonPool-worker-11 8
ForkJoinPool.commonPool-worker-15 5
ForkJoinPool.commonPool-worker-2 9
ForkJoinPool.commonPool-worker-4 10
ForkJoinPool.commonPool-worker-9 4

main 线程进程静态初始化程序，然后通过为流中的每个元素创建一个Task来开始处理forEachOrdered（为了维护顺序，使用了一个复杂的基于树的算法，请参阅 ForEachOps.ForEachOrderedTask ：创建任务，并从代码中查看每个任务正在等待另一个任务完成运行）。所有任务都提交给 ForkJoinPool 。我认为死锁发生是因为第一个任务由来自 ForkJoinPool 的工作线程处理，并且此线程等待 main 用于构建lambda的线程。并且 main Thread已经开始处理其Task并正在等待另一个工作线程完成其要运行的Task。因此死锁。

The main Thread processes the static initializer, then starts processing the forEachOrdered by creating a Task for each element in the stream (to maintain the order, a complex tree-based algorithm is used, see ForEachOps.ForEachOrderedTask: tasks are created and it looks from the code that there is each task is waiting that another task is completed to run). All the tasks are submitted to the ForkJoinPool. I think the deadlock occures because the first Task is processed by a worker Thread from the ForkJoinPool and this Thread waits on the main Thread to build the lambda. And the main Thread has already started processing its Task and is waiting for another worker thread to complete its Task to run. Hence the deadlock.

在Example4中，我们生成了一个异步运行的新线程（即我们不等待结果）。这就是为什么 main Thread没有被锁定，现在有时间在运行时构建Lambdas。

In the Example4, we spawn a new Thread that is ran asynchronously (i.e. we don't wait for the result). This is why the main Thread is not locked and has now the time to build the Lambdas at runtime.

外卖课程是：如果你混合静态初始化器，线程和lambdas，你应该真正理解这些概念是如何实现的，否则你可能会遇到死锁。

The takeaway lesson is : if you mix static initializers, threads and lambdas, you should really understand how these concepts are implemented, otherwise you may have deadlocks.

这篇关于为什么在使用forEachOrdered的静态初始化程序块中使用lambda进行并行流处理会产生死锁，而不是forEach？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！