IntStream导致将数组元素错误地设置为0(JVM Bug，Java 11) [英] IntStream leads to array elements being wrongly set to 0 (JVM Bug, Java 11)

问题描述

在下面的类P中，方法test似乎返回相同的false:

In the class P below, the method test seems to return identically false:

import java.util.function.IntPredicate;
import java.util.stream.IntStream;

public class P implements IntPredicate {
    private final static int SIZE = 33;

    @Override
    public boolean test(int seed) {
        int[] state = new int[SIZE];
        state[0] = seed;
        for (int i = 1; i < SIZE; i++) {
            state[i] = state[i - 1];
        }
        return seed != state[SIZE - 1];
    }

    public static void main(String[] args) {
        long count = IntStream.range(0, 0x0010_0000).filter(new P()).count();
        System.out.println(count);
    }
}

将类P与IntStream组合在一起，但是，方法test可以(错误地)返回true. 上面main方法中的代码产生一些正整数，例如716208. 每次执行后结果都会改变.

Combining class P with IntStream, however, the method test can (wrongly) return true. The code in the main method above results in some positive integer, like 716208. The result changes after every execution.

之所以出现这种意外行为，是因为在执行过程中int数组state[]可以设置为零. 如果是测试代码，例如

This unexpected behavior occurs because the int array state[] can be set to zero during the execution. If a testing code, such as

if (seed == 0xf_fff0){
    System.out.println(Arrays.toString(state));
} 

将

插入到方法test的尾部，则程序将输出类似于[1048560, 1048560, 1048560, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]的行.

is inserted at the tail of the method test, then the program will output a line like [1048560, 1048560, 1048560, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0].

问题:为什么将int数组state[]设置为零?

Question: Why can the int array state[] be set to zero?

我已经知道如何避免这种行为:只需将int[]替换为ArrayList.

I already know how to avoid this behavior: just replacing int[] with ArrayList.

我检查过:

• 带有OpenJDK运行时环境的Windows 10 +和debian 10+(内部版本15.0.1 + 9-18)，OpenJDK 64位服务器VM(内部版本15.0.1 + 9-18，混合模式，共享)
• debian 9 + OpenJDK运行时环境AdoptOpenJDK(内部版本13.0.1 + 9)OpenJDK 64位服务器VM AdoptOpenJDK(内部版本13.0.1 + 9，混合模式，共享)

推荐答案

可以用一个更简单的示例来重现该问题，即:

One can reproduce the problem with an even simpler example, namely:

class Main {
    private final static int SIZE = 33;

    public static boolean test2(int seed) {
        int[] state = new int[SIZE];
        state[0] = seed;
        for (int i = 1; i < SIZE; i++) {
            state[i] = state[i - 1];
        }
        return seed != state[SIZE - 1];
    }

    public static void main(String[] args) {
        long count = IntStream.range(0, 0x0010_0000).filter(Main::test2).count();
        System.out.println(count);

    }
}

此问题是由JVM优化标志引起的，该标志允许对循环(即，-XX:+AllowVectorizeOnDemand)进行矢量化(SIMD).可能是由于在相交范围(即 state[i] = state[i - 1];)相同的数组上应用矢量化而引起的.如果JVM将(对于IntStream.range(0, 0x0010_0000)的某些元素)优化循环，则可能会重现类似的问题:

The problem is caused by the JVM optimization flag that allows the vectorization (SIMD) of loops (i.e., -XX:+AllowVectorizeOnDemand). Likely arises from applying vectorization on the same array with intersecting ranges (i.e., state[i] = state[i - 1];). A similar issue would be possible to reproduce if the JVM would (for some of the elements of the IntStream.range(0, 0x0010_0000)), optimize the loop:

   for (int i = 1; i < SIZE; i++)
       state[i] = state[i - 1];

进入:

    System.arraycopy(state, 0, state, 1, SIZE - 1);

例如:

class Main {
    private final static int SIZE = 33;

    public static boolean test2(int seed) {
        int[] state = new int[SIZE];
        state[0] = seed;
        System.arraycopy(state, 0, state, 1, SIZE - 1);
        if(seed == 100)
           System.out.println(Arrays.toString(state));
        return seed != state[SIZE - 1];
    }

    public static void main(String[] args) {
        long count = IntStream.range(0, 0x0010_0000).filter(Main::test2).count();
        System.out.println(count);
    }
}

输出:

[100, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

新更新:01/01/2021

我已将电子邮件发送给与该标志实现/集成有关的一名开发人员-XX:+AllowVectorizeOnDemandand收到以下答复:

I have sent an email to one of the Developers involved in the implementation/integration of that flag -XX:+AllowVectorizeOnDemandand received the following reply:

众所周知，AllowVectorizeOnDemand代码的一部分已损坏.

It is known that part of AllowVectorizeOnDemand code is broken.

已修复(排除了执行错误代码的错误代码) 向量化)，然后反向移植到jdk 11.0.11中:

There was fix (it excluded executing broken code which does incorrect vectorization) which was backported into jdk 11.0.11:

https://hg.openjdk.java.net/jdk-updates/jdk11u-dev/rev/69dbdd271e04

如果可以，请尝试从以下版本构建和测试最新的OpenJDK11u https://hg.openjdk.java.net/jdk-updates/jdk11u- dev/

If you can, try build and test latest OpenJDK11u from https://hg.openjdk.java.net/jdk-updates/jdk11u-dev/

从第一个链接中，可以阅读以下内容:

From the first link, one can read the following:

@错误8251994 @summary测试Streams $ RangeIntSpliterator :: forEachRemaining的向量化 @需要vm.compiler2.enabled& vm.compMode！="Xint"

@bug 8251994 @summary Test vectorization of Streams$RangeIntSpliterator::forEachRemaining @requires vm.compiler2.enabled & vm.compMode != "Xint"

@运行主compile.vectorization.TestForEachRem test1 @运行主Compiler.vectorization.TestForEachRem test2 @运行主Compiler.vectorization.TestForEachRem test3 @run main compile.vectorization.TestForEachRem test4

@run main compiler.vectorization.TestForEachRem test1 @run main compiler.vectorization.TestForEachRem test2 @run main compiler.vectorization.TestForEachRem test3 @run main compiler.vectorization.TestForEachRem test4

从关于该错误的JIRA 故事的评论中，您可以阅读:

From the comments on the JIRA story on that bug, one can read:

我找到了问题的原因.为了提高对向量进行向量化的机会 循环，超级字尝试通过以下方式将负载提升到循环的开始 用相应的(相同的内存片)替换其内存输入 循环的内存Phi: http://hg .openjdk.java.net/jdk/jdk/file/8f73aeccb27c/src/hotspot/share/opto/superword.cpp#l471

I found the cause of the issue. To improve a chance to vectorize a loop, superword tries to hoist loads to the beginning of loop by replacing their memory input with corresponding (same memory slice) loop's memory Phi : http://hg.openjdk.java.net/jdk/jdk/file/8f73aeccb27c/src/hotspot/share/opto/superword.cpp#l471

最初，负载是由同一商店上的相应商店订购的 内存片.但是当他们吊起时，他们会松开那个命令- 没有什么可以执行命令.在test6的情况下，顺序保留 (幸运地?)仅在矢量大小为32字节(avx2)提升后才 它们变成无序的16(avx = 0或avx1)或64(avx512)字节 向量. (...)

Originally loads are ordered by corresponding stores on the same memory slice. But when they are hoisted they loose that ordering - nothing enforce the order. In test6 case the ordering is preserved (luckily?) after hoisting only when vector size is 32 bytes (avx2) but they become unordered with 16 (avx=0 or avx1) or 64 (avx512) bytes vectors. (...)

我有一个简单的解决方法(使用原始的负载排序索引)，但是 导致问题的代码，我认为它是虚假的/不完整的- 它对列出的JDK-8076284更改案例没有帮助:

I have simple fix (use original loads ordering indexes) but looking on the code which causing the issue I see that it is bogus/incomplete - it does not help cases listed for JDK-8076284 changes:

https://mail.openjdk. java.net/pipermail/hotspot-compiler-dev/2015-April/017645.html

使用展开和克隆信息进行矢量化很有趣 想法，但据我所知还不完整.即使使用pack_parallel()方法 能够创建包，它们都可以通过filter_packs()方法删除. 另外，上述情况在没有提升载荷的情况下进行了矢量化处理 和pack_parallel-我验证了它.该代码现在无济于事 会将其标记为不运行.需要更多的工作才能变得有用. 我不愿意删除该代码，因为可能在将来我们将拥有 是时候投资了.

Using unrolling and cloning information to vectorize is interesting idea but as I see it is not complete. Even if pack_parallel() method is able created packs they are all removed by filter_packs() method. And additionally the above cases are vectorized without hoisting loads and pack_parallel - I verified it. That code is useless now and I will put it under flag to not run it. It needs more work to be useful. I reluctant to remove the code because may be in a future we will have time to invest into it.

这也许可以解释为什么当我比较带有和不带有标记-XX:+AllowVectorizeOnDemand的版本的程序集时，我注意到带有标记的版本适用于以下代码:

This might explain why when I was comparing the assembly of the versions with and without the flag -XX:+AllowVectorizeOnDemand, I notice that the version with the flag for the following code:

   for (int i = 1; i < SIZE; i++)
       state[i] = state[i - 1];

(我提取了一个名为hotstop的方法以方便在程序集中查找它)，具有:

(that I extract on a method called hotstop to facilitate looking for it in the assembly), had:

00000001162bacf5: mov    %r8d,0x10(%rsi,%r10,4)
0x00000001162bacfa: mov    %r8d,0x14(%rsi,%r10,4)
0x00000001162bacff: mov    %r8d,0x18(%rsi,%r10,4)
0x00000001162bad04: mov    %r8d,0x1c(%rsi,%r10,4)
0x00000001162bad09: mov    %r8d,0x20(%rsi,%r10,4)
0x00000001162bad0e: mov    %r8d,0x24(%rsi,%r10,4)
0x00000001162bad13: mov    %r8d,0x28(%rsi,%r10,4)
0x00000001162bad18: mov    %r8d,0x2c(%rsi,%r10,4)  ;*iastore {reexecute=0 rethrow=0 return_oop=0}
                                             ; - AAAAAA.Main::hotstop@15 (line 21)

在我看来，这就像一个循环unrolling，另一方面，方法java.util.stream.Streams$RangeIntSpliterator::forEachRemaining仅在带有标志的版本的汇编中出现.

Which looks to me like a loop unrolling, a side from that, the method java.util.stream.Streams$RangeIntSpliterator::forEachRemaining showed up only in the assembly of the version with the flag.

这篇关于IntStream导致将数组元素错误地设置为0(JVM Bug，Java 11)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！