验证简单的/ lambda比较的JMH测量 [英] Verify JMH measurements of simple for/lambda comparisons
问题描述
我想做一些性能测量和简单for循环和等效流实现的比较。我相信流的情况会比同等的非流代码慢一些，但我想确定我正在测量正确的东西。
I wanted to do some performance measurements and comparisons of simple for loops and equivalent streams implementations. I believe it's the case that streams will be somewhat slower than equivalent non-streams code, but I wanted to be sure I'm measuring the right things.
我是包括我在这里的整个jmh课程。
I'm including my entire jmh class here.
import java.util.ArrayList;
import java.util.List;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
@State(Scope.Benchmark)
public class MyBenchmark {
List<String> shortLengthListConstantSize = null;
List<String> mediumLengthListConstantSize = null;
List<String> longerLengthListConstantSize = null;
List<String> longLengthListConstantSize = null;
@Setup
public void setup() {
shortLengthListConstantSize = populateList(2);
mediumLengthListConstantSize = populateList(12);
longerLengthListConstantSize = populateList(300);
longLengthListConstantSize = populateList(300000);
}
private List<String> populateList(int size) {
List<String> list = new ArrayList<>();
for (int ctr = 0; ctr < size; ++ ctr) {
list.add("xxx");
}
return list;
}
@Benchmark
public long shortLengthConstantSizeFor() {
long count = 0;
for (String val : shortLengthListConstantSize) {
if (val.length() == 3) { ++ count; }
}
return count;
}
@Benchmark
public long shortLengthConstantSizeForEach() {
IntHolder intHolder = new IntHolder();
shortLengthListConstantSize.forEach(s -> { if (s.length() == 3) ++ intHolder.value; } );
return intHolder.value;
}
@Benchmark
public long shortLengthConstantSizeLambda() {
return shortLengthListConstantSize.stream().filter(s -> s.length() == 3).count();
}
@Benchmark
public long shortLengthConstantSizeLambdaParallel() {
return shortLengthListConstantSize.stream().parallel().filter(s -> s.length() == 3).count();
}
@Benchmark
public long mediumLengthConstantSizeFor() {
long count = 0;
for (String val : mediumLengthListConstantSize) {
if (val.length() == 3) { ++ count; }
}
return count;
}
@Benchmark
public long mediumLengthConstantSizeForEach() {
IntHolder intHolder = new IntHolder();
mediumLengthListConstantSize.forEach(s -> { if (s.length() == 3) ++ intHolder.value; } );
return intHolder.value;
}
@Benchmark
public long mediumLengthConstantSizeLambda() {
return mediumLengthListConstantSize.stream().filter(s -> s.length() == 3).count();
}
@Benchmark
public long mediumLengthConstantSizeLambdaParallel() {
return mediumLengthListConstantSize.stream().parallel().filter(s -> s.length() == 3).count();
}
@Benchmark
public long longerLengthConstantSizeFor() {
long count = 0;
for (String val : longerLengthListConstantSize) {
if (val.length() == 3) { ++ count; }
}
return count;
}
@Benchmark
public long longerLengthConstantSizeForEach() {
IntHolder intHolder = new IntHolder();
longerLengthListConstantSize.forEach(s -> { if (s.length() == 3) ++ intHolder.value; } );
return intHolder.value;
}
@Benchmark
public long longerLengthConstantSizeLambda() {
return longerLengthListConstantSize.stream().filter(s -> s.length() == 3).count();
}
@Benchmark
public long longerLengthConstantSizeLambdaParallel() {
return longerLengthListConstantSize.stream().parallel().filter(s -> s.length() == 3).count();
}
@Benchmark
public long longLengthConstantSizeFor() {
long count = 0;
for (String val : longLengthListConstantSize) {
if (val.length() == 3) { ++ count; }
}
return count;
}
@Benchmark
public long longLengthConstantSizeForEach() {
IntHolder intHolder = new IntHolder();
longLengthListConstantSize.forEach(s -> { if (s.length() == 3) ++ intHolder.value; } );
return intHolder.value;
}
@Benchmark
public long longLengthConstantSizeLambda() {
return longLengthListConstantSize.stream().filter(s -> s.length() == 3).count();
}
@Benchmark
public long longLengthConstantSizeLambdaParallel() {
return longLengthListConstantSize.stream().parallel().filter(s -> s.length() == 3).count();
}
public static class IntHolder {
public int value = 0;
}
}
我在Win7笔记本电脑上运行这些。我不关心绝对测量，只关心相对。以下是这些最新结果：
I'm running these on a Win7 laptop. I don't care about absolute measurements, just relative. Here are the latest results from these:
Benchmark Mode Cnt Score Error Units
MyBenchmark.longLengthConstantSizeFor thrpt 200 2984.554 ± 57.557 ops/s
MyBenchmark.longLengthConstantSizeForEach thrpt 200 2971.701 ± 110.414 ops/s
MyBenchmark.longLengthConstantSizeLambda thrpt 200 331.741 ± 2.196 ops/s
MyBenchmark.longLengthConstantSizeLambdaParallel thrpt 200 2827.695 ± 682.662 ops/s
MyBenchmark.longerLengthConstantSizeFor thrpt 200 3551842.518 ± 42612.744 ops/s
MyBenchmark.longerLengthConstantSizeForEach thrpt 200 3616285.629 ± 16335.379 ops/s
MyBenchmark.longerLengthConstantSizeLambda thrpt 200 2791292.093 ± 12207.302 ops/s
MyBenchmark.longerLengthConstantSizeLambdaParallel thrpt 200 50278.869 ± 1977.648 ops/s
MyBenchmark.mediumLengthConstantSizeFor thrpt 200 55447999.297 ± 277442.812 ops/s
MyBenchmark.mediumLengthConstantSizeForEach thrpt 200 57381287.954 ± 362751.975 ops/s
MyBenchmark.mediumLengthConstantSizeLambda thrpt 200 15925281.039 ± 65707.093 ops/s
MyBenchmark.mediumLengthConstantSizeLambdaParallel thrpt 200 60082.495 ± 581.405 ops/s
MyBenchmark.shortLengthConstantSizeFor thrpt 200 132278188.475 ± 1132184.820 ops/s
MyBenchmark.shortLengthConstantSizeForEach thrpt 200 124158664.044 ± 1112991.883 ops/s
MyBenchmark.shortLengthConstantSizeLambda thrpt 200 18750818.019 ± 171239.562 ops/s
MyBenchmark.shortLengthConstantSizeLambdaParallel thrpt 200 474054.951 ± 1344.705 ops/s
<在之前的一个问题中，我确认这些基准似乎是功能相同的（只是寻找addit眼睛）。这些数字看起来是否一致，可能是这些基准的独立运行？
In an earlier question, I confirmed that these benchmarks appear to be "functionally equivalent" (just looking for additional eyes). Do these numbers appear to be in line, perhaps with independent runs of these benchmarks?
我一直不确定JMH输出的另一件事是确切地确定了什么吞吐量数字代表。例如，Cnt列中的200究竟代表什么？吞吐量单位是每秒操作次数，那么操作究竟表示的是对基准方法执行一次调用？例如，在最后一行，这将代表基准方法在一秒钟内执行474k次。
Another thing that I've always been uncertain about with JMH output, is determining exactly what the throughput numbers represent. For instance, what does the "200" in the "Cnt" column exactly represent? The throughput units are in "operations per second", so what exactly does the "operation" represent, is that the execution of one call to the benchmark method? For instance, in the last row, that would represent 474k executions of the benchmark method in a second.
更新：
我注意到当我将for与lambda进行比较时，从short列表开始并转到更长的列表，它们之间的比例相当大，但是减少了，直到长列表，其中比率甚至大于短列表（14％，29％，78％和11％）。我觉得这很令人惊讶。我希望随着实际业务逻辑中的工作增加，流开销的比率会降低。有人对此有什么想法吗？
I note that when I compare the "for" with the "lambda", starting with the "short" list and going to longer lists, the ratio between them is pretty large, but decreases, until the "long" list, where the ratio is even larger than for the "short" list (14%, 29%, 78%, and 11%). I find this surprising. I would have expected the ratio of the streams overhead to decrease as the work in the actual business logic increases. Anyone have any thoughts on that?
推荐答案
例如，200中的200是什么？ Cnt专栏完全代表？
For instance, what does the "200" in the "Cnt" column exactly represent?
cnt 列是迭代次数 - 即a次多少次重复测试。您可以使用以下注释控制该值：
The cnt column is the number of iterations - i.e. how many times a tests is repeated. You can control that value using the following annotations:
- 对于实际测量：
@Measurement（iterations = 10， time = 50，timeUnit = TimeUnit.MILLISECONDS） - 对于预热阶段：
@Warmup（迭代= 10，时间= 1， timeUnit = TimeUnit.SECONDS）
- For the actual measurements:
@Measurement(iterations = 10, time = 50, timeUnit = TimeUnit.MILLISECONDS) - For the warmup phase:
@Warmup(iterations = 10, time = 1, timeUnit = TimeUnit.SECONDS)
这里迭代次数是 cnt ; 时间是一次迭代所需的持续时间， timeUnit 是的度量单位时间价值。
Here iterations is cnt; time is the required duration of one iteration, and timeUnit is the unit of measurement of the time value.
吞吐量单位为每秒操作数
The throughput units are in "operations per second"
您可以通过多种方式控制输出。例如，您可以使用 @OutputTimeUnit（TimeUnit.XXXX）更改时间单位，这样您就可以获得ops / us，ops / ms
You can control the output in several ways. For instance you can change the unit of measurement for the time using @OutputTimeUnit(TimeUnit.XXXX), so you can get ops/us, ops/ms
您还可以更改模式：而不是测量操作/时间，您可以测量平均时间，采样时间等你可以通过 @BenchmarkMode（{Mode.AverageTime}）注释来控制这个。
You can also change the mode: instead of measureing ops/time you can measure "average time", "sample time", etc. You can control this via the @BenchmarkMode({Mode.AverageTime}) annotation
<那么操作究竟代表什么，是对基准方法的一次调用的执行
so what exactly does the "operation" represent, is that the execution of one call to the benchmark method
所以，让我们说一个迭代是1秒长，你得到1000操作/秒。这意味着benchamrk方法已执行1000次。
So lets say one iteration is 1 second long and you get 1000 ops/sec. This means that the benchamrk method has been executed 1000 times.
换句话说，一个操作是一次执行基准测试方法，除非你有 @OperationsPerInvocation（XXX）注释，这意味着教授方法的调用将计为XXX操作。
In other words one operation is one execution of the benchmark method, unless you have the @OperationsPerInvocation(XXX) annotation, which means tha teach invocation of the methods will count as XXX operations.
在所有迭代中计算错误。
The error is calculated across all iterations.
还有一个提示：您可以执行参数化基准测试，而不是对每个可能的大小进行硬编码：
One more tip: instead of hardcoding each possible size, you can do a parameterized benchmark:
@Param({"3", "12", "300", "3000"})
private int length;
然后你可以在你的设置中使用那个参数：
Then you can use that param in your setup:
@Setup(Level.Iteration)
public void setUp(){
populateList(length)
}
这篇关于验证简单的/ lambda比较的JMH测量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
