显示Java 8流处理的进度 [英] Show progress of Java 8 stream processing

问题描述

我有一个 Stream 处理数百万个元素.其背后的Map-Reduce算法需要几毫秒的时间，因此任务完成大约需要20分钟.

I have a Stream processing a few millions of elements. The Map-Reduce algorithm behind it takes a few milliseconds, so task completion takes about twenty minutes.

Stream<MyData> myStream = readData();
MyResult result = myStream
    .map(row -> process(row))
    .peek(stat -> System.out.println("Hi, I processed another item"))
    .reduce(MyStat::aggregate);

我想要一种显示总体进度的方法，而不是每个元素打印一行(这导致每秒数千行，耗时并且不提供有关总体进度的任何有用信息).我想显示类似的内容:

I'd like a way to display overall progress, instead of printing a line per element (which results in thousands of lines per second, takes time and doesn't provide any useful information regarding overall progress). I would like to display something similar to:

 5% (08s)
10% (14s)
15% (20s)
...

什么是最好的(和/或最简单的)方法?

What would be the best (and/or easiest) way to do this?

推荐答案

首先，Streams并不是要实现此类任务(与经典数据结构相反).如果您已经知道流将处理多少个元素，则可以使用以下选项，我再说一遍，而不是流的目标.

First of all, Streams are not meant to achieve these kind of tasks (as opposed to a classic data structure). If you know already how many elements your stream will be processing you might go with the following option, which is, I repeat, not the goal of streams.

Stream<MyData> myStream = readData();
final AtomicInteger loader = new AtomicInteger();
int fivePercent = elementsCount / 20;
MyResult result = myStream
    .map(row -> process(row))
    .peek(stat -> {
        if (loader.incrementAndGet() % fivePercent == 0) {
            System.out.println(loader.get() + " elements on " + elementsCount + " treated");
            System.out.println((5*(loader.get() / fivePercent)) + "%");
        }
    })
    .reduce(MyStat::aggregate);

这篇关于显示Java 8流处理的进度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！