在 Flink 中，为什么 DataStream 不支持聚合 [英] In Flink, why is aggregation not supported by DataStream

问题描述

我是 Flink 的新手.有时在某些情况下，我想在 DataStream 上进行聚合而无需先执行 keyBy.为什么 Flink 不支持 DataStream 上的聚合(sum、min、max 等)?

I am a newbie to Flink. Sometimes there are cases where I want to do aggregation on a DataStream without needed to do a keyBy first. Why doesn't Flink support aggregation (sum, min, max, etc.) on a DataStream?

谢谢，艾哈迈德.

推荐答案

With FLIP-134 Flink 社区决定弃用 DataStream API 中的所有这些关系方法:

With FLIP-134 the Flink community has decided to deprecate all of these relational methods from the DataStream API:

• DataStream#project
• Windowed/KeyedStream#sum,mi​​n,max,minBy,maxBy
• DataStream#keyBy 其中用字段名或索引指定的键(包括ConnectedStreams#keyBy)

• DataStream#project
• Windowed/KeyedStream#sum,min,max,minBy,maxBy
• DataStream#keyBy where the key specified with field name or index (including ConnectedStreams#keyBy)

这个决定背后的基本原理是 Table/SQL 是一个更完整、更高效的关系 API，并且它已经支持批处理和流.使用这些 API，您可以轻松执行全局聚合，而无需先执行 keyBy 或 GROUP BY.

The rationale behind this decision is that Table/SQL is a more complete and more performant relational API, and it already supports both batch and streaming. With these APIs you can easily perform global aggregations, without having to first do a keyBy or GROUP BY.

示例:

StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
StreamTableEnvironment tableEnv = StreamTableEnvironment.create(env);

SingleOutputStreamOperator<Integer> numbers = env.fromElements(0, 1, 1, 0, 3, 2);

Table data = tableEnv.fromDataStream(numbers, $("n"));

Table results = data.select($("n").max());

tableEnv
        .toRetractStream(results, Row.class)
        .print();

env.execute();

这篇关于在 Flink 中，为什么 DataStream 不支持聚合的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！