如何在 Apache Flink 中连接两个流 [英] How to concatenate two streams in Apache Flink

问题描述

例如我想将 1, 2, 3 和 4, 5 的流组合成一个，所以结果应该是: 1, 2, 3, 4, 5.换句话说:如果第一个源用尽 - 从第二个获取元素.我最接近的尝试是:

E.g. i want to compose stream of 1, 2, 3 and 4, 5 in single one, so result should be: 1, 2, 3, 4, 5. In other words: if first source is exhausted - get elements from second one. My closest attempt, which unfortunately does not preserve items order, is:

val a = streamEnvironment.fromElements(1, 2, 3)

val b = streamEnvironment.fromElements(4, 5)

val c = a.union(b)

c.map(x => println(s"X=$x")) // X=4, 5, 1, 2, 3 or something like that

也做了类似的尝试，包括日期时间，但结果相同.

Also did similar attempt with datetime included, but with same result.

推荐答案

这目前是不可能的，至少对于高级 DataStream API 是不可能的.

This is not possible right now, at least not with the high level DataStream API.

有可能实现一个低级运算符，先读取输入，然后读取另一个输入.但是，这将完全阻止一个输入，该输入与 Flink 处理水印和执行检查点的方式不符.

It might be possible to implement a low-level operator that first reads on input and then the other input. However, this will completely block one input which does not work well with the way that Flink handles watermarks and performs checkpoints.

将来，这将可以使用所谓的 辅助输入.

In the future, this will be possible using so-called side inputs.

这篇关于如何在 Apache Flink 中连接两个流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！