Flink 流顺序 [英] Flink stream order

问题描述

Flink 是否保证流的执行顺序?

我有两个 Kafka 主题(每个主题都有一个分区)，流 1 和流 2，并使用 keyBy，流由 coprocess 函数处理.>

在我的测试过程中，我可以看到两个流的内容并不总是按顺序执行.

我可以将并行度设置为 1 来解决此问题，但想了解我所看到的是否符合预期?

source1.keyBy(e -> e.f0);source2.keyBy(e -> {返回新的 JSONObject(e.f1).getString("someOtherKey");});source1.connect(source2).process(new KeyedCoProcessFunction()).uid(uid").name(名称");

解决方案

如果事件在执行图中采用相同的路径，则事件将保持其相对顺序.否则，所有赌注都将取消.

以下是使用 keyBy 可能发生的情况的示例.最初的事件被排序为 1, 2, A

但是 keyBy 将 A 发送到与 1 和 2 不同的路径:

之后我们可能会看到 A, 1, 2(如图所示)，或 1, A, 2 或 1, 2, A.

Does Flink guarantee the order of execution of a stream?

I have two Kafka topics (each with single partition), stream 1 and stream 2, and use keyBy, the streams are processed by a coprocess function.

During my testing I can see the the contents of both streams are not always executed in order.

I can set parallelism to 1 to get around this issue, but wanted to understand if what I am seeing is what is expected?

source1.keyBy(e -> e.f0);
source2.keyBy(e -> {
    return new JSONObject(e.f1).getString("someOtherKey");
});

source1.connect(source2)
       .process(new KeyedCoProcessFunction())
       .uid("uid")
       .name("name");

解决方案

Events will maintain their relative ordering if they take the same path through the execution graph. Otherwise, all bets are off.

Here's an example of what can happen with a keyBy. Initally the events are ordered as 1, 2, A

but a keyBy sends A down a different path than 1 and 2:

after which we might see A, 1, 2 (as shown here), or 1, A, 2, or 1, 2, A.

这篇关于Flink 流顺序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！