我的Flink应用程序需要水印吗?如果没有，我是否需要WatermarkStrategy.noWatermarks? [英] Does my Flink application need watermarks? If not, do I need WatermarkStrategy.noWatermarks?

问题描述

我不确定我的Flink应用程序是否真的需要水印.他们什么时候有必要?

I'm not sure if my Flink application actually requires Watermarks. When are they necessary?

如果我不需要它们， WatermarkStrategy.noWatermarks()的目的是什么?

And if I don't need them, what is the purpose of WatermarkStrategy.noWatermarks()?

推荐答案

时间 t 的水印标记数据流中的位置，并断言此时该流已完成.通过时间 t .

A Watermark for time t marks a location in a data stream and asserts that the stream, at that point, is now complete up through time t.

水印的唯一作用是触发基于事件时间的计时器的触发.

The only purpose watermarks serve is to trigger the firing of event-time-based timers.

基于事件时间的计时器直接由 KeyedProcessFunction API公开，也由内部

Event-time-based timers are directly exposed by the KeyedProcessFunction API, and are also used internally by

• 事件时间窗口
• CEP(模式匹配)库，如果您指定要进行基于事件时间的处理，则该库使用水印对传入流进行排序
• 仅在执行基于事件时间的处理时再次链接SQL，例如ORDER BY，版本化表联接，windows，MATCH_RECOGNIZE等.

不需要水印的常见情况包括仅依赖处理时间或进行批处理的应用程序.或者在处理具有时间戳记但从不依赖事件时间计时器的数据时(例如，简单的逐事件处理).

Common cases where you don't need watermarks include applications that only rely on processing time, or when doing batch processing. Or when processing data that has timestamps, but never relying on event-time timers (e.g., simple event-by-event processing).

由FLIP-27引入的Flink的新源接口确实需要 WatermarkStrategy :

Flink's new source interface, introduced by FLIP-27, does require a WatermarkStrategy:

env.fromSource(source, watermarkStrategy, sourceName);

在实际上不需要水印的情况下，可以在此界面中使用 WatermarkStrategy.noWatermarks().

In cases where you don't actually need watermarks, you can use WatermarkStrategy.noWatermarks() in this interface.

这篇关于我的Flink应用程序需要水印吗?如果没有，我是否需要WatermarkStrategy.noWatermarks?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！