如何检查flink中的DataStream是否为空或是否有数据 [英] How to check DataStream in flink is empty or having data

问题描述

我是Apache flink的新手，我有一个数据流，如果满足某些条件，则该数据流将实现流程功能，则该数据流有效，如果不满足条件，则将其写入sideoutput.我能够打印数据流，是否可以检查数据流为空或空.我尝试使用 datastream.equals(null)方法但不起作用，请提出如何知道数据流是否正确的建议空与否

I am new to Apache flink i have a datastream which implements a process function if certain conditions is met then the datastream is valid and if its not meeting the conditions i am writing it to sideoutput. I am able to print the datastream is it possible to check the datastream is empty or null.I tried using datastream.equals(null) method but its not working.Please suggest how to know whether a datastream is empty or not

推荐答案

通过空"，我认为您的意思是没有数据在流动.在这种情况下，您希望做什么?

By "empty", I assume you mean that no data is flowing. What are you hoping to do in this case?

Flink没有空"流的明确定义.流始终连接到一个或多个源，这些源可以是有界的也可以是无界的.有界源(例如文件和集合)最终会通过到达终点而终止(这时它们会发出带有MAX_WATERMARK值的水印，您可以使用计时器来观察它)，但是通常无法知道无界源(例如，Kafka主题)可能会产生任何(更多)数据.

Flink doesn't have a well-defined notion of an "empty" stream. Streams are always connected to one or more sources, which can be bounded or unbounded. Bounded sources (like files and collections) eventually terminate by reaching their end (at which point they emit a watermark with the value MAX_WATERMARK, which you could watch for with a timer), but in general there is no way of knowing whether an unbounded source (e.g., a Kafka topic) might produce any (more) data.

但是，您可以观察一些指标，例如NumRecordsOut或NumRecordsOutPerSecond，这些指标将告诉您是否正在生成任何输出.或者，您的流程功能可以收集有关其行为的信息，并在侧面输出中报告该信息(就像您已经在做的一样).

There are, however, metrics you can observe, such as NumRecordsOut, or NumRecordsOutPerSecond, that will tell you if any output is being produced. Or your process function could collect information about its behavior and report this on a side output (rather like what you are already doing).

这篇关于如何检查flink中的DataStream是否为空或是否有数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！