Apache Flink-数据集API-侧面输出 [英] Apache Flink - Dataset api - Side outputs

问题描述

Flink是否在Dataset(Batch Api)中支持侧面输出功能?如果不是，从文件加载时如何处理有效和无效记录?

Does Flink supports Side Outputs feature in Dataset(Batch Api) ? If not, how to handle valid and invalid records when loading from file ?

推荐答案

您始终可以执行以下操作:

You can always do something like this:

DataSet<EventOrInvalidRecord> goodAndBadTogether = input.map(new CreateObjectIfPossible())
goodAndBadTogether.filter(new KeepOnlyGood())...
goodAndBadTogether.filter(new KeepOnlyBad())...

在某些情况下，另一个合理的选择是继续使用DataStream API，即使您没有流源也是如此.

Another reasonable option in some cases is to go ahead and use the DataStream API, even if you don't have streaming sources.

这篇关于Apache Flink-数据集API-侧面输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！