如何在Apache Flink中对数据集进行排序? [英] How to sort a dataset in Apache Flink?

问题描述

我有一个DataSet形式的Tuple Dataset.我希望对字符串字段的整个"数据集进行排序，然后仅在文件中获取Long值.Flink确实提供了排序分区，但这在这里无济于事，因为我需要对数据集进行完全排序.

I have a Tuple Dataset of the form DataSet>. I wish to sort the "entire" Dataset on field String and then get only the Long values in a file. Flink does provide sort-partition but that does not help here as I need to sort the Dataset completely.

推荐答案

如果将并行度设置为<代码> 1 :

DataSet<Tuple2<String, Long>> data = ...
DataSet<Tuple2<String, Long>> sorted = data
  .sortPartition(0, Order.ASCENDING).setParallelism(1); // sort in one partition
DataSet<Long> longs = sorted.map(new LongExtractor());  // map to extract long

这篇关于如何在Apache Flink中对数据集进行排序?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！