Apache Beam:平面地图与地图? [英] Apache Beam : FlatMap vs Map?

问题描述

我想了解我应该在哪种情况下使用 FlatMap 或 Map.文档 对我来说似乎不太清楚.

I want to understand in which scenario that I should use FlatMap or Map. The documentation did not seem clear to me.

我还是不明白我应该在哪种场景下使用FlatMap或Map的转换.

I still do not understand in which scenario I should use the transformation of FlatMap or Map.

谁能给我举个例子，让我明白它们的区别?

Could someone give me an example so I can understand their difference?

我了解 Spark 中 FlatMap 与 Map 的区别，但不确定是否有任何相似之处?

I understand the difference of FlatMap vs Map in Spark, and however not sure if there any similarity?

推荐答案

Beam 中的这些变换与 Spark(Scala 也是如此)完全相同.

These transforms in Beam are exactly same as Spark (Scala too).

一个 Map 变换，maps 从 N 个元素的 PCollection 到另一个 N 个元素的 PCollection.

A Map transform, maps from a PCollection of N elements into another PCollection of N elements.

FlatMap 转换将 N 个元素的 PCollections 映射到 N 个零个或多个元素的集合，然后将这些元素展平成单个 <代码>PCollection.

A FlatMap transform maps a PCollections of N elements into N collections of zero or more elements, which are then flattened into a single PCollection.

举个简单的例子，发生以下情况:

As a simple example, the following happens:

beam.Create([1, 2, 3]) | beam.Map(lambda x: [x, 'any'])
# The result is a collection of THREE lists: [[1, 'any'], [2, 'any'], [3, 'any']]

鉴于:

beam.Create([1, 2, 3]) | beam.FlatMap(lambda x: [x, 'any'])
# The lists that are output by the lambda, are then flattened into a
# collection of SIX single elements: [1, 'any', 2, 'any', 3, 'any']

这篇关于Apache Beam:平面地图与地图?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！