Hadoop MapReduce：两个值作为Mapper-Reducer中的关键字 [英] Hadoop MapReduce: Two values as key in Mapper-Reducer

问题描述

我怎样才能建立一个带有两个组件的密钥？原因是我有一个无向图。如果A和B通过通信关联（方向无关），则在两个节点A和B之间存在边缘。该通信有一个数字参数。所以我想实现的是有一个把A和B结合在一起的密钥作为一个集合，以便从A到B和B到A的通信可以被认为是等同的，并且可以被总结为获得统计信息。 b
$ b

说：

AB 5

BA 10

然后键应该在语义上是A或B在一起，所以包含A和B作为键的集合应该具有值5 + 10 = 15。

wordcount示例具有特定单词的关键字。在我的情况下，我想拥有两个组件的关键集合。在地图和缩小阶段，只要A到B或B到A满足，我想总结一下。

Thx！

<除了David的（正确的）答案：如果你的问题必须和图形一起看，那么看看 http://incubator.apache.org/giraph/ 也是。

how can I build a key with two components? The reason for this is I have an undirected graph. There is an edge between two nodes A and B if A and B were associated through a communication (the direction is irrelevant). This communication has a numerical parameter. So what I would like to achieve is to have a key which combines A and B together as a set, so that the communication from A to B and B to A can be considered equivalent and be summed up to get stats

Say:

A B 5

B A 10

The key then should be semantically "A or B together", so that the set containing A and B as key should have the value 5+10=15.

The wordcount example has as key the specific words. In my case, I want to have as key a set with two components. During the map and reduce phases, I would like to sum as long as A to B or B to A satisfies.

Thx!

解决方案

In addition to the (correct) answer by David: If your problem has to with graphs then have a look at http://incubator.apache.org/giraph/ also.

这篇关于Hadoop MapReduce：两个值作为Mapper-Reducer中的关键字的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！