编写用于计数记录数的MApreduce代码 [英] Writing MApreduce code for counting number of records

问题描述

我想编写一个mapreduce代码来计算给定的CSV文件中的记录数。我没有得到在地图上做什么以及该怎么做，减少了我应该如何解决这个问题，任何人都可以提出一些建议？

解决方案

您的映射器必须发出一个固定键（只需使用值为count的Text）固定值1你可以在wordcount例子中看到）。
$ b 然后，简单地使用 LongSumReducer 作为您的缩减程序。

工作将是一个关键计数的记录，价值是您正在寻找的记录数。

您可以选择（戏剧性地）改进性能通过使用与组合器相同的LongSumReducer。

I want to write a mapreduce code for counting number of records in given CSV file.I am not getting what to do in map and what to do in reduce how should I go about solving this can anyone suggest something?

解决方案

Your mapper must emit a fixed key ( just use a Text with the value "count") an a fixed value of 1 (same as you see in the wordcount example).

Then simply use a LongSumReducer as your reducer.

The output of your job will be a record with the key "count" and the value isthe number of records you are looking for.

You have the option of (dramatically!) improving the performance by using the same LongSumReducer as a combiner.

这篇关于编写用于计数记录数的MApreduce代码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！