编写 MApreduce 代码以计算记录数 [英] Writing MApreduce code for counting number of records

问题描述

我想编写一个 mapreduce 代码来计算给定 CSV 文件中的记录数.我不知道在 map 中做什么以及在 reduce 中做什么我应该如何解决这个问题，谁能提出一些建议?

I want to write a mapreduce code for counting number of records in given CSV file.I am not getting what to do in map and what to do in reduce how should I go about solving this can anyone suggest something?

推荐答案

您的映射器必须发出一个固定键(只需使用值为count"的 Text)一个固定值 1(与您在 wordcount 中看到的相同)示例).

Your mapper must emit a fixed key ( just use a Text with the value "count") an a fixed value of 1 (same as you see in the wordcount example).

然后只需使用 LongSumReducer 作为你的减速器.

Then simply use a LongSumReducer as your reducer.

您的工作的输出将是一条带有键count"的记录，值是您要查找的记录数.

The output of your job will be a record with the key "count" and the value isthe number of records you are looking for.

您可以选择(显着！)通过使用相同的 LongSumReducer 作为组合器来提高性能.

You have the option of (dramatically!) improving the performance by using the same LongSumReducer as a combiner.

这篇关于编写 MApreduce 代码以计算记录数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！