如何在context.write(k,v)中输出值 [英] How to just output value in context.write(k,v)

查看:866
本文介绍了如何在context.write(k,v)中输出值的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述



但是,如果我这样编码:
context.write(data,null)在我的mapreduce作业中,我只想输出一些行。 ;

程序会抛出java.lang.NullPointerException。



我不想像下面的代码:
context.write(data,new Text());

因为我必须修剪输出文件中每一行的空格。

有没有什么好方法可以解决它?
提前致谢。






对不起,这是我的错。我仔细检查了程序,发现原因是我将Reducer设置为组合器。



如果我不使用组合器,语句
context.write(数据,空);减速机中
正常工作。在输出数据文件中,只有数据行。






从hadoop权威指南中分享NullWritable解释: p>

NullWritable是一种特殊类型的Writable,因为它具有零长度序列化。没有字节
被写入或读取流。它用作占位符;例如,在
MapReduce中,如果您不需要
来使用该位置,则可以将键或值声明为NullWritable,它可以有效地存储常量空值。当你想存储一个值列表时,NullWritable也可以作为SequenceFile中的一个键使用,而不是
到键值对。它是一个不可变的单例:可以通过调用
来获取实例NullWritable.get()。

解决方案

您应该为此,请使用 NullWritable


In my mapreduce job, I just want to output some lines.

But if I code like this: context.write(data, null);

the program will throw java.lang.NullPointerException.

I don't want to code like below: context.write(data, new Text(""));

because I have to trim the blank space in every line in the output files.

Is there any good ways to solve it? Thanks in advance.


Sorry, it's my mistake. I checked the program carefully, found the reason is I set the Reducer as combiner.

If I do not use the combiner, the statement context.write(data, null); in reducer works fine. In the output data file, there is just the data line.


Share the NullWritable explanation from hadoop definitive guide:

NullWritable is a special type of Writable, as it has a zero-length serialization. No bytes are written to, or read from, the stream. It is used as a placeholder; for example, in MapReduce, a key or a value can be declared as a NullWritable when you don’t need to use that position—it effectively stores a constant empty value. NullWritable can also be useful as a key in SequenceFile when you want to store a list of values, as opposed to key-value pairs. It is an immutable singleton: the instance can be retrieved by calling NullWritable.get().

解决方案

You should use NullWritable for this purpose.

这篇关于如何在context.write(k,v)中输出值的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
相关文章
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆