如何在context.write(k,v)中输出值 [英] How to just output value in context.write(k,v)
问题描述
但是,如果我这样编码:
context.write(data,null)在我的mapreduce作业中,我只想输出一些行。 ;
程序会抛出java.lang.NullPointerException。
我不想像下面的代码:
context.write(data,new Text());
因为我必须修剪输出文件中每一行的空格。
有没有什么好方法可以解决它?
提前致谢。
对不起,这是我的错。我仔细检查了程序,发现原因是我将Reducer设置为组合器。
如果我不使用组合器,语句
context.write(数据,空);减速机中
正常工作。在输出数据文件中,只有数据行。
从hadoop权威指南中分享NullWritable解释: p>
NullWritable是一种特殊类型的Writable,因为它具有零长度序列化。没有字节
被写入或读取流。它用作占位符;例如,在
MapReduce中,如果您不需要
来使用该位置,则可以将键或值声明为NullWritable,它可以有效地存储常量空值。当你想存储一个值列表时,NullWritable也可以作为SequenceFile中的一个键使用,而不是
到键值对。它是一个不可变的单例:可以通过调用
来获取实例NullWritable.get()。
您应该为此,请使用 NullWritable 。
In my mapreduce job, I just want to output some lines.
But if I code like this: context.write(data, null);
the program will throw java.lang.NullPointerException.
I don't want to code like below: context.write(data, new Text(""));
because I have to trim the blank space in every line in the output files.
Is there any good ways to solve it? Thanks in advance.
Sorry, it's my mistake. I checked the program carefully, found the reason is I set the Reducer as combiner.
If I do not use the combiner, the statement context.write(data, null); in reducer works fine. In the output data file, there is just the data line.
Share the NullWritable explanation from hadoop definitive guide:
NullWritable is a special type of Writable, as it has a zero-length serialization. No bytes are written to, or read from, the stream. It is used as a placeholder; for example, in MapReduce, a key or a value can be declared as a NullWritable when you don’t need to use that position—it effectively stores a constant empty value. NullWritable can also be useful as a key in SequenceFile when you want to store a list of values, as opposed to key-value pairs. It is an immutable singleton: the instance can be retrieved by calling NullWritable.get().
You should use NullWritable for this purpose.
这篇关于如何在context.write(k,v)中输出值的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!