Hadoop的：在一个键值对原语的值的数组 [英] Hadoop: Array of primitives as value in a key value pair

问题描述

我已经问了一个很resemblant问题在previous线程<一个href=\"http://stackoverflow.com/questions/9342243/hadoop-how-can-i-have-an-array-of-doubles-as-a-value-in-a-key-value-pair\">Hadoop:我怎么能有双打的一个键值对数组作为价值？。

I have asked a very resemblant question in a previous thread Hadoop: How can i have an array of doubles as a value in a key-value pair? .

我的问题是，我想通过一个双数组作为值从地图中减少的阶段。我得到的答案是序列化，转换为文本，它传递给减速和反序列化。这是一个很好的解决方案，但它像序列化和反序列化两次。

My problem is that i want to pass a double array as value from map to reduce phase. The answer i got was to serialize, convert to Text, pass it to the reducer and deserialize. This is a fine solution but its like serializing and deserializing it twice.

ArrayWritable只接受实现可写像FloatWritable比如类型。因此，另一种解决方案是我的双打数组转换为DoubleWritables的数组。但是，这需要一些时间太长和Writables是一个非常昂贵的资源。是不是有喜欢ArrayWritable阵列=新ArrayWritable（Double.class）一个非常简单的解决方案？

ArrayWritable only accepts types that implement Writable like FloatWritable for example. So another solution is to convert my array of doubles to an array of DoubleWritables. But this requires some time too and Writables are a very expensive resource. Isn't there a very simple solution like ArrayWritable array=new ArrayWritable(Double.class) ???

推荐答案

只是实现自己的可写接口。

例如，

public class DoubleArrayWritable implements Writable {
    private double[] data;

    public DoubleArrayWritable() {

    }

    public DoubleArrayWritable(double[] data) {
        this.data = data;
    }

    public double[] getData() {
        return data;
    }

    public void setData(double[] data) {
        this.data = data;
    }

    public void write(DataOutput out) throws IOException {
        int length = 0;
        if(data != null) {
            length = data.length;
        }

        out.writeInt(length);

        for(int i = 0; i < length; i++) {
            out.writeDouble(data[i]);
        }
    }

    public void readFields(DataInput in) throws IOException {
        int length = in.readInt();

        data = new double[length];

        for(int i = 0; i < length; i++) {
            data[i] = in.readDouble();
        }
    }
}

这篇关于Hadoop的：在一个键值对原语的值的数组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！