如何序列化Hadoop中的List集合对象？ [英] How to serialize List collection object in Hadoop?

问题描述

Writable 接口仅适用于Java基本类型。我有以下类属性。

 私人字符串关键字; 
私人列表<状态>鸣叫; 
私人长查询时间= 0; 
 $ b $公共TweetStatus（字符串键，List< Status> tweets，long queryTime）{
 this.keywords = keys; 
 this.tweets = tweets; 
 this.queryTime = queryTime; 
} 
  

如何序列化 List 对象？

解决方案

Writable 仅限Java原语。

对。基本上你需要将你的对象分解成一系列可以序列化的对象。

因此，从最初的原则来说，要序列化一个列表，你需要序列化的大小该列表，然后序列化列表中的每个元素。这样，当你需要反序列化时，你知道需要反序列化多少个元素。

像这样的东西应该让你在写作（双关语）轨道上：

  class TweetStatusWritable实现Writable {
 private String keywords; 
私人列表<状态>鸣叫; 
私人长查询时间; 
 
 //为上述三个字段添加getters 
 
 public void readFields（DataInput in）{
 this.keywords = in.readUTF（）; 
 int size = in.readInt（）; 
 this.tweets = new List< Status>（）; 
 for（int i = 0; i< size; i ++）{
 Status status = //反序列化状态
的一个实例tweets.add（status）; 
} 
 this.queryTime = in.readLong（）; 
} 
 
 public void write（DataOutput out）{
 out.writeUTF（this.keywords）; 
 out.writeInt（this.tweets.size（））; 
 for（int i = 0; i< this.tweets.size（）; i ++）{
 // serialize tweets [i] to out 
} 
 out.writeLong （queryTime）; 
} 
 
 TweetStatusWritable（
字符串关键字，
 List< Status> tweets，
 long queryTime 
）{
 this .keywords =关键字; 
 this.tweets = tweets; 
 this.queryTime = queryTime; 
} 
} 
  

Is there a way to serialize java collections in Hadoop?

The Writable interface is for Java primitives only. I have following class attributes.

private String keywords;
private List<Status> tweets;
private long queryTime = 0;

public TweetStatus(String keys, List<Status> tweets, long queryTime){
    this.keywords = keys;
    this.tweets = tweets;
    this.queryTime = queryTime;
}

How I can serialize List object?

解决方案

The Writable interface is for Java primitives only.

Right. Basically you need to break down your object into a sequence of objects that you can serialize.

So, from first principles, to serialize a list you need to serialize the size of the list and then serialize each element of the list. This way, when you need to deserialize, you know how many elements you need to deserialize.

Something like this should get you on the write (pun!) track:

class TweetStatusWritable implements Writable {
    private String keywords;
    private List<Status> tweets;
    private long queryTime;

    // add getters for the above three fields

    public void readFields(DataInput in) {
        this.keywords = in.readUTF();
        int size = in.readInt();
        this.tweets = new List<Status>();
        for(int i = 0; i < size; i++) {
            Status status = // deserialize an instance of Status
            tweets.add(status);
        }
        this.queryTime = in.readLong();
    }

    public void write(DataOutput out) {
        out.writeUTF(this.keywords);
        out.writeInt(this.tweets.size());
        for(int i = 0; i < this.tweets.size(); i++) {
             // serialize tweets[i] onto out
        }       
        out.writeLong(queryTime);
    }

    public TweetStatusWritable(
        String keywords,
        List<Status> tweets,
        long queryTime
    ) {
        this.keywords = keywords;
        this.tweets = tweets;
        this.queryTime = queryTime;
    }
}

这篇关于如何序列化Hadoop中的List集合对象？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！