Hadoop Writable 和 java.io.serialization 有什么联系和区别? [英] What are the connections and differences between Hadoop Writable and java.io.serialization?

问题描述

为了实现Writable接口，可以在Hadoop中序列化对象.那么Hadoop Writable 和java.io.serialization 之间的联系和区别是什么?

To implement Writable interface, object can be serialized in Hadoop. So what are the connections and differences between Hadoop Writable and java.io.serialization?

推荐答案

底层存储差异:

Java 可序列化

Serializable 不假设存储值的类是已知的，并且用它的类标记实例，即.它写入关于对象的元数据，包括类名、字段名和类型，以及它的超类.ObjectOutputStream 和 ObjectInputStream 对此进行了一些优化，以便在第一个之后为类的实例编写 5 字节的句柄.但是带有句柄的对象序列不能被随机访问，因为它们依赖于流状态.这使排序等事情变得复杂.

Serializable does not assume the class of stored values is known and tags instances with its class ie. it writes the metadata about the object, which includes the class name, field names and types, and its superclass. ObjectOutputStream and ObjectInputStream optimize this somewhat, so that 5-byte handles are written for instances of a class after the first. But object sequences with handles cannot be then accessed randomly, since they rely on stream state. This complicates things like sorting.

Hadoop 可写

在定义Writable"时，您知道预期的类.因此 Writables 不会将它们的类型存储在序列化表示中，因为在反序列化时，您知道预期会发生什么.例如.如果输入键是 LongWritable，则要求一个空的 LongWritable 实例从输入数据流中填充自身.由于不需要存储元信息(类名、字段、它们的类型、超类)，这导致二进制文件更加紧凑、直接随机访问和更高的性能.

While defining a "Writable", you know the expected class. So Writables don't store their type in the serialized representation as while deserializing, you know what is expected. eg. if the input key is a LongWritable, so an empty LongWritable instance is asked to populate itself from the input data stream. As no meta info needs to be stored (classname, fields, their types, super classes) is done, this results in considerably more compact binary files, straightforward random access and higher performance.

一些不错的读物:

对于 Java Serializable:

• http://docs.oracle.com/javase/7/docs/platform/serialization/spec/serial-arch.html
• http://www.javaworld.com/javaworld/jw-02-2006/jw-0227-control.html

Hadoop 可写

• http://www.mail-archive.com/[e-mail]/msg00378.html
• http://www.tom-e-white.com/2008/07/rpc-and-serialization-with-hadoop.html

这篇关于Hadoop Writable 和 java.io.serialization 有什么联系和区别?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！