是否可以在配置单元外部表中压缩json? [英] Is it possible to compress json in hive external table?

问题描述

我想知道如何在配置单元外部表中压缩json数据.如何做呢?我创建了这样的外部表:

I want to know how to compress json data in hive external table. How can it be done? I have created external table like this:

 CREATE EXTERNAL TABLE tweets (
id BIGINT,created_at STRING,source STRING,favorited BOOLEAN
)ROW FORMAT SERDE "com.cloudera.hive.serde.JSONSerDe" LOCATION "/user/cloudera/tweets";

我已经设置了压缩属性

set mapred.output.compress=true;
set hive.exec.compress.output=true;
set mapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec;
set io.compression.codecs=org.apache.hadoop.io.compress.GzipCodec;

输入文件:测试

{ "id": 596344698102419451, "created_at": "MonApr0101: 32: 06+00002013", "source": "blank", "favorited": false }

之后，我将json文件加载到hdfs位置"/user/cloudera/tweets".

after that i have load my json file into hdfs location "/user/cloudera/tweets".

但未压缩.

能否让我知道如何在蜂巢外部表中进行压缩?有人可以帮我在蜂巢外部表中压缩吗?

Can you please let me know how to do compression in hive external table ? Can someone help me to compress in hive external table?

先谢谢了.

推荐答案

只需gzip文件，然后将它们(* .gz)放在表位置

Just gzip your files and put them as is (*.gz) into the table location

这篇关于是否可以在配置单元外部表中压缩json?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！