如何将文本文件加载到作为序列文件存储的Hive表中 [英] How to load a text file into a Hive table stored as sequence files

问题描述

我有一个hive表作为序列文件存储。

我需要将一个文本文件加载到此表中。如何将数据加载到此表中？

解决方案

您可以将文本文件加载到文本文件Hive表中，然后插入数据从这张表到您的序列文件中。

以制表符分隔的文件开始：

 ％cat /tmp/input.txt 
ab 
 a2 b2 
  

创建序列文件

  hive>创建存储为sequencefile的表test_sq（k字符串，v字符串）; 
  

尝试加载;如预期的那样，这将失败：

  hive>将数据本地inpath'/tmp/input.txt'加载到表test_sq中; 
  

但是有了这张表：

 分群>创建表test_t（k字符串，v字符串）以'\ t'结尾存储为文本文件的行格式分隔字段; 
  

加载正常：

 分群>将数据本地inpath'/tmp/input.txt'加载到表test_t中; 
确定
配置单元> select * from test_t; 
 OK 
ab 
 a2 b2 
  

现在加载到序列表中从文本表：

 插入表test_sq select * from test_t; 
  

也可以使用覆盖来加载/插入以替换全部。 p>

I have a hive table stored as a sequencefile.

I need to load a text file into this table. How do I load the data into this table?

解决方案

You can load the text file into a textfile Hive table and then insert the data from this table into your sequencefile.

Start with a tab delimited file:

% cat /tmp/input.txt
a       b
a2      b2

create a sequence file

hive> create table test_sq(k string, v string) stored as sequencefile;

try to load; as expected, this will fail:

hive> load data local inpath '/tmp/input.txt' into table test_sq;

But with this table:

hive> create table test_t(k string, v string) row format delimited fields terminated by '\t' stored as textfile;

The load works just fine:

hive> load data local inpath '/tmp/input.txt' into table test_t;
OK
hive> select * from test_t;
OK
a       b
a2      b2

Now load into the sequence table from the text table:

insert into table test_sq select * from test_t;

Can also do load/insert with overwrite to replace all.

这篇关于如何将文本文件加载到作为序列文件存储的Hive表中的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！