PIG：如何删除列名中的'::' [英] PIG: How to remove '::' in the column name

问题描述

FINAL = {input_md5 :: type：chararray，input_md5 :: name：chararray，input_md5 :: id：long，input_md5 :: age：chararray，test_1 :: type：chararray，test_2 :: name：chararray}

I我试图将 input_md5 的所有列存储到一个配置单元表。
像所有的 input_md5 ::类型：chararray，input_md5 ::名称：chararray，input_md5 :: ID：长，input_md5 ::年龄：chararray 不采取 test_1 :: type：chararray，test_2 :: name：chararray

input_md5 .Something象下面这样：

STORE = FOREACH FINAL生成所有input_md5 ::类型。
我知道pig有：

FOREACH FINAL GENERATE所有input_md5 :: type都是类型语法，但我有很多列，所以我不能在我的代码中使用作为。

因为当我尝试：
STORE = FOREACH FINAL GENERATE input_md5 :: type .. bus_input_md5 :: name;

Pig引发错误：

org.apache.hive.hcatalog.common.HCatException：2007：分区模式中的列位置无效：预期的列< type>在位置1处找到< input_md5 :: type>

在此先感谢，

用下面的一些过滤条件创建一个关系：

DUMMY_RELATION = FILTER SOURCE_TABLE BY type ==''; （我带了一个名为type的列，在表中的事项的任何列，所有是我们需要其模式）

FINAL_DATASET = UNION DUMMY_RELATION，SCHEMA_1，SCHEMA_2;
（这个新的 DUMMY_RELATION n应该放在工会的第一位）
现在你如果你的源表（对DUMMY_RELATION）和目标表具有相同的列顺序，那么你的列名将匹配hive表的列名。

b
$ b

感谢我自己:)

I have a pig relation like below:

FINAL= {input_md5::type: chararray,input_md5::name: chararray,input_md5::id: long,input_md5::age: chararray,test_1:: type: chararray,test_2::name:chararray}

I am trying to store all columns for input_md5 relation to a hive table. like all input_md5::type: chararray,input_md5::name: chararray,input_md5::id: long,input_md5::age: chararray not taking test_1:: type: chararray,test_2::name:chararray

is there any command in pig which filters only columns of input_md5.Something like below:

STORE= FOREACH FINAL GENERATE all input_md5::type . I know that pig have :

FOREACH FINAL GENERATE all input_md5::type as type syntax, but i have many columns so I cannot use as in my code.

Because when i try: STORE= FOREACH FINAL GENERATE input_md5::type .. bus_input_md5::name;

Pig throws an error:

org.apache.hive.hcatalog.common.HCatException : 2007 : Invalid column position in partition schema : Expected column <type> at position 1, found column <input_md5::type>

Thanks in advance,

解决方案

Resolved this issue , below is the fix:

Create a relation with some filter condition as below:

DUMMY_RELATION= FILTER SOURCE_TABLE BY type== ''; (I took a column named type ,this can be filtered by any column in the table , all that matters is we need its schema)

FINAL_DATASET= UNION DUMMY_RELATION,SCHEMA_1,SCHEMA_2;

(this new DUMMY_RELATIONn should be placed 1st in the union) Now you no more have :: operator And your column names would match hive table's column names, provided your source table (to DUMMY_RELATION) and target table have same column order.

Thanks to myself :)

这篇关于PIG：如何删除列名中的'::'的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！