PIG:如何删除列名中的“::" [英] PIG: How to remove '::' in the column name

问题描述

我有一个像下面这样的猪关系:

I have a pig relation like below:

FINAL= {input_md5::type: chararray,input_md5::name: chararray,input_md5::id: long,input_md5::age: chararray,test_1:: type: chararray,test_2::name:chararray}

我正在尝试将 input_md5 关系的所有列存储到配置单元表.像所有 input_md5::type: chararray,input_md5::name: chararray,input_md5::id: long,input_md5::age: chararray 一样不带 test_1:: type: chararray,test_2::name:chararray

I am trying to store all columns for input_md5 relation to a hive table. like all input_md5::type: chararray,input_md5::name: chararray,input_md5::id: long,input_md5::age: chararray not taking test_1:: type: chararray,test_2::name:chararray

pig 中是否有任何命令仅过滤 input_md5 的列.类似于以下内容:

is there any command in pig which filters only columns of input_md5.Something like below:

STORE= FOREACH FINAL GENERATE all input_md5::type .我知道猪有:

FOREACH FINAL GENERATE all input_md5::type as type 语法，但我有很多列，所以我不能在我的代码中使用 as.

FOREACH FINAL GENERATE all input_md5::type as type syntax, but i have many columns so I cannot use as in my code.

因为当我尝试:STORE= FOREACH FINAL GENERATE input_md5::type .. bus_input_md5::name;

猪抛出一个错误:

org.apache.hive.hcatalog.common.HCatException:2007:分区架构中的列位置无效:预期列<类型>在位置 1，找到列

提前致谢，

推荐答案

解决了这个问题，下面是修复:

Resolved this issue , below is the fix:

用一些过滤条件创建一个关系如下:

Create a relation with some filter condition as below:

DUMMY_RELATION= FILTER SOURCE_TABLE BY type== '';(我选取了一个名为 type 的列，它可以被表中的任何列过滤，重要的是我们需要它的架构)

DUMMY_RELATION= FILTER SOURCE_TABLE BY type== ''; (I took a column named type ,this can be filtered by any column in the table , all that matters is we need its schema)

FINAL_DATASET= UNION DUMMY_RELATION,SCHEMA_1,SCHEMA_2;

(这个新的 DUMMY_RELATIONn 应该放在联合中的第一名)现在您不再有 :: 运算符并且您的列名将匹配 hive 表的列名，前提是您的源表(到 DUMMY_RELATION)和目标表具有相同的列顺序.

(this new DUMMY_RELATIONn should be placed 1st in the union) Now you no more have :: operator And your column names would match hive table's column names, provided your source table (to DUMMY_RELATION) and target table have same column order.

感谢我自己:)

这篇关于PIG:如何删除列名中的“::"的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！