猪减压后存储数据 [英] Store data after decompression in pig

问题描述

我的文件格式为-

 ({"food":"Tacos", "person":"Alice", "amount":3})
    ({"food":"Tomato Soup", "person":"Sarah", "amount":2})
    ({"food":"Grilled Cheese", "person":"Alex", "amount":5})

我尝试使用以下代码存储它

I tried to store this using the following code

STORE STOCK_A 
    INTO 'default.ash_json_pigtest' 
    USING HCatStorer();

存储的数据如下所示.

 {"food":"Tacos", "person":"Alice", "amount":3}             None    None
    {"food":"Tomato Soup", "person":"Sarah", "amount":2}    None    None
    {"food":"Grilled Cheese", "person":"Alex", "amount":5}  None    None

预期的输出是

    Tacos           Alice   3
    Tomato Soup     Sarah   2
    Grilled Cheese  Alex    5

我该如何实现?预先感谢.

How can I achieve this? Thanks in advance.

推荐答案

您的问题不是如何存储数据，而是如何加载数据.您有一个JSON文件，但是您正在将整个JSON读入一个字段，因此每行仅获得一个字段.当将其保存到HCatalog表中时，您将获得一行包含JSON的1行和两个空字段.

Your problem is not how you store the data, but how you are loading it. You have a JSON file but you are reading the whole JSON into one field, so you get only one field per row. When you save it into your HCatalog table, you get 1 row with the JSON in one field and two null fields.

使用JsonLoader加载数据，而不用PigStorage加载数据，或者使用JsonLoader加载数据:

Instead of loading the data with PigStorage or whatever you are using, load it with JsonLoader:

STOCK_TABLE = LOAD 'your.data' USING JsonLoader('food:chararray, person:chararray, amount:int');

您可以DUMP数据检查现在是否正确:

You can DUMP the data to check that now it's correct:

DUMP STOCK_A;

(Tacos,Alice,3)
(Tomato Soup,Sarah,2)
(Grilled Cheese,Alex,5)

代替:

DUMP STOCK_A;

({"food":"Tacos", "person":"Alice", "amount":3})
({"food":"Tomato Soup", "person":"Sarah", "amount":2})
({"food":"Grilled Cheese", "person":"Alex", "amount":5})

这篇关于猪减压后存储数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！