使用PIG读取XML [英] Reading XML using PIG
本文介绍了使用PIG读取XML的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我正在尝试使用PIG从xml文件读取数据,但是输出不完整.
I am trying to read the data from the xml file using PIG but I am getting incomplete output.
输入文件-
<document>
<url>htp://www.abc.com/</url>
<category>Sports</category>
<usercount>120</usercount>
<reviews>
<review>good site</review>
<review>This is Avg site</review>
<review>Bad site</review>
</reviews>
</document>
我正在使用的代码是:
register 'Desktop/piggybank-0.11.0.jar';
A = load 'input3' using org.apache.pig.piggybank.storage.XMLLoader('document') as (data:chararray);
B = foreach A GENERATE FLATTEN(REGEX_EXTRACT_ALL(data,'(?s)<document>.*?<url>([^>]*?)</url>.*?<category>([^>]*?)</category>.*?<usercount>([^>]*?)</usercount>.*?<reviews>.*?<review>\\s*([^>]*?)\\s*</review>.*?</reviews>.*?</document>')) as (url:chararray,catergory:chararray,usercount:int,review:chararray);
我得到的输出是:
(htp://www.abc.com/,Sports,120,good site)
输出不完整.有人可以帮我解决我所缺少的吗?
which is incomplete output.Can someone please help on what I am missing?
推荐答案
呵呵!终于使用cross
使它工作了.我正在使用XPath
,如果需要,您可以使用正则表达式.我发现,XPath方式比正则表达式更简单,更干净.我想,您也可以看到它.不要忘记用XML替换testXML.xml
.
huh!! Finally got it working using cross
. I'm using XPath
, you can use regex if you want. I find, XPath way to be easier and cleaner than regex. I guess, you can see it too. Don't forget to replace the testXML.xml
with your XML.
XPath方式:
DEFINE XPath org.apache.pig.piggybank.evaluation.xml.XPath();
A = LOAD 'testXML.xml' using org.apache.pig.piggybank.storage.XMLLoader('document') as (x:chararray);
B = FOREACH A GENERATE XPath(x, 'document/url'), XPath(x, 'document/category'), XPath(x, 'document/usercount');
C = LOAD 'testXML.xml' using org.apache.pig.piggybank.storage.XMLLoader('review') as (review:chararray);
D = FOREACH C GENERATE XPath(review,'review');
E = cross B,D;
dump E;
正则表达式方式:
A = LOAD 'testXML.xml' using org.apache.pig.piggybank.storage.XMLLoader('document') as (x:chararray);
B = FOREACH A GENERATE FLATTEN(REGEX_EXTRACT_ALL(x,'(?s)<document>.*?<url>([^>]*?)</url>.*?<category>([^>]*?)</category>.*?<usercount>([^>]*?)</usercount>.*?</document>')) as (url:chararray,catergory:chararray,usercount:int);
C = LOAD 'testXML.xml' using org.apache.pig.piggybank.storage.XMLLoader('review') as (review:chararray);
D = FOREACH C GENERATE FLATTEN(REGEX_EXTRACT_ALL(review,'<review>([^>]*?)</review>'));
E = cross B,D;
dump E;
输出:
(htp://www.abc.com/,Sports,120,Bad site)
(htp://www.abc.com/,Sports,120,This is Avg site)
(htp://www.abc.com/,Sports,120,good site)
这不是您所期望的吗? ;)
Isn't that you were expecting? ;)
这篇关于使用PIG读取XML的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文