在Pig脚本中按顺序还是并行加载操作? [英] Load operations in pig script sequential or parallel?

问题描述

我在Pig脚本中有2条load语句，如下所示:

I have 2 load statements in a pig script as below:

a =使用HCatLoader()加载"file1.dat"；

a=load 'file1.dat' using HCatLoader();

b =使用HcatLoader()加载'file2.dat';

b=load 'file2.dat' using HcatLoader();

在这些之后，我分别对a和b进行了一些转换.如果我们以批处理模式运行此Pig脚本，两个文件的加载和转换是顺序发生还是并行发生?我以为Pig会优化此脚本并并行运行两个负载.但不是100％肯定.有人可以对此发表评论吗?

After these, I have some transformations on a and b seperately. If we run this pig script in batch mode, does the load and transformations of both files happen sequentially or in parallel? I was thinking that pig optimises this script and runs both the loads in parallel. But not 100% sure. Can anyone comment on this?

推荐答案

每个加载命令将并行运行，但是pig脚本中的命令将串行运行.

Each load command will run in parallel, but the commands in pig script will run serial.

这篇关于在Pig脚本中按顺序还是并行加载操作?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！