在Java中执行PigServer时出错 [英] Error executing PigServer in Java
问题描述
我正在尝试从Java机器上远程运行Pig脚本,因为我已经在下面的代码中编写了
I am trying to run pig scripts remotely from my java machine, for that i have written below code
代码:
import java.io.IOException;
import java.util.Properties;
import org.apache.pig.ExecType;
import org.apache.pig.PigServer;
import org.apache.pig.backend.executionengine.ExecException;
public class Javapig{
public static void main(String[] args) {
try {
Properties props = new Properties();
props.setProperty("fs.default.name", "hdfs://hdfs://192.168.x.xxx:8022");
props.setProperty("mapred.job.tracker", "192.168.x.xxx:8021");
PigServer pigServer = new PigServer(ExecType.MAPREDUCE, props);
runIdQuery(pigServer, "fact");
}
catch(Exception e) {
System.out.println(e);
}
}
public static void runIdQuery(PigServer pigServer, String inputFile) throws IOException {
pigServer.registerQuery("A = load '" + inputFile + "' using org.apache.hive.hcatalog.pig.HCatLoader();");
pigServer.registerQuery("B = FILTER A by category == 'Aller';");
pigServer.registerQuery("DUMP B;");
System.out.println("Done");
}
}
但是在执行时,我遇到了以下错误.
but while executing i am getting below error.
错误
ERROR 4010: Cannot find hadoop configurations in classpath (neither hadoop-site.xml nor core-site.xml was found in the classpath).
我不知道我在做什么错.
I don't know what am i doing wrong.
推荐答案
嗯,自我描述错误...
Well, self describing error...
在类路径中找不到hadoop-site.xml和core-site.xml
neither hadoop-site.xml nor core-site.xml was found in the classpath
您需要在应用程序的类路径中同时使用这两个文件.
You need both of those files in the classpath of your application.
理想情况下,假设您具有Maven结构,则可以从$HADOOP_CONF_DIR
文件夹中获取这些文件,并将其复制到Java的src/main/resources
中.
You ideally would get those from your $HADOOP_CONF_DIR
folder, and you would copy them into your Java's src/main/resources
, assuming you have a Maven structure
此外,对于那些文件,您应该将Configuration
对象用于Hadoop
Also, with those files, you should rather use a Configuration
object for Hadoop
PigServer(ExecType execType, org.apache.hadoop.conf.Configuration conf)
这篇关于在Java中执行PigServer时出错的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!