如何在Java代码中使用S3DistCp [英] How to use S3DistCp in java code
本文介绍了如何在Java代码中使用S3DistCp的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我想以语法方式将作业的输出从EMR集群复制到Amazon S3.
I want to copy output of job from EMR cluster to Amazon S3 pro-grammatically.
如何在Java代码中使用S3DistCp
进行相同操作.
How to use S3DistCp
in java code to do the same.
推荐答案
hadoop ToolRunner
可以运行它.
下面是用法示例:
hadoop ToolRunner
can run this.. since S3DistCP extends Tool
Below is the usage example:
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.hadoop.util.ToolRunner;
import com.amazon.external.elasticmapreduce.s3distcp.S3DistCp
public class CustomS3DistCP{
private static final Log log = LogFactory.getLog(CustomS3DistCP.class);
public static void main(String[] args) throws Exception {
log.info("Running with args: " + args);
System.exit(ToolRunner.run(new S3DistCp(), args));
}
您的类路径中必须有s3distcp jar 您可以从Shell脚本调用此程序.
you have to have s3distcp jar in your classpath You can call this program from a shell script.
希望有帮助!
这篇关于如何在Java代码中使用S3DistCp的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文