Pig Batch 模式:如何设置日志记录级别以隐藏 INFO 日志消息? [英] Pig Batch mode: how to set logging level to hide INFO log messages?
问题描述
使用 Apache Pig 版本 0.10.1.21(重新导出).当我执行一个 pig 脚本时,有很多 INFO 日志记录行看起来像这样:
Using Apache Pig version 0.10.1.21 (rexported). When I execute a pig script, there are a lots of INFO logging lines which looks like that:
2013-05-18 14:30:12,810 [Thread-28] INFO org.apache.hadoop.mapred.Task - Task 'attempt_local_0005_r_000000_0' done.
2013-05-18 14:30:18,064 [main] WARN org.apache.pig.tools.pigstats.PigStatsUtil - Failed to get RunningJob for job job_local_0005
2013-05-18 14:30:18,094 [Thread-31] WARN org.apache.hadoop.mapred.JobClient - No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
2013-05-18 14:30:18,114 [Thread-31] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1
2013-05-18 14:30:18,254 [Thread-32] INFO org.apache.hadoop.mapred.Task - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3fcb2dd1
2013-05-18 14:30:18,265 [Thread-32] INFO org.apache.hadoop.mapred.MapTask - io.sort.mb = 10
pig 脚本中是否有 SET 命令或命令行标志来允许日志级别?基本上我想隐藏 [Thread-xx] INFO 消息.只显示警告和错误.我试过命令行调试标志.不幸的是,INFO 消息仍然显示:
Is there a SET command within the pig script or a command line flag to allow the logging level? Basically I would like to hide the [Thread-xx] INFO messages. Only showing WARNING and ERROR. I have tried the command line debug flag. Unfortunately, the INFO messages still show up:
pig -x local -d WARN MyScript.pig
希望有解决方案.在此先感谢您的帮助.
Hope there is a solution. Thanks in advance for any help.
已解决:回答 由 Loran Bendig 提供,设置 log4j.properties.为方便起见,总结在此
SOLVED: Answer by Loran Bendig, set the log4j.properties. Summarized here for convenience
第一步:将log4j配置文件复制到我的pig脚本所在的文件夹中.
Step1: copy the log4j config file to the folder where my pig scripts are located.
cp /etc/pig/conf.dist/log4j.properties log4j_WARN
Step2:编辑 log4j_WARN 文件并确保存在这两行
Step2: Edit log4j_WARN file and make sure these two lines are present
log4j.logger.org.apache.pig=WARN, A
log4j.logger.org.apache.hadoop = WARN, A
Step3:运行 pig 脚本并指示它使用自定义 log4j
Step3: Run pig script and instruct it to use the custom log4j
pig -x local -4 log4j_WARN MyScript.pig
推荐答案
另一个设置也可以是这样的:
Another setting could be also like this:
创建一个名为 nolog.conf 的文件,内容如下
Create a file named nolog.conf, with the following content
log4j.rootLogger=fatal
然后运行pig如下
pig -x local -4 nolog.conf
这篇关于Pig Batch 模式:如何设置日志记录级别以隐藏 INFO 日志消息?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!