如何停止在Spark控制台上显示的INFO消息? [英] How to stop INFO messages displaying on spark console?

问题描述

我想停止Spark Shell上出现的各种消息.

I'd like to stop various messages that are coming on spark shell.

我试图编辑log4j.properties文件以停止这些消息.

I tried to edit the log4j.properties file in order to stop these message.

这是log4j.properties

# Define the root logger with appender file
log4j.rootCategory=WARN, console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n

# Settings to quiet third party logs that are too verbose
log4j.logger.org.eclipse.jetty=WARN
log4j.logger.org.eclipse.jetty.util.component.AbstractLifeCycle=ERROR
log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=INFO
log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO

但是消息仍在控制台上显示.

But messages are still getting displayed on the console.

以下是一些示例消息

15/01/05 15:11:45 INFO SparkEnv: Registering BlockManagerMaster
15/01/05 15:11:45 INFO DiskBlockManager: Created local directory at /tmp/spark-local-20150105151145-b1ba
15/01/05 15:11:45 INFO MemoryStore: MemoryStore started with capacity 0.0 B.
15/01/05 15:11:45 INFO ConnectionManager: Bound socket to port 44728 with id = ConnectionManagerId(192.168.100.85,44728)
15/01/05 15:11:45 INFO BlockManagerMaster: Trying to register BlockManager
15/01/05 15:11:45 INFO BlockManagerMasterActor$BlockManagerInfo: Registering block manager 192.168.100.85:44728 with 0.0 B RAM
15/01/05 15:11:45 INFO BlockManagerMaster: Registered BlockManager
15/01/05 15:11:45 INFO HttpServer: Starting HTTP Server
15/01/05 15:11:45 INFO HttpBroadcast: Broadcast server star

如何停止这些操作?

推荐答案

感谢@AkhlD和@Sachin Janani建议对.conf文件进行更改.

Thanks @AkhlD and @Sachin Janani for suggesting changes in .conf file.

以下代码解决了我的问题:

Following code solved my issue:

1)在导入部分添加了import org.apache.log4j.{Level, Logger}

1) Added import org.apache.log4j.{Level, Logger} in import section

2)在创建spark上下文对象之后，即在val sc = new SparkContext(conf)之后添加了以下行:

2) Added following line after creation of spark context object i.e. after val sc = new SparkContext(conf):

val rootLogger = Logger.getRootLogger()
rootLogger.setLevel(Level.ERROR)

这篇关于如何停止在Spark控制台上显示的INFO消息?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！