How to stop messages from being displayed on spark console?

0 votes

I'm receiving various log msgs on spark shell and I want to remove them. On a website, I came to know that it can be done by editing the file

Here are the contents of

# Define the root logger with appender file
log4j.rootCategory=WARN, console
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n

# Settings to quiet third party logs that are too verbose$exprTyper=INFO$SparkILoopInterpreter=INFO
Apr 24, 2018 in Apache Spark by Ashish
• 2,630 points

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

In your file you need to replace this line 

log4j.rootCategory=INFO, console


log4j.rootCategory=ERROR, console.

Hope it helps
answered Apr 24, 2018 by kurt_cobain
• 9,240 points

Related Questions In Apache Spark

0 votes
1 answer

How to stop INFO messages displaying on Spark console?

Just do the following: Edit your conf/ file ...READ MORE

answered Aug 21, 2018 in Apache Spark by nitinrawat895
• 9,610 points
0 votes
1 answer

How to save and retrieve the Spark RDD from HDFS?

You can save the RDD using saveAsObjectFile and saveAsTextFile method. ...READ MORE

answered May 29, 2018 in Apache Spark by Shubham
• 12,890 points
0 votes
1 answer

How to delay live entity updates on Spark ?

You can do this by increasing the ...READ MORE

answered Mar 6 in Apache Spark by Rohit
0 votes
1 answer

How is Apache Spark different from the Hadoop approach?

In Hadoop MapReduce the input data is ...READ MORE

answered May 7, 2018 in Apache Spark by BD Master
0 votes
1 answer
+1 vote
2 answers

Execute Pig Script from Grunt Shell

From your current directory run  pig -x local Then ...READ MORE

answered Oct 25, 2018 in Big Data Hadoop by Kunal
0 votes
1 answer
0 votes
2 answers

ansible-command not found

Use some other variable instead of PATH. READ MORE

answered Apr 23 in Ansible by Vismaya
0 votes
1 answer

Efficient way to read specific columns from parquet file in spark

As parquet is a column based storage ...READ MORE

answered Apr 20, 2018 in Apache Spark by kurt_cobain
• 9,240 points
0 votes
1 answer

How to get Spark dataset metadata?

There are a bunch of functions that ...READ MORE

answered Apr 26, 2018 in Apache Spark by kurt_cobain
• 9,240 points

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.