How do spark extra listeners work?

0 votes
Hi. I have read that we can use extra listeners for Spark. Can you tell me more about how this works?
Feb 23, 2019 in Apache Spark by Jimmy
436 views

1 answer to this question.

0 votes

Yes. You can use extra listeners by setting it in the spark.extraListeners property. The spark.extraListeners is a comma-separated list of classes that implement SparkListener. when initializing SparkContext, instances of these classes will be created and registered with Spark's listener bus. 

To add extra listeners to Spark, you can set this property while using the spark submit command. Example:

./bin/spark-submit --conf spark.extraListereners <Comma-separated list of listener classes>
answered Feb 23, 2019 by Rishi

Related Questions In Apache Spark

0 votes
1 answer

How does partitioning work in Spark?

By default a partition is created for ...READ MORE

answered May 31, 2018 in Apache Spark by nitinrawat895
• 10,840 points
90 views
0 votes
1 answer

How to set extra JVM options for Spark application?

You cans set extra JVM options that ...READ MORE

answered Mar 28, 2019 in Apache Spark by Raj
730 views
0 votes
1 answer

How do I turn off INFO Logging in Spark?

Execute this command in the spark directory: cp ...READ MORE

answered Jul 12, 2019 in Apache Spark by ravikiran
• 4,600 points
516 views
0 votes
1 answer

How do I access the Map Task ID in Spark?

You can access task information using TaskContext: import org.apache.spark.TaskContext sc.parallelize(Seq[Int](), ...READ MORE

answered Jul 23, 2019 in Apache Spark by ravikiran
• 4,600 points
75 views
0 votes
1 answer

How to work with Matrix Multiplication in Apache Spark?

Hey, You can follow this below solution for ...READ MORE

answered Jul 31, 2019 in Apache Spark by Gitika
• 25,460 points
1,350 views
0 votes
1 answer

How to stop messages from being displayed on spark console?

In your log4j.properties file you need to ...READ MORE

answered Apr 24, 2018 in Apache Spark by kurt_cobain
• 9,290 points
1,643 views
+1 vote
1 answer
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
3,915 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
539 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
20,819 views