How do spark extra listeners work?

0 votes
Hi. I have read that we can use extra listeners for Spark. Can you tell me more about how this works?
Feb 23 in Apache Spark by Jimmy
328 views

1 answer to this question.

0 votes

Yes. You can use extra listeners by setting it in the spark.extraListeners property. The spark.extraListeners is a comma-separated list of classes that implement SparkListener. when initializing SparkContext, instances of these classes will be created and registered with Spark's listener bus. 

To add extra listeners to Spark, you can set this property while using the spark submit command. Example:

./bin/spark-submit --conf spark.extraListereners <Comma-separated list of listener classes>
answered Feb 23 by Rishi

Related Questions In Apache Spark

0 votes
1 answer

How does partitioning work in Spark?

By default a partition is created for ...READ MORE

answered May 31, 2018 in Apache Spark by nitinrawat895
• 10,710 points
68 views
0 votes
1 answer

How to set extra JVM options for Spark application?

You cans set extra JVM options that ...READ MORE

answered Mar 28 in Apache Spark by Raj
512 views
0 votes
1 answer

How do I turn off INFO Logging in Spark?

Execute this command in the spark directory: cp ...READ MORE

answered Jul 12 in Apache Spark by ravikiran
• 4,560 points
197 views
0 votes
1 answer

How do I access the Map Task ID in Spark?

You can access task information using TaskContext: import org.apache.spark.TaskContext sc.parallelize(Seq[Int](), ...READ MORE

answered Jul 23 in Apache Spark by ravikiran
• 4,560 points
51 views
0 votes
1 answer

How to work with Matrix Multiplication in Apache Spark?

Hey, You can follow this below solution for ...READ MORE

answered Jul 31 in Apache Spark by Gitika
• 25,340 points
605 views
0 votes
1 answer

How to stop messages from being displayed on spark console?

In your log4j.properties file you need to ...READ MORE

answered Apr 24, 2018 in Apache Spark by kurt_cobain
• 9,260 points
1,306 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,710 points
3,295 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,710 points
390 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
16,225 views