How do spark extra listeners work?

0 votes
Hi. I have read that we can use extra listeners for Spark. Can you tell me more about how this works?
Feb 23, 2019 in Apache Spark by Jimmy
586 views

1 answer to this question.

0 votes

Yes. You can use extra listeners by setting it in the spark.extraListeners property. The spark.extraListeners is a comma-separated list of classes that implement SparkListener. when initializing SparkContext, instances of these classes will be created and registered with Spark's listener bus. 

To add extra listeners to Spark, you can set this property while using the spark submit command. Example:

./bin/spark-submit --conf spark.extraListereners <Comma-separated list of listener classes>
answered Feb 23, 2019 by Rishi

Related Questions In Apache Spark

0 votes
1 answer

How does partitioning work in Spark?

By default a partition is created for ...READ MORE

answered May 31, 2018 in Apache Spark by nitinrawat895
• 10,870 points
123 views
0 votes
1 answer

How to set extra JVM options for Spark application?

You cans set extra JVM options that ...READ MORE

answered Mar 28, 2019 in Apache Spark by Raj
959 views
0 votes
1 answer

How do I turn off INFO Logging in Spark?

Execute this command in the spark directory: cp ...READ MORE

answered Jul 12, 2019 in Apache Spark by ravikiran
• 4,600 points
863 views
0 votes
1 answer

How do I access the Map Task ID in Spark?

You can access task information using TaskContext: import org.apache.spark.TaskContext sc.parallelize(Seq[Int](), ...READ MORE

answered Jul 23, 2019 in Apache Spark by ravikiran
• 4,600 points
98 views
0 votes
1 answer

How to work with Matrix Multiplication in Apache Spark?

Hey, You can follow this below solution for ...READ MORE

answered Jul 31, 2019 in Apache Spark by Gitika
• 26,270 points
1,745 views
0 votes
1 answer

How to stop messages from being displayed on spark console?

In your log4j.properties file you need to ...READ MORE

answered Apr 24, 2018 in Apache Spark by kurt_cobain
• 9,310 points
2,069 views
+1 vote
1 answer
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,870 points
4,519 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,870 points
640 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
25,459 views