How to enable write ahead logs for receivers

0 votes
I want to input data received by receivers to be written in write-ahead logs so that if there are any driver failures, then recovery would be easy. How can I do this?
Mar 19, 2019 in Apache Spark by Roshan
902 views

1 answer to this question.

0 votes

To enable write-ahead logs, run the following command in the spark shell:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.streaming.receiver.writeAheadLog.enable=true
answered Mar 19, 2019 by Jai

Related Questions In Apache Spark

0 votes
1 answer

How to enable SSL for Spark application?

You can do it dynamically like this: val ...READ MORE

answered Mar 15, 2019 in Apache Spark by Karan
2,618 views
0 votes
1 answer
0 votes
1 answer

How to add third party java jars for use in PySpark?

You can add external jars as arguments ...READ MORE

answered Jul 4, 2018 in Apache Spark by nitinrawat895
• 11,380 points

edited Nov 19, 2021 by Sarfaraz 8,713 views
0 votes
1 answer

How to enable Spark event logging?

To make Spark store the event logs, ...READ MORE

answered Mar 6, 2019 in Apache Spark by Rohit
2,913 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
11,075 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,572 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
109,062 views
0 votes
1 answer

How to change the location of Spark event logs?

You can change the location where you ...READ MORE

answered Mar 6, 2019 in Apache Spark by Rohit
4,574 views
0 votes
1 answer

How to enable dynamic resource allocation in Spark?

To dynamically enable dynamic resource allocation, you ...READ MORE

answered Mar 12, 2019 in Apache Spark by veer
1,782 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP