Does spark streaming provides checkpoint

0 votes

Hi,

Does Spark streaming provides checkpoint to recovery or fault tolerance?

Thank You

Feb 4, 2020 in Apache Spark by akhtar
• 38,240 points
825 views

1 answer to this question.

0 votes

Hi@akhtar,

Yes, Spark streaming uses checkpoint. Checkpoint is the process to make streaming applications resilient to failures. There are mainly two types of checkpoint one is Metadata checkpoint and another one is Data checkpoint.

Metadata checkpoint is used for recovery from a node failure.

Data checkpoint is used for fault tolerance in HDFS.

Thank You

answered Feb 4, 2020 by MD
• 95,380 points

Related Questions In Apache Spark

0 votes
1 answer

Why does sortBy transformation trigger a Spark job?

Actually, sortBy/sortByKey depends on RangePartitioner (JVM). So ...READ MORE

answered May 8, 2018 in Apache Spark by kurt_cobain
• 9,390 points
915 views
0 votes
1 answer

Getting error while connecting zookeeper in Kafka - Spark Streaming integration

I guess you need provide this kafka.bootstrap.servers ...READ MORE

answered May 24, 2018 in Apache Spark by Shubham
• 13,490 points
2,210 views
0 votes
1 answer
0 votes
1 answer

How does partitioning work in Spark?

By default a partition is created for ...READ MORE

answered May 31, 2018 in Apache Spark by nitinrawat895
• 11,380 points
532 views
+1 vote
2 answers
0 votes
1 answer

Is it possible to run Apache Spark without Hadoop?

Though Spark and Hadoop were the frameworks designed ...READ MORE

answered May 3, 2019 in Big Data Hadoop by ravikiran
• 4,620 points
654 views
0 votes
1 answer

What do we exactly mean by “Hadoop” – the definition of Hadoop?

The official definition of Apache Hadoop given ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by Shubham
1,283 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
9,351 views
0 votes
1 answer

What is the difference between spark streaming and spark structured streaming?

Hi@akhtar Generally, Spark streaming  is used for real time ...READ MORE

answered Feb 4, 2020 in Apache Spark by MD
• 95,380 points
2,541 views
0 votes
1 answer

Is Spark Sql provides indexing to improve processing speed?

Hi@akhtar, There is no concept of indexing in ...READ MORE

answered Feb 4, 2020 in Apache Spark by MD
• 95,380 points
298 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP