Does spark streaming provides checkpoint

0 votes

Hi,

Does Spark streaming provides checkpoint to recovery or fault tolerance?

Thank You

Feb 4, 2020 in Apache Spark by akhtar
• 38,170 points
307 views

1 answer to this question.

0 votes

Hi@akhtar,

Yes, Spark streaming uses checkpoint. Checkpoint is the process to make streaming applications resilient to failures. There are mainly two types of checkpoint one is Metadata checkpoint and another one is Data checkpoint.

Metadata checkpoint is used for recovery from a node failure.

Data checkpoint is used for fault tolerance in HDFS.

Thank You

answered Feb 4, 2020 by MD
• 95,160 points

Related Questions In Apache Spark

0 votes
1 answer

Why does sortBy transformation trigger a Spark job?

Actually, sortBy/sortByKey depends on RangePartitioner (JVM). So ...READ MORE

answered May 8, 2018 in Apache Spark by kurt_cobain
• 9,390 points
578 views
0 votes
1 answer

Getting error while connecting zookeeper in Kafka - Spark Streaming integration

I guess you need provide this kafka.bootstrap.servers ...READ MORE

answered May 24, 2018 in Apache Spark by Shubham
• 13,480 points
1,661 views
0 votes
1 answer
0 votes
1 answer

How does partitioning work in Spark?

By default a partition is created for ...READ MORE

answered May 31, 2018 in Apache Spark by nitinrawat895
• 11,380 points
327 views
+1 vote
2 answers
0 votes
1 answer

Is it possible to run Apache Spark without Hadoop?

Though Spark and Hadoop were the frameworks designed ...READ MORE

answered May 2, 2019 in Big Data Hadoop by ravikiran
• 4,620 points
359 views
0 votes
1 answer

What do we exactly mean by “Hadoop” – the definition of Hadoop?

The official definition of Apache Hadoop given ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by Shubham
1,002 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
7,197 views
0 votes
1 answer

What is the difference between spark streaming and spark structured streaming?

Hi@akhtar Generally, Spark streaming  is used for real time ...READ MORE

answered Feb 4, 2020 in Apache Spark by MD
• 95,160 points
1,065 views
0 votes
1 answer

Is Spark Sql provides indexing to improve processing speed?

Hi@akhtar, There is no concept of indexing in ...READ MORE

answered Feb 4, 2020 in Apache Spark by MD
• 95,160 points
156 views