Does spark streaming provides checkpoint?

0 votes

Hi,

Does Spark streaming provides checkpoint to recovery or fault tolerance?

Thank You

Feb 4 in Apache Spark by akhtar
• 24,690 points
156 views

1 answer to this question.

0 votes

Hi@akhtar,

Yes, Spark streaming uses checkpoint. Checkpoint is the process to make streaming applications resilient to failures. There are mainly two types of checkpoint one is Metadata checkpoint and another one is Data checkpoint.

Metadata checkpoint is used for recovery from a node failure.

Data checkpoint is used for fault tolerance in HDFS.

Thank You

answered Feb 4 by MD
• 55,300 points

Related Questions In Apache Spark

0 votes
1 answer

Why does sortBy transformation trigger a Spark job?

Actually, sortBy/sortByKey depends on RangePartitioner (JVM). So ...READ MORE

answered May 8, 2018 in Apache Spark by kurt_cobain
• 9,320 points
379 views
0 votes
1 answer

Getting error while connecting zookeeper in Kafka - Spark Streaming integration

I guess you need provide this kafka.bootstrap.servers ...READ MORE

answered May 24, 2018 in Apache Spark by Shubham
• 13,450 points
1,274 views
0 votes
1 answer
0 votes
1 answer

How does partitioning work in Spark?

By default a partition is created for ...READ MORE

answered May 31, 2018 in Apache Spark by nitinrawat895
• 10,950 points
220 views
+1 vote
2 answers
0 votes
1 answer

Is it possible to run Apache Spark without Hadoop?

Though Spark and Hadoop were the frameworks designed ...READ MORE

answered May 2, 2019 in Big Data Hadoop by ravikiran
• 4,600 points
233 views
0 votes
1 answer

What do we exactly mean by “Hadoop” – the definition of Hadoop?

The official definition of Apache Hadoop given ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by Shubham
713 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,950 points
5,846 views
0 votes
1 answer

What is the difference between spark streaming and spark structured streaming?

Hi@akhtar Generally, Spark streaming  is used for real time ...READ MORE

answered Feb 4 in Apache Spark by MD
• 55,300 points
566 views
0 votes
1 answer

Is Spark Sql provides indexing to improve processing speed?

Hi@akhtar, There is no concept of indexing in ...READ MORE

answered Feb 4 in Apache Spark by MD
• 55,300 points
109 views