How fault tolerance is achieved in Apache Spark

Question

Hi,

I have a doubt that Hadoop uses replication factor to achieve fault tolerance so for Apache spark how this is achieved?

Answer 1

Hey,

In Apache Spark, the data storage model is based on RDD.

• RDDs help to achieve fault tolerance through the lineage.
• RDD always has information on how to build from other datasets.
• If any partition of an RDD is lost due to failure, lineage helps build only that particular lost partition.