The number of stages in a job is equal to the number of RDDs in DAG however under one of the cgiven conditions the scheduler can truncate the lineage identify it

Question

14)The number of stages in a job is equal to the number of RDDs in DAG. however, under one of the cgiven conditions, the scheduler can truncate the lineage. identify it.

a)  when there is no movement of data from parent RDD
b)  when there is unshuffle in partition
c)  when the RDD is cached or persisted
d)  when the RDD is shuffled

Answer 1

Hi@ritu,

Spark's internal scheduler may truncate the lineage of the RDD graph if an existing RDD has already been persisted in cluster memory or on disk. So the right answer may be option C.