Does Caching stand as the only advantage in Spark compared to Hadoop?

0 votes
I am a beginner in Apache Spark. I see there is a lot of focus drawn to RDDs in Spark and the faster execution is made possible because of the addition of a caching unit.

Is it fair enough to create a whole new framework like Spark just to include a cache in MapReduce Tasks?

Since am a learner, I think I have a lot to learn but can anyone this doubt of mine?
Jul 31 in Big Data Hadoop by nitinrawat895
• 10,710 points
21 views

1 answer to this question.

0 votes
  1. Spark has much lower per job and per task overhead. It gives it ability to be applied to the cases where Hadoop MR is not applicable. It is cases when reply is needed in 1-30 seconds. 
    Low per task overhead makes Spark more efficient for even big jobs with a lot of short tasks. As a very rough estimation - when task takes 1 second Spark will be 2 times more efficient then Hadoop MR.

  2. Spark has lower abstraction then MR - it is graph of computations. As a result it is possible to implement more efficient processing then MR - specifically in cases when sorting is not needed. In other words - in MR we always pay for the sorting, but in Spark - we do not have to.

answered Jul 31 by ravikiran
• 4,560 points

Related Questions In Big Data Hadoop

0 votes
11 answers
0 votes
1 answer

Is it compulsory to have the hadoop user as sudo user?

No, it is not important to have ...READ MORE

answered May 8, 2018 in Big Data Hadoop by Shubham
• 13,300 points
343 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,710 points
3,303 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,710 points
391 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
16,285 views
0 votes
1 answer
0 votes
1 answer

Explain to me the Elasticsearch and Hadoop in a much better manner

I understand your problem, I suggest you download ...READ MORE

answered May 10 in Big Data Hadoop by ravikiran
• 4,560 points
43 views