Does Caching stand as the only advantage in Spark compared to Hadoop

0 votes
I am a beginner in Apache Spark. I see there is a lot of focus drawn to RDDs in Spark and the faster execution is made possible because of the addition of a caching unit.

Is it fair enough to create a whole new framework like Spark just to include a cache in MapReduce Tasks?

Since am a learner, I think I have a lot to learn but can anyone this doubt of mine?
Jul 31, 2019 in Big Data Hadoop by nitinrawat895
• 11,380 points
174 views

1 answer to this question.

0 votes
  1. Spark has much lower per job and per task overhead. It gives it ability to be applied to the cases where Hadoop MR is not applicable. It is cases when reply is needed in 1-30 seconds. 
    Low per task overhead makes Spark more efficient for even big jobs with a lot of short tasks. As a very rough estimation - when task takes 1 second Spark will be 2 times more efficient then Hadoop MR.

  2. Spark has lower abstraction then MR - it is graph of computations. As a result it is possible to implement more efficient processing then MR - specifically in cases when sorting is not needed. In other words - in MR we always pay for the sorting, but in Spark - we do not have to.

answered Jul 31, 2019 by ravikiran
• 4,620 points

Related Questions In Big Data Hadoop

0 votes
11 answers
0 votes
1 answer

Is it compulsory to have the hadoop user as sudo user?

No, it is not important to have ...READ MORE

answered May 8, 2018 in Big Data Hadoop by Shubham
• 13,490 points
1,526 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
8,944 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,604 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
80,564 views
0 votes
1 answer

Method to schedule the number of Mappers and Reducers in a Hadoop MapReduce Tsk.

The number of Mapper tasks for a ...READ MORE

answered Apr 25, 2019 in Big Data Hadoop by ravikiran
• 4,620 points
364 views
0 votes
1 answer

Explain to me the Elasticsearch and Hadoop in a much better manner

I understand your problem, I suggest you download ...READ MORE

answered May 10, 2019 in Big Data Hadoop by ravikiran
• 4,620 points
219 views
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP