Why is Spark faster than Hadoop Map Reduce

0 votes
Can anyone explain, why certain programs are faster in Spark that in MapReduce?
Apr 30, 2018 in Apache Spark by Data_Nerd
• 2,340 points

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

Firstly, it's the In-memory computation, if the file is present in HDFS, it takes more time to load data from HDFS, do the processing and store the result back to HDFS (in case there are multiple MR jobs). For Spark, data is stored in the cache memory, and when the final transformation is done (action), only then it is stored in HDFS. This saves a lot of time.

Spark uses lazy evaluation with the help of DAG (Directed Acyclic Graph) of consecutive transformations. This reduces data shuffling and the execution is optimized. 

Lastly, Spark has its own SQL, Machine Learning, Graph and Streaming components unlike Hadoop, where you have to install all the other frameworks separately and data movement between these frameworks is a nasty job. 

Hope it helps.

answered Apr 30, 2018 by shams
• 3,580 points

Related Questions In Apache Spark

0 votes
1 answer

Why is Spark map output compressed?

Spark thinks that it is a good ...READ MORE

answered Feb 23 in Apache Spark by Wasim
0 votes
1 answer

How is Apache Spark different from the Hadoop approach?

In Hadoop MapReduce the input data is ...READ MORE

answered May 7, 2018 in Apache Spark by BD Master
0 votes
1 answer

Is it possible to run Spark and Mesos along with Hadoop?

Yes, it is possible to run Spark ...READ MORE

answered May 29, 2018 in Apache Spark by Data_Nerd
• 2,340 points
0 votes
1 answer

Is it mandatory to start Hadoop to run spark application?

No, it is not mandatory, but there ...READ MORE

answered Jun 14, 2018 in Apache Spark by nitinrawat895
• 9,030 points
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,030 points
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
0 votes
1 answer
+1 vote
3 answers

What is the difference between rdd and dataframes in Apache Spark ?

Comparison between Spark RDD vs DataFrame 1. Release ...READ MORE

answered Aug 27, 2018 in Apache Spark by shams
• 3,580 points
+5 votes
11 answers

Concatenate columns in apache spark dataframe

its late but this how you can ...READ MORE

answered Mar 21 in Apache Spark by anonymous

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.