Passing keys and values to the reducers during a standard sort and shuffle phase of MapReduce

0 votes
I am working with MapReduce and I want to know that how are keys and values presented and passed to the reducers during a standard sort and shuffle phase of MapReduce?

Can someone help!
Jul 27, 2018 in Big Data Hadoop by Meci Matt
• 9,460 points
2,119 views

1 answer to this question.

0 votes

Let me explain you the whole scenario

Reducer has 3 primary phases: 
 
1. Shuffle The Reducer copies the sorted output from each Mapper using HTTP across the network. 
 
2. Sort The framework merge sorts Reducer inputs by keys (since different Mappers may have output the same key). 
 
The shuffle and sort phases occur simultaneously i.e. while outputs are being fetched they are merged. 
 
SecondarySort To achieve a secondary sort on the values returned by the value iterator, the application should extend the key with the secondary key and define a grouping comparator. The keys will be sorted using the entire key, but will be grouped using the grouping comparator to decide which keys and values are sent in the same call to reduce. 
 
3. Reduce In this phase the reduce(Object, Iterable, Context) method is called for each <key, (collection of values)> in the sorted inputs. 
 
The output of the reduce task is typically written to a RecordWriter via TaskInputOutputContext.write(Object, Object). 
 
The output of the Reducer is not re-sorted. 
Hope this will help!

answered Jul 27, 2018 by nitinrawat895
• 11,380 points

Related Questions In Big Data Hadoop

0 votes
1 answer

Method to schedule the number of Mappers and Reducers in a Hadoop MapReduce Tsk.

The number of Mapper tasks for a ...READ MORE

answered Apr 25, 2019 in Big Data Hadoop by ravikiran
• 4,620 points
624 views
0 votes
1 answer
0 votes
1 answer

How to print the content of a file in console present in HDFS?

Yes, you can use hdfs dfs command ...READ MORE

answered Apr 19, 2018 in Big Data Hadoop by Shubham
• 13,490 points
5,376 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,283 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,599 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,731 views
0 votes
1 answer
0 votes
1 answer

What happens in a MapReduce job when you set the number of reducers to one?

If you set number of reducers as ...READ MORE

answered Jul 31, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
1,825 views
0 votes
2 answers
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP