How to write Reducer code for Hadoop?

0 votes

Hi Team. 

I have written the mapper code for Hadoop with python language. Now I want to write a reducer code. How can I write that?

Oct 16 in Big Data Hadoop by akhtar
• 33,370 points
43 views

1 answer to this question.

0 votes

Hi@akhtar,

You can create a reducer code for Hadoop in python language. You can see the below example. It will give you some ideas.

#!/usr/bin/python
from operator import itemgetter 
import sys 
current_word = ""
current_count = 0 
word = "" 
# Input takes from standard input for myline in sys.stdin: 
   # Remove whitespace either side 
   myline = myline.strip() 
   # Split the input we got from mapper.py word, 
   count = myline.split('\t', 1) 
   # Convert count variable to integer 
   try: 
      count = int(count) 
   except ValueError: 
      # Count was not a number, so silently ignore this line continue
   if current_word == word: 
   current_count += count 
   else: 
      if current_word: 
         # Write result to standard output print '%s\t%s' % (current_word, current_count) 
      current_count = count
      current_word = word
# Do not forget to output the last word if needed! 
if current_word == word: 
   print '%s\t%s' % (current_word, current_count)
answered Oct 16 by MD
• 79,190 points

Related Questions In Big Data Hadoop

+1 vote
2 answers

How to authenticate username & password while using Connector for Cloudera Hadoop in Tableau?

Hadoop server installed was kerberos enabled server. ...READ MORE

answered Aug 21, 2018 in Big Data Hadoop by Priyaj
• 57,700 points
483 views
0 votes
1 answer

How to create a project for the first time in Hadoop.?

If you want to learn Hadoop framework ...READ MORE

answered Jul 26, 2018 in Big Data Hadoop by Neha
• 6,300 points
297 views
0 votes
1 answer
0 votes
1 answer

What is Custom partitioner in Hadoop? How to write partition function ?

Don't think that in Hadoop the same ...READ MORE

answered Sep 18, 2018 in Big Data Hadoop by Frankie
• 9,810 points
651 views
0 votes
1 answer
0 votes
1 answer

What is the command to find the free space in HDFS?

You can use dfsadmin which runs a ...READ MORE

answered Apr 29, 2018 in Big Data Hadoop by Shubham
• 13,450 points
670 views
0 votes
1 answer

How to find the used cache in HDFS

hdfs dfsadmin -report This command tells fs ...READ MORE

answered May 4, 2018 in Big Data Hadoop by Shubham
• 13,450 points
754 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,950 points
6,347 views
0 votes
1 answer

How to write mapper code for Hadoop in Python?

Hi@akhtar, Any job in Hadoop must have two ...READ MORE

answered Oct 16 in Big Data Hadoop by MD
• 79,190 points
38 views
0 votes
2 answers

How to set properties for secondary namenode in Hadoop?

Secondary NameNode in HDFS Secondary NameNode in Hadoop ...READ MORE

answered Mar 31 in Big Data Hadoop by anonymous
384 views