How to write Reducer code for Hadoop

0 votes

Hi Team. 

I have written the mapper code for Hadoop with python language. Now I want to write a reducer code. How can I write that?

Oct 16, 2020 in Big Data Hadoop by akhtar
• 38,220 points
123 views

1 answer to this question.

0 votes

Hi@akhtar,

You can create a reducer code for Hadoop in python language. You can see the below example. It will give you some ideas.

#!/usr/bin/python
from operator import itemgetter 
import sys 
current_word = ""
current_count = 0 
word = "" 
# Input takes from standard input for myline in sys.stdin: 
   # Remove whitespace either side 
   myline = myline.strip() 
   # Split the input we got from mapper.py word, 
   count = myline.split('\t', 1) 
   # Convert count variable to integer 
   try: 
      count = int(count) 
   except ValueError: 
      # Count was not a number, so silently ignore this line continue
   if current_word == word: 
   current_count += count 
   else: 
      if current_word: 
         # Write result to standard output print '%s\t%s' % (current_word, current_count) 
      current_count = count
      current_word = word
# Do not forget to output the last word if needed! 
if current_word == word: 
   print '%s\t%s' % (current_word, current_count)
answered Oct 16, 2020 by MD
• 95,320 points

Related Questions In Big Data Hadoop

+1 vote
2 answers

How to authenticate username & password while using Connector for Cloudera Hadoop in Tableau?

Hadoop server installed was kerberos enabled server. ...READ MORE

answered Aug 21, 2018 in Big Data Hadoop by Priyaj
• 58,140 points
697 views
0 votes
1 answer

How to create a project for the first time in Hadoop.?

If you want to learn Hadoop framework ...READ MORE

answered Jul 27, 2018 in Big Data Hadoop by Neha
• 6,300 points
498 views
0 votes
1 answer
0 votes
1 answer

What is Custom partitioner in Hadoop? How to write partition function ?

Don't think that in Hadoop the same ...READ MORE

answered Sep 18, 2018 in Big Data Hadoop by Frankie
• 9,810 points
993 views
0 votes
1 answer
0 votes
1 answer

What is the command to find the free space in HDFS?

You can use dfsadmin which runs a ...READ MORE

answered Apr 29, 2018 in Big Data Hadoop by Shubham
• 13,480 points
1,014 views
0 votes
1 answer

How to find the used cache in HDFS

hdfs dfsadmin -report This command tells fs ...READ MORE

answered May 4, 2018 in Big Data Hadoop by Shubham
• 13,480 points
1,139 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
8,055 views
0 votes
1 answer

How to write mapper code for Hadoop in Python?

Hi@akhtar, Any job in Hadoop must have two ...READ MORE

answered Oct 16, 2020 in Big Data Hadoop by MD
• 95,320 points
126 views
0 votes
2 answers

How to set properties for secondary namenode in Hadoop?

Secondary NameNode in HDFS Secondary NameNode in Hadoop ...READ MORE

answered Mar 31, 2020 in Big Data Hadoop by anonymous
1,137 views