Output types of mapper and reducer does not match

0 votes
 job.setOutputValueClass(IntWritable.class);  

sets the o/p value of Mapper or Reducer? When o/p types of mapper and reducer do not match, it gives me an error.

Some snippets from my code:

public static class Map extends Mapper<LongWritable, Text, IntWritable ,Text>

public static class Reduce extends Reducer<IntWritable, Text,IntWritable , IntWritable>

job.setOutputValueClass(IntWritable.class);
Jul 22, 2019 in Big Data Hadoop by Karan
1,099 views

1 answer to this question.

0 votes

job.setOutputValueClass will set the types expected as output from both the map and reduce phases. In your code,

public static class Map extends Mapper<LongWritable, Text, IntWritable ,Text>

public static class Reduce extends Reducer<IntWritable, Text,IntWritable , IntWritable>

job.setOutputValueClass(IntWritable.class);

The mapper output value that is Text does not match with the outputValueClass that is IntWritable.Class since you are setting the property as job.setOutputValueClass(IntWritable.class); That is the reason it might give you Type mismatch error.

If your Mapper emits different types than the Reducer, you can set the types emitted by the mapper with the JobConf's setMapOutputKeyClass() and setMapOutputValueClass() methods. These implicitly sets the input types expected by the Reducer.

answered Jul 22, 2019 by Reena

Related Questions In Big Data Hadoop

0 votes
1 answer

How to solve error caused due to output types of mapper and reducer not matching?

job.setOutputValueClass will set the types expected as ...READ MORE

answered Jul 9, 2019 in Big Data Hadoop by Rishab
201 views
0 votes
1 answer

Setting the Number of Map tasks and Reduce Tasks not Working

The command that you are running is ...READ MORE

answered May 3, 2018 in Big Data Hadoop by Ashish
• 2,650 points
551 views
0 votes
1 answer

Executing mapper and reducer individually

This is what happens: Map reduce framework will ...READ MORE

answered Dec 18, 2018 in Big Data Hadoop by Omkar
• 69,130 points
75 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
6,617 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,043 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
44,893 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
2,497 views
0 votes
1 answer

Unable to run Name node and datanode on Single Node cluster: Does not contain a valid host:port

Add the following properties in the core-site.xml file, it ...READ MORE

answered May 22, 2019 in Big Data Hadoop by Sanam
254 views
0 votes
1 answer

Method to schedule the number of Mappers and Reducers in a Hadoop MapReduce Tsk.

The number of Mapper tasks for a ...READ MORE

answered Apr 25, 2019 in Big Data Hadoop by ravikiran
• 4,620 points
157 views