How to solve error caused due to output types of mapper and reducer not matching?

0 votes
job.setOutputValueClass(IntWritable.class);  

sets the o/p value of Mapper or Reducer? When o/p types of mapper and reducer do not match, it gives me an error.

Code:

public static class Map extends Mapper<LongWritable, Text, IntWritable ,Text>

public static class Reduce extends Reducer<IntWritable, Text,IntWritable , IntWritable>

job.setOutputValueClass(IntWritable.class);
Jul 9 in Big Data Hadoop by Karan
17 views

1 answer to this question.

0 votes

job.setOutputValueClass will set the types expected as output from both the map and reduce phases. In your code,

public static class Map extends Mapper<LongWritable, Text, IntWritable ,Text>
public static class Reduce extends Reducer<IntWritable, Text,IntWritable , IntWritable>
job.setOutputValueClass(IntWritable.class);

The mapper output value that is Text does not match with the outputValueClass that is IntWritable.Class  since you are setting the property as job.setOutputValueClass(IntWritable.class); That is the reason it might give you Type mismatch error.

If your Mapper emits different types than the Reducer, you can set the types emitted by the mapper with the JobConf's setMapOutputKeyClass() and setMapOutputValueClass() methods. These implicitly sets the input types expected by the Reducer.

answered Jul 9 by Rishab

Related Questions In Big Data Hadoop

0 votes
1 answer
0 votes
1 answer

How to concatenate hdfs files and store in output file?

You can use a combination of cat and put command. Something ...READ MORE

answered Dec 5, 2018 in Big Data Hadoop by Omkar
• 67,120 points
271 views
0 votes
1 answer

How to compress output of the mapreduce output in Hive?

To compress the output of the MapReduce ...READ MORE

answered May 20 in Big Data Hadoop by Hiran
11 views
0 votes
1 answer

How to see the history of Job output-dir in MapReduce?

Hi, You can use this command to the ...READ MORE

answered Jun 14 in Big Data Hadoop by Gitika
• 19,720 points
18 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,110 points
2,046 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,110 points
196 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,476 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
764 views
0 votes
1 answer

How to set the number of Map & Reduce tasks?

The map tasks created for a job ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by Shubham
• 13,190 points
45 views
0 votes
11 answers

How to check size of HDFS directory?

It is the same syntax. Use the ...READ MORE

answered Jun 6 in Big Data Hadoop by Sowmya
6,046 views