Getting error in MapReduce job.setInputFormatClass

0 votes

I am writing a Mapreduce program. I have to pass the input file in key value pair So when I am writing

job.setInputFormatClass(KeyValueTextInputFormat.class);

the compiler is throwing an error saying that can’t use InputFormat. Can anyone help me out?

 Below is my code:

import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.KeyValueTextInputFormat;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.util.GenericOptionsParser;
import co.raghu.dictionary.Dictionary.AllTranslationsReducer;
import co.raghu.dictionary.Dictionary.WordMapper;

public class MRDriver {

public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
    Configuration conf1 = new Configuration();

        String[] extraArgs = new GenericOptionsParser(conf1, args).getRemainingArgs();

        if (extraArgs.length != 2) {
          System.err.println("Usage: wordcount <input> <output>");
          System.exit(2);

        }

        Job job = new Job(conf1, "dictionary");
        System.out.println("Job= "+job.toString());
        job.setJarByClass(Dictionary.class);
        job.setMapperClass(WordMapper.class);
        job.setReducerClass(AllTranslationsReducer.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(Text.class);
        job.setInputFormatClass(KeyValueTextInputFormat.class);
        FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
        FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
        System.exit(job.waitForCompletion(true) ? 0 : 1);
    }
}
Apr 15, 2018 in Big Data Hadoop by coldcode
• 2,040 points
442 views

1 answer to this question.

0 votes

In old Hadoop API(i.e. below Hadoop 0.20.0), you have to use org.apache.hadoop.mapred.KeyValueTextInputFormat package.

While in new Hadoop API import KeyValueTextInputFormat.class class from package org.apache.hadoop.mapreduce.lib.input.KeyValueTextInputFormat package.

answered Apr 15, 2018 by Shubham
• 13,350 points

Related Questions In Big Data Hadoop

0 votes
0 answers

Error running hadoop mapreduce in Python using Hadoop Streaming

I was trying a sample mapredyce code ...READ MORE

Apr 2, 2018 in Big Data Hadoop by nitinrawat895
• 10,760 points
189 views
0 votes
1 answer

Getting error in Hadoop Streaming: Type mismatch in Key from Map

In Hadoop streaming you have to customize ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by coldcode
• 2,040 points
282 views
0 votes
1 answer

Getting error in Hadoop: Output file already exist

When you executed your code earlier, you ...READ MORE

answered Apr 19, 2018 in Big Data Hadoop by Shubham
• 13,350 points
1,748 views
0 votes
1 answer

Error in Hadoop Mapreduce

The file that you are referring here ...READ MORE

answered Apr 19, 2018 in Big Data Hadoop by Shubham
• 13,350 points
122 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,760 points
3,551 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
18,188 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
1,318 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

Getting error in command hdfs dfs -mkdir

You are getting this error because the ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 13,350 points
506 views