Getting error in MapReduce job setInputFormatClass

0 votes

I am writing a Mapreduce program. I have to pass the input file in key value pair So when I am writing

job.setInputFormatClass(KeyValueTextInputFormat.class);

the compiler is throwing an error saying that can’t use InputFormat. Can anyone help me out?

 Below is my code:

import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.KeyValueTextInputFormat;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.util.GenericOptionsParser;
import co.raghu.dictionary.Dictionary.AllTranslationsReducer;
import co.raghu.dictionary.Dictionary.WordMapper;

public class MRDriver {

public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
    Configuration conf1 = new Configuration();

        String[] extraArgs = new GenericOptionsParser(conf1, args).getRemainingArgs();

        if (extraArgs.length != 2) {
          System.err.println("Usage: wordcount <input> <output>");
          System.exit(2);

        }

        Job job = new Job(conf1, "dictionary");
        System.out.println("Job= "+job.toString());
        job.setJarByClass(Dictionary.class);
        job.setMapperClass(WordMapper.class);
        job.setReducerClass(AllTranslationsReducer.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(Text.class);
        job.setInputFormatClass(KeyValueTextInputFormat.class);
        FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
        FileOutputFormat.setOutputPath(job, new Path(otherArgs[1]));
        System.exit(job.waitForCompletion(true) ? 0 : 1);
    }
}
Apr 15, 2018 in Big Data Hadoop by coldcode
• 2,080 points
1,835 views

1 answer to this question.

0 votes

In old Hadoop API(i.e. below Hadoop 0.20.0), you have to use org.apache.hadoop.mapred.KeyValueTextInputFormat package.

While in new Hadoop API import KeyValueTextInputFormat.class class from package org.apache.hadoop.mapreduce.lib.input.KeyValueTextInputFormat package.

answered Apr 15, 2018 by Shubham
• 13,490 points

Related Questions In Big Data Hadoop

0 votes
1 answer

Error running hadoop mapreduce in Python using Hadoop Streaming

Hi As you write mapper and reducer program  ...READ MORE

answered Jan 21, 2020 in Big Data Hadoop by anonymous
2,171 views
0 votes
1 answer

Getting error in Hadoop Streaming: Type mismatch in Key from Map

In Hadoop streaming you have to customize ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by coldcode
• 2,080 points
1,003 views
0 votes
1 answer

Getting error in Hadoop: Output file already exist

When you executed your code earlier, you ...READ MORE

answered Apr 19, 2018 in Big Data Hadoop by Shubham
• 13,490 points
8,013 views
0 votes
1 answer

Error in Hadoop Mapreduce

The file that you are referring here ...READ MORE

answered Apr 19, 2018 in Big Data Hadoop by Shubham
• 13,490 points
860 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,558 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,214 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,260 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

Getting error in command hdfs dfs -mkdir

You are getting this error because the ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 13,490 points
2,802 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP