Problem while running MapReduce code.

0 votes

Hi,

I am getting error for classes which are imported. I have written below code. Kindly provide step by step information on creating classed. Shall I create 3 java programs for driver, mapper and reducer?

package AlphabetCountDemo;

import java.util.*;

import java.io.*;


import org.apache.hadoop.conf.Configuration;

import org.apache.hadoop.fs.*;

import org.apache.hadoop.io.*;

import org.apache.hadoop.mapreduce.*;

import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;

import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;


public class AlphabetCountClass {


public static void main(String[] args) throws ClassNotFoundException,IOException,InterruptedException{

  // TODO Auto-generated method stub

  Configuration conf = new Configuration();

  Job job = Job.getInstance(conf, "WordCount");

  job.setMapperClass(AlphabetWordCountMapper.class);

  job.setReducerClass(AlphabetWordCountReducer.class);

  job.setCombinerClass(AlphabetWordCountReducer.class);

  job.setMapOutputKeyClass(IntWritable.class);

  job.setMapOutputValueClass(IntWritable.class);

  job.setOutputKeyClass(IntWritable.class);

  job.setOutputValueClass(IntWritable.class);

  job.setJarByClass(AlphabetWordCount.class);

  FileInputFormat.addInputPath(job, new Path(args[0]));

  FileOutputFormat.setOutputPath(job, new Path(args[1]));

  System.exit(job.waitForCompletion(true) ? 0 : 1);

 }

}

May 23, 2019 in Big Data Hadoop by Haman
102 views

1 answer to this question.

0 votes
You will have to create a Mapper class and a Reducer class, and then when you create a Driver class and add these classes you will not get the errors.

You can create three different classes for Mapper, Reducer and the Driver or else you can create a single Driver class and then create two inner classes for Mapper and Reducer.
answered May 23, 2019 by Rishi

Related Questions In Big Data Hadoop

0 votes
1 answer

UnsupportedClassVersionError while running Mapreduce program.

Seems like the jar file was not ...READ MORE

answered Jan 18, 2019 in Big Data Hadoop by Omkar
• 69,000 points
44 views
0 votes
1 answer

Running into an issue while running the mapred job on the server

Try this command hadoop dfs -put /var/tmp/students.txt / hadoop ...READ MORE

answered Dec 20, 2018 in Big Data Hadoop by Omkar
• 69,000 points
62 views
0 votes
1 answer

Issue While Running Oozie Sample Project

Hey @supriya.  Seems like you have not set ...READ MORE

answered Jan 3, 2019 in Big Data Hadoop by Omkar
• 69,000 points
99 views
0 votes
1 answer

Mapreduce problem with output part file

As the error suggests and you have ...READ MORE

answered Feb 7, 2019 in Big Data Hadoop by Omkar
• 69,000 points
92 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,870 points
4,546 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,870 points
644 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
25,756 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,310 points
1,707 views
0 votes
1 answer

Error running hadoop mapreduce in Python using Hadoop Streaming

Hi As you write mapper and reducer program  ...READ MORE

answered Jan 21 in Big Data Hadoop by anonymous
311 views
–1 vote
1 answer

Not able to execute mapreduce code

Follow the below steps to execute the ...READ MORE

answered Jan 7, 2019 in Big Data Hadoop by Omkar
• 69,000 points
70 views