Problem while running MapReduce code

0 votes

Hi,

I am getting error for classes which are imported. I have written below code. Kindly provide step by step information on creating classed. Shall I create 3 java programs for driver, mapper and reducer?

package AlphabetCountDemo;

import java.util.*;

import java.io.*;


import org.apache.hadoop.conf.Configuration;

import org.apache.hadoop.fs.*;

import org.apache.hadoop.io.*;

import org.apache.hadoop.mapreduce.*;

import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;

import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;


public class AlphabetCountClass {


public static void main(String[] args) throws ClassNotFoundException,IOException,InterruptedException{

  // TODO Auto-generated method stub

  Configuration conf = new Configuration();

  Job job = Job.getInstance(conf, "WordCount");

  job.setMapperClass(AlphabetWordCountMapper.class);

  job.setReducerClass(AlphabetWordCountReducer.class);

  job.setCombinerClass(AlphabetWordCountReducer.class);

  job.setMapOutputKeyClass(IntWritable.class);

  job.setMapOutputValueClass(IntWritable.class);

  job.setOutputKeyClass(IntWritable.class);

  job.setOutputValueClass(IntWritable.class);

  job.setJarByClass(AlphabetWordCount.class);

  FileInputFormat.addInputPath(job, new Path(args[0]));

  FileOutputFormat.setOutputPath(job, new Path(args[1]));

  System.exit(job.waitForCompletion(true) ? 0 : 1);

 }

}

May 23, 2019 in Big Data Hadoop by Haman
494 views

1 answer to this question.

0 votes
You will have to create a Mapper class and a Reducer class, and then when you create a Driver class and add these classes you will not get the errors.

You can create three different classes for Mapper, Reducer and the Driver or else you can create a single Driver class and then create two inner classes for Mapper and Reducer.
answered May 23, 2019 by Rishi

Related Questions In Big Data Hadoop

0 votes
1 answer

UnsupportedClassVersionError while running Mapreduce program.

Seems like the jar file was not ...READ MORE

answered Jan 18, 2019 in Big Data Hadoop by Omkar
• 69,210 points
539 views
0 votes
1 answer

Running into an issue while running the mapred job on the server

Try this command hadoop dfs -put /var/tmp/students.txt / hadoop ...READ MORE

answered Dec 20, 2018 in Big Data Hadoop by Omkar
• 69,210 points
348 views
0 votes
1 answer

Issue While Running Oozie Sample Project

Hey @supriya.  Seems like you have not set ...READ MORE

answered Jan 4, 2019 in Big Data Hadoop by Omkar
• 69,210 points
656 views
0 votes
1 answer

Mapreduce problem with output part file

As the error suggests and you have ...READ MORE

answered Feb 7, 2019 in Big Data Hadoop by Omkar
• 69,210 points
745 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,556 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,184 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,202 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,260 views
0 votes
1 answer

Error running hadoop mapreduce in Python using Hadoop Streaming

Hi As you write mapper and reducer program  ...READ MORE

answered Jan 21, 2020 in Big Data Hadoop by anonymous
2,171 views
–1 vote
1 answer

Not able to execute mapreduce code

Follow the below steps to execute the ...READ MORE

answered Jan 7, 2019 in Big Data Hadoop by Omkar
• 69,210 points
465 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP