Hadoop: No Such Method Exception

0 votes

My MapReduce code is as follows:

import java.io.IOException;
import java.util.Iterator;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.*;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapred.Reporter;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;

public class MaxTemperature {

public class MaxTemperatureMapper extends Mapper <LongWritable, Text, Text, IntWritable>{
    private static final int MISSING = 9999;

    public void map(LongWritable key, Text Value, Context context, Reporter reporter) throws IOException, InterruptedException{
        String line = Value.toString();
        String year = line.substring(15,19);
        int airTemperature;
        if(line.charAt(87)=='+')
            airTemperature = Integer.parseInt(line.substring(88,92));
        else
            airTemperature = Integer.parseInt(line.substring(87,91));
        String quality = line.substring(92,93);
        if(airTemperature!=MISSING && quality.matches("[01459]"))
            context.write(new Text(year), new IntWritable(airTemperature));
    }
}

public class MaxTemperatureReducer extends Reducer<Text, IntWritable, Text, IntWritable>{
    public void reduce (Text Key, Iterator<IntWritable> Values, Context context, Reporter reporter) throws IOException, InterruptedException{

        int maxValue = Integer.MIN_VALUE;
        while(Values.hasNext())
            maxValue = Math.max(maxValue, Values.next().get());
        context.write(Key, new IntWritable(maxValue));

    }
}

public static void main(String[] args) throws Exception {
    if(args.length!=2){
        System.err.println("Usage: WeatherTemperature  <input path> <output path>");
        System.exit(-1);
    }
    Configuration conf = new Configuration();
    Job job = new Job(conf, "Maximum Temperature Calculator");
    job.setJarByClass(MaxTemperature.class);

    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(IntWritable.class);

    job.setMapperClass(MaxTemperatureMapper.class);
    job.setReducerClass(MaxTemperatureReducer.class);

//  job.setInputFormatClass(TextInputFormat.class);
    //job.setOutputFormatClass(TextOutputFormat.class);
    FileInputFormat.addInputPath(job, new Path(args[0]));
    FileOutputFormat.setOutputPath(job, new Path(args[1]));
    job.waitForCompletion(true);
    }
}

I run the .jar of this program using the following command:

hadoop jar weather.jar MaxTemperature input output

And I'm getting the following error:

12/06/13 00:52:05 INFO mapred.JobClient: Task Id :     attempt_201206121354_0007_m_000000_0, Status : FAILED
java.lang.RuntimeException: java.lang.NoSuchMethodException:      MaxTemperature$MaxTemperatureMapper.<init>()
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:602)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
    at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.NoSuchMethodException: MaxTemperature$MaxTemperatureMapper.<init>()
    at java.lang.Class.getConstructor0(Class.java:2706)
    at java.lang.Class.getDeclaredConstructor(Class.java:1985)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
    ... 7 more

please help

Oct 26, 2018 in Big Data Hadoop by slayer
• 29,040 points

recategorized Oct 26, 2018 by Omkar 50 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

Mapper and reducer classes need to be defined static.

public static class MaxTemperatureMapper extends Mapper<....

public static class MaxTemperatureReducer extends Reducer<....
answered Oct 26, 2018 by Omkar
• 66,910 points

Related Questions In Big Data Hadoop

0 votes
1 answer

“no such file or directory" in case of hadoop fs -ls

The behaviour that you are seeing is ...READ MORE

answered May 9, 2018 in Big Data Hadoop by nitinrawat895
• 9,490 points

edited May 9, 2018 by nitinrawat895 1,826 views
0 votes
1 answer

Hadoop Pipe: shared libraries, No such file or directory error

Add -rpath my.zip/lib to LDFLAGS_HDP. For compilation to work, you ...READ MORE

answered Aug 30, 2018 in Big Data Hadoop by Frankie
• 9,710 points
36 views
0 votes
1 answer
0 votes
1 answer

Hadoop security GroupMappingServiceProvider exception for Spark job via Dataproc API

One of the reason behin you getting ...READ MORE

answered Mar 22, 2018 in Big Data Hadoop by nitinrawat895
• 9,490 points
89 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,490 points
1,845 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,490 points
160 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
9,162 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
657 views
0 votes
1 answer

Hadoop: `.' no such file or directory while installing

Try hadoop fs -mkdir -p /user/[Username] and then run ...READ MORE

answered Nov 2, 2018 in Big Data Hadoop by Omkar
• 66,910 points
47 views
0 votes
1 answer

Hadoop Mapreduce: Class Not Found Exception

Try adding this Job job = new Job(conf, ...READ MORE

answered Oct 29, 2018 in Big Data Hadoop by Omkar
• 66,910 points
269 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.