What is the usage of Configured class in Hadoop programs

0 votes

Most of Hadoop MapReduce programs are like this:

public class MyApp extends Configured Implements Tool {
    @Override
    public int run(String[] args) throws Exception {
        Job job = new Job(getConf());
        /* process command line options */
        return job.waitForCompletion(true) ? 0 : 1;
    }
    public static void main(String[] args) throws Exception {
        int exitCode = ToolRunner.run(new MyApp(), args);
        System.exit(exitCode);
    }
}

What is the usage of Configured? As Tool and Configured both have getConf() and setConf() in common.

Nov 22, 2018 in Big Data Hadoop by Neha
• 6,300 points

edited Nov 22, 2018 by Neha 1,059 views

1 answer to this question.

0 votes

Configured is a default implementation of the Configurable interface - basically its setConfmethod retains a private instance variable to the passed Configuration object and getConf()returns that reference

Tool is an extension of the Configurable interface, providing an addition run(..) method and is used with ToolRunner to parse out command line options (using the GenericOptionsParser) and build a Configuration object which is then passed to the setConf(..) method.

Your main class will typically extend Configured such that the Configurable interface methods required in Tool will be implemented for you.

In general you should be using the ToolRunner utility class to launch your MapReduce jobs as it handles the common task of parsing out command line arguments and building the Configuration object. I'd look at the API Docs for ToolRunner for more info.

answered Nov 22, 2018 by Frankie
• 9,830 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What is the use of sequence file in Hadoop?

Sequence files are binary files containing serialized ...READ MORE

answered Apr 6, 2018 in Big Data Hadoop by Ashish
• 2,650 points
9,185 views
0 votes
12 answers

What is Zookeeper? What is the purpose of Zookeeper in Hadoop Ecosystem?

Hey, Apache Zookeeper says that it is a ...READ MORE

answered Apr 29, 2019 in Big Data Hadoop by Gitika
• 65,910 points
28,268 views
0 votes
1 answer

What is the meaning of Write Ahead Log in Hadoop?

Write Ahead Log (WAL) is a file ...READ MORE

answered Nov 21, 2018 in Big Data Hadoop by Sunil
1,696 views
0 votes
1 answer

What is the use of fsimage in hadoop?

The NameNode stores modifications to the file ...READ MORE

answered Dec 21, 2018 in Big Data Hadoop by Omkar
• 69,210 points
23,894 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,555 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,184 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,200 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,260 views
0 votes
1 answer

What is the Data format and database choices in Hadoop and Spark?

Use Parquet. I'm not sure about CSV ...READ MORE

answered Sep 4, 2018 in Big Data Hadoop by Frankie
• 9,830 points
716 views
0 votes
1 answer

What is the difference between Hadoop MapReduce and built-in MapReduce?

Differences are as follows: Hadoop's MR can be ...READ MORE

answered Sep 11, 2018 in Big Data Hadoop by Frankie
• 9,830 points
1,181 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP