Can we use different i p and o p format classes in mapreduce code

0 votes

Can we use different i/p and o/p format classes- (say one TextInputFormat and another KeyValueTextInputFormat)?

job.setInputFormatClass(TextInputFormat.class);

job.setOutputFormatClass(TextOutputFormat.class);
Jul 10, 2019 in Big Data Hadoop by Joshua
607 views

1 answer to this question.

0 votes

Yes, InputFormatClass and OutputFormatClass are independent of each other.

InputFormatClass

InputFormat defines how the input files are split up and read in Hadoop.  Initially, the data for a MapReduce task is stored in input files, and input files typically reside in HDFS. Although these files format is arbitrary, line-based log files and binary format can be used. Using InputFormat we define how these input files are split and read. 

OutputFormatClass

The Hadoop Output Format checks the Output-Specification of the job. It determines how RecordWriter implementation is used to write output to output files. As we know, Reducer takes as input a set of an intermediate key-value pair produced by the mapper and runs a reducer function on them to generate output that is again zero or more key-value pairs.  RecordWriter writes these output key-value pairs from the Reducer phase to output files. OutputFormat instances provided by Hadoop are used to write to files on the HDFS or local disk. OutputFormat describes the output-specification for a Map-Reduce job.

answered Jul 10, 2019 by Jimmy

Related Questions In Big Data Hadoop

0 votes
1 answer

Can we use different input and output format classes?

Yes, InputFormatClass and OutputFormatClass are independent of ...READ MORE

answered Jul 22, 2019 in Big Data Hadoop by Jishan
465 views
0 votes
1 answer
0 votes
1 answer

How can I use my host machine’s web browser to check my HDFS services running in the VM?

The sole purpose of the virtual machine ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by Shubham
• 13,490 points
1,094 views
0 votes
1 answer

In Hadoop MapReduce, how can i set an Object as the Value for Map output?

Try this and see if it works: public ...READ MORE

answered Nov 21, 2018 in Big Data Hadoop by Omkar
• 69,210 points
754 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,600 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,206 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,745 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,284 views
0 votes
1 answer

Why should we use "extends Mapper" for Mapreduce code?

The Mapper class belongs to package org.apache.hadoop.mapreduce ...READ MORE

answered Feb 8, 2019 in Big Data Hadoop by Omkar
• 69,210 points
823 views
0 votes
1 answer

Can I use pattern matching in Hive?

Yes, it is possible to use pattern ...READ MORE

answered May 21, 2019 in Big Data Hadoop by Khan
1,180 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP