Does Hadoop merge the output files after reduce phase?

0 votes
I am running a mapreduce task and I want to output files to be merged as a single file. Does Hadoop merge the files after the reduce phase? If not, how can I merge the files?
Jan 22 in Big Data Hadoop by digger
• 27,620 points
66 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

No, the files after the reduce phase are not merged by Hadoop. The number of files you get is the same as the number of reduce tasks. To merge the files manually, you can use the following command:

hadoop fs -getmerge /output/dir/on/hdfs/ /desired/local/output/file.txt

answered Jan 22 by Omkar
• 65,850 points

Related Questions In Big Data Hadoop

0 votes
1 answer

Does map/reduce merge output files after reduce phase?

Hey there, instead of doing the file ...READ MORE

answered Sep 25, 2018 in Big Data Hadoop by digger
• 27,620 points
67 views
0 votes
1 answer

How does Hadoop accesses the files which are distributed among different boundaries?

Hadoop's MapReduce function does not work on ...READ MORE

answered May 7 in Big Data Hadoop by ravikiran
• 1,460 points
10 views
0 votes
1 answer

Moving files in Hadoop using the Java API?

I would recommend you to use FileSystem.rename(). ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 12,150 points
528 views
0 votes
1 answer

What does hadoop fs -du command gives as output?

The first value is the size of ...READ MORE

answered Apr 27, 2018 in Big Data Hadoop by Shubham
• 12,150 points
509 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,030 points
1,654 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,030 points
130 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
8,013 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
560 views
0 votes
1 answer

In Hadoop MapReduce, how can i set an Object as the Value for Map output?

Try this and see if it works: public ...READ MORE

answered Nov 20, 2018 in Big Data Hadoop by Omkar
• 65,850 points
18 views
0 votes
1 answer

Hadoop: How to get the column name along with the output in Hive?

You can get the column names by ...READ MORE

answered Nov 20, 2018 in Big Data Hadoop by Omkar
• 65,850 points
73 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.