Does Hadoop merge the output files after reduce phase

Question

I am running a mapreduce task and I want to output files to be merged as a single file. Does Hadoop merge the files after the reduce phase? If not, how can I merge the files?

Answer 1

No, the files after the reduce phase are not merged by Hadoop. The number of files you get is the same as the number of reduce tasks. To merge the files manually, you can use the following command:

hadoop fs -getmerge /output/dir/on/hdfs/ /desired/local/output/file.txt