What are SUCCESS and part-r-00000 files in Hadoop?

0 votes

I run Hadoop on my Virtual Box running CentOS. Whenever I submit any job to Hadoop, always 2 different files are created as output. One is SUCCESS and other is part-r-00000 files. I saw that output always presents in part-r-00000 file, but why SUCCESS file is created along with part-r-00000 file? What is the significance of the files created or is it just randomly created?

Apr 12, 2018 in Big Data Hadoop by Shubham
• 13,300 points
2,472 views

1 answer to this question.

0 votes

Yes, both the files i.e. SUCCESS and part-r-00000 are by-default created.

On the successful completion of a job, the MapReduce runtime creates a _SUCCESS file in the output directory. This may be useful for applications that need to see if a result set is complete just by inspecting HDFS.

The output files are by default named as part-x-yyyyy
where:
1) x is
either 'm' or 'r', depending on whether the job was a map only job, or reduce
2)
yyyyy is the Mapper, or Reducer task number (zero based)

So if a job which has 10 reducers, files generated will have named part-r-00000 to part-r-00009, one for each reducer task.

It is possible to change the default name.
This is all you need to do in the Driver class to change the default of the output file:

job.getConfiguration().set("mapreduce.output.basename", "edureka")

Hope this will answer your question to some extent.

answered Apr 12, 2018 by nitinrawat895
• 10,690 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What are active and passive “NameNodes” in Hadoop?

In HA (High Availability) architecture, we have ...READ MORE

answered Dec 13, 2018 in Big Data Hadoop by Frankie
• 9,810 points
298 views
0 votes
1 answer

What are the site-specific configuration files in Hadoop?

There are different site specific configuration to ...READ MORE

answered Apr 9 in Big Data Hadoop by Gitika
• 25,340 points
42 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
1,117 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,690 points
3,045 views
0 votes
1 answer

How to get started with Hadoop?

Well, hadoop is actually a framework that ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by coldcode
• 2,020 points
95 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
15,001 views
0 votes
1 answer

Is Kafka and Zookeeper are required in a Big Data Cluster?

Apache Kafka is one of the components ...READ MORE

answered Mar 22, 2018 in Big Data Hadoop by nitinrawat895
• 10,690 points
405 views
0 votes
3 answers

What are differences between NameNode and Secondary NameNode?

File metadata information is stored by Namenode ...READ MORE

answered Apr 7 in Big Data Hadoop by anonymous
1,972 views