Why do we use job.waitForCompletion(true)?

0 votes

Hi. I am new to mapreduce and hadoop and I was going through some code and I came across this line. 

job.waitForCompletion(true)

Why is it used? Is it because internally the code uses threads.

Jul 9 in Big Data Hadoop by Kunal
224 views

1 answer to this question.

0 votes

The main reason for job.waitForCompletion exists is that its method call returns only when the job gets finished, and it returns with its success or failure status which can be used to determine that further steps are to be run or not.

 Actually, the files are split into blocks and each block is executed on a separate node. All the map tasks run in parallel and they are fed to the reducer after they are done. There is no question of synchronization as you would think about in a multi-threaded program. In a multi-threaded program, all the threads are running on the same box and since they share some of the data you have to synchronize them. But here threads are not involved unless you use threads to submit jobs in parallel and wait for their completion. For that, you have to use a job class instance per thread. 

answered Jul 9 by Josh

Related Questions In Big Data Hadoop

0 votes
1 answer

Why do we use job.waitForCompletion(true) ?

The main reason for job.waitForCompletion exists is that ...READ MORE

answered Jul 22 in Big Data Hadoop by Kiran
41 views
0 votes
1 answer

Why we are configuring mapred.job.tracker in YARN?

I really dont know the reason behind ...READ MORE

answered Mar 29, 2018 in Big Data Hadoop by Ashish
• 2,630 points
246 views
0 votes
1 answer

Why do we need Hadoop framework?

The function of Distributes File System is ...READ MORE

answered Apr 9 in Big Data Hadoop by Gitika
• 25,340 points

edited Apr 12 by Gitika 65 views
0 votes
1 answer

Why we use --split by command in Sqoop?

The command --split-by is used to specify the ...READ MORE

answered Apr 11 in Big Data Hadoop by Gitika
• 25,340 points
969 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,710 points
3,324 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,710 points
398 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
16,424 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
1,197 views
+1 vote
1 answer

Why do we use STORE command in pig?

Hey, We use store command to store the ...READ MORE

answered May 7 in Big Data Hadoop by Gitika
• 25,340 points
45 views
0 votes
1 answer

Why should we use "extends Mapper" for Mapreduce code?

The Mapper class belongs to package org.apache.hadoop.mapreduce ...READ MORE

answered Feb 8 in Big Data Hadoop by Omkar
• 67,660 points
53 views