Pig: Does these jar files run in NodeManager to produce result?

0 votes
By executing Pig Scripts, we get Map-Reduce jar files. Does these jar files run in NodeManager to produce the result?
Jul 10 in Big Data Hadoop by Will
14 views

1 answer to this question.

0 votes
Below is what happens with the map-reduce jars:

After the jar of the job is submitted, now once your job has been submitted the Resource Manager will assign a new application id to this job which will be then passed on to the client.

Client will copy the jar file and other job resources to HDFS. Basically, client submits the job through Resource Manager.

Resource Manager, being master node, allocate the resources needed for the job to run and keeps track of cluster utilization. It also, initiates an application master for each job who is responsible to co-ordinate the job execution.

Application master gets the meta data info from namenode to determine where the blocks (input split) for input are located and then supervises the respective nodemanagers to submit the tasks

Basically, the App Master creates a map task object for each input split, as well as a number of reduce task objects.
answered Jul 10 by Lucy

Related Questions In Big Data Hadoop

0 votes
1 answer

What are the extra files we need to run when we run a Hive action in Oozie?

Hi, Below are the extra files that need ...READ MORE

answered Jun 25 in Big Data Hadoop by Gitika
• 19,720 points
30 views
0 votes
0 answers

How to run Hadoop in Docker containers?

I want to incorporate Hadoop in Docker ...READ MORE

Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 10,110 points
58 views
0 votes
5 answers
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,110 points
2,048 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,110 points
196 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,478 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
764 views
0 votes
7 answers

How to run a jar file in hadoop?

I used this command to run my ...READ MORE

answered Dec 10, 2018 in Big Data Hadoop by Dasinto
4,017 views
0 votes
1 answer

Hadoop Pig: How to include external jar file in PIG?

You can do this: register /local/path/to/Jar_name.jar READ MORE

answered Nov 16, 2018 in Big Data Hadoop by Omkar
• 67,120 points
25 views