Pig: Does these jar files run in NodeManager to produce result?

0 votes
By executing Pig Scripts, we get Map-Reduce jar files. Does these jar files run in NodeManager to produce the result?
Jul 10, 2019 in Big Data Hadoop by Will
44 views

1 answer to this question.

0 votes
Below is what happens with the map-reduce jars:

After the jar of the job is submitted, now once your job has been submitted the Resource Manager will assign a new application id to this job which will be then passed on to the client.

Client will copy the jar file and other job resources to HDFS. Basically, client submits the job through Resource Manager.

Resource Manager, being master node, allocate the resources needed for the job to run and keeps track of cluster utilization. It also, initiates an application master for each job who is responsible to co-ordinate the job execution.

Application master gets the meta data info from namenode to determine where the blocks (input split) for input are located and then supervises the respective nodemanagers to submit the tasks

Basically, the App Master creates a map task object for each input split, as well as a number of reduce task objects.
answered Jul 10, 2019 by Lucy

Related Questions In Big Data Hadoop

0 votes
1 answer

What are the extra files we need to run when we run a Hive action in Oozie?

Hi, Below are the extra files that need ...READ MORE

answered Jun 25, 2019 in Big Data Hadoop by Gitika
• 25,950 points
70 views
0 votes
1 answer

How to run Hadoop in Docker containers?

Hi, You can run Hadoop in Docker container. Follow ...READ MORE

answered Jan 24 in Big Data Hadoop by MD
• 2,870 points
241 views
0 votes
5 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
4,149 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,840 points
587 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
22,456 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,310 points
1,553 views
0 votes
7 answers

How to run a jar file in hadoop?

I used this command to run my ...READ MORE

answered Dec 10, 2018 in Big Data Hadoop by Dasinto
7,630 views
0 votes
1 answer

Hadoop Pig: How to include external jar file in PIG?

You can do this: register /local/path/to/Jar_name.jar READ MORE

answered Nov 16, 2018 in Big Data Hadoop by Omkar
• 68,940 points
53 views