Pig Does these jar files run in NodeManager to produce result

0 votes
By executing Pig Scripts, we get Map-Reduce jar files. Does these jar files run in NodeManager to produce the result?
Jul 10, 2019 in Big Data Hadoop by Will
115 views

1 answer to this question.

0 votes
Below is what happens with the map-reduce jars:

After the jar of the job is submitted, now once your job has been submitted the Resource Manager will assign a new application id to this job which will be then passed on to the client.

Client will copy the jar file and other job resources to HDFS. Basically, client submits the job through Resource Manager.

Resource Manager, being master node, allocate the resources needed for the job to run and keeps track of cluster utilization. It also, initiates an application master for each job who is responsible to co-ordinate the job execution.

Application master gets the meta data info from namenode to determine where the blocks (input split) for input are located and then supervises the respective nodemanagers to submit the tasks

Basically, the App Master creates a map task object for each input split, as well as a number of reduce task objects.
answered Jul 10, 2019 by Lucy

Related Questions In Big Data Hadoop

0 votes
1 answer

What are the extra files we need to run when we run a Hive action in Oozie?

Hi, Below are the extra files that need ...READ MORE

answered Jun 25, 2019 in Big Data Hadoop by Gitika
• 65,870 points
143 views
0 votes
1 answer

How to run Hadoop in Docker containers?

Hi, You can run Hadoop in Docker container. Follow ...READ MORE

answered Jan 24, 2020 in Big Data Hadoop by MD
• 95,060 points
550 views
0 votes
5 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
6,860 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,098 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
48,471 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
2,602 views
0 votes
7 answers

How to run a jar file in hadoop?

I used this command to run my ...READ MORE

answered Dec 10, 2018 in Big Data Hadoop by Dasinto
13,892 views
0 votes
1 answer

Hadoop Pig: How to include external jar file in PIG?

You can do this: register /local/path/to/Jar_name.jar READ MORE

answered Nov 16, 2018 in Big Data Hadoop by Omkar
• 69,090 points
251 views