ApplicationMaster in YARN Architecture

0 votes

I have recently started learning Hadoop and I am finding it difficult the overall architecture or job execution flow w.r.t. YARN. I was following the official documentation on YARN where I found that:

ApplicationMaster has the responsibility of negotiating appropriate resource containers from the Scheduler (ResourceManager)

and ApplicationsManager is responsible for negotiating the first container for executing the ApplicationMaster.

Link : http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html

Following are the things that I am confused with:

  1. The documentation says that ApplicationMaster has the responsilibility to request ResourceManager for Container. Now, who is responsible for providing the container for ApplicationMaster in the first place?
  2. How does YARN knows about creating a container for ApplicationMaster?
May 7, 2018 in Big Data Hadoop by Atul
• 10,240 points
192 views

1 answer to this question.

0 votes

Let me help you with your confusion:

  1. YARN client submits the application to ResourceManager, providing it all of the information that is required in order to launch the ApplicationMaster for the application.

  2. Now, the resource manager has a component called ApplicationManager who is responsible for for accepting job-submissions and negotiating the first container for running ApplicationMaster. So, first container is launched for Application Master who is responsible for requesting the resources/containers from ResourceManager for running the very application. 

Hope your doubt is clear.
answered May 7, 2018 by Ashish
• 2,630 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What is a container in YARN?

A container basically represents a resource on ...READ MORE

answered Apr 9, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
688 views
0 votes
1 answer

Map and Reduce task memory settings in Hadoop YARN

It's preferable and generally, it is recommended ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
61 views
0 votes
1 answer

Why minimum 3 Journal Nodes are required in Hadoop HA architecture?

Initially in Hadoop 1.x, the NameNode was ...READ MORE

answered Apr 20, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
1,190 views
0 votes
1 answer

Where can I find logs in Spark on YARN?

You can access logs through the command yarn ...READ MORE

answered Nov 8, 2018 in Big Data Hadoop by Omkar
• 67,120 points
30 views
0 votes
1 answer

Apache Hadoop Yarn example program

You can go to this location $Yarn_Home/share/hadoop/mapreduce . You'll ...READ MORE

answered Apr 4, 2018 in Big Data Hadoop by nitinrawat895
• 10,110 points
204 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,110 points
2,046 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,472 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
764 views
0 votes
1 answer

Why we are configuring mapred.job.tracker in YARN?

I really dont know the reason behind ...READ MORE

answered Mar 29, 2018 in Big Data Hadoop by Ashish
• 2,630 points
163 views
+1 vote
1 answer

Is Hadoop only Framework in Big Data Ecosystem ?

Actually there are many other frameworks, one of ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by Ashish
• 2,630 points
52 views