ApplicationMaster in YARN Architecture

0 votes

I have recently started learning Hadoop and I am finding it difficult the overall architecture or job execution flow w.r.t. YARN. I was following the official documentation on YARN where I found that:

ApplicationMaster has the responsibility of negotiating appropriate resource containers from the Scheduler (ResourceManager)

and ApplicationsManager is responsible for negotiating the first container for executing the ApplicationMaster.

Link : http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html

Following are the things that I am confused with:

  1. The documentation says that ApplicationMaster has the responsilibility to request ResourceManager for Container. Now, who is responsible for providing the container for ApplicationMaster in the first place?
  2. How does YARN knows about creating a container for ApplicationMaster?
May 7, 2018 in Big Data Hadoop by Atul
• 10,240 points
272 views

1 answer to this question.

0 votes

Let me help you with your confusion:

  1. YARN client submits the application to ResourceManager, providing it all of the information that is required in order to launch the ApplicationMaster for the application.

  2. Now, the resource manager has a component called ApplicationManager who is responsible for for accepting job-submissions and negotiating the first container for running ApplicationMaster. So, first container is launched for Application Master who is responsible for requesting the resources/containers from ResourceManager for running the very application. 

Hope your doubt is clear.
answered May 7, 2018 by Ashish
• 2,630 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What is a container in YARN?

A container basically represents a resource on ...READ MORE

answered Apr 9, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
857 views
0 votes
1 answer

Map and Reduce task memory settings in Hadoop YARN

It's preferable and generally, it is recommended ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
92 views
0 votes
1 answer

Why minimum 3 Journal Nodes are required in Hadoop HA architecture?

Initially in Hadoop 1.x, the NameNode was ...READ MORE

answered Apr 20, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
1,870 views
0 votes
1 answer

Where can I find logs in Spark on YARN?

You can access logs through the command yarn ...READ MORE

answered Nov 8, 2018 in Big Data Hadoop by Omkar
• 67,660 points
49 views
0 votes
1 answer

Apache Hadoop Yarn example program

You can go to this location $Yarn_Home/share/hadoop/mapreduce . You'll ...READ MORE

answered Apr 4, 2018 in Big Data Hadoop by nitinrawat895
• 10,710 points
240 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,710 points
3,327 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
16,446 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
1,198 views
0 votes
1 answer

Why we are configuring mapred.job.tracker in YARN?

I really dont know the reason behind ...READ MORE

answered Mar 29, 2018 in Big Data Hadoop by Ashish
• 2,630 points
246 views
+1 vote
1 answer

Is Hadoop only Framework in Big Data Ecosystem ?

Actually there are many other frameworks, one of ...READ MORE

answered Mar 26, 2018 in Big Data Hadoop by Ashish
• 2,630 points
82 views