Why does my yarn application not have logs even with logging enabled

0 votes

I have enabled logs using yarn-site.xml. After this I restarted yarn as well.

After running a sample MR application, I was able to see the applicationID using command: 

yarn application -list. 

Now, for checking logs when I issue the command - yarn logs -applicationId <application ID>, and I am get the following prompt instead of logs:

hdfs://<ip address>/var/log/hadoop-yarn/path/to/application/  does not have any log files

Please help me to figure out what went wrong and what is the correct way to check logs.

Thank you.

Mar 16, 2018 in Big Data Hadoop by Ashish
• 2,650 points
4,550 views

1 answer to this question.

0 votes

yarn application -list command shows you all the application having the following state: SUBMITED ACCEPTED RUNNING Logs are always collected once the application has been finished into dir set to yarn.nodemanager.remote-app-log-dir.Basically, the Log Aggregation feature aggregates logs across all containers on a worker node and stores them as one aggregated log file per worker node after the application has been finished.

 Hence, the application for which you are trying to get the logs has not been completed yet. So, wait for the application to be finished and then you use the same command i.e.: yarn logs -applicationId <your_application_id>

 To check the state of all application use command:

 yarn application -list -appStates FINISHED ALL

answered Mar 19, 2018 by kurt_cobain
• 9,390 points

Related Questions In Big Data Hadoop

0 votes
1 answer
0 votes
1 answer

Why we are configuring mapred.job.tracker in YARN?

I really dont know the reason behind ...READ MORE

answered Mar 29, 2018 in Big Data Hadoop by Ashish
• 2,650 points
1,610 views
0 votes
1 answer

Not able to find HDFS directory on my system?

I guess you didn't understand it correctly dfs.datanode.data.dir, ...READ MORE

answered Apr 17, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
7,059 views
0 votes
1 answer

Why am I not able to see the Hadoop daemons that are running?

I guess you are starting the services ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by Shubham
• 13,490 points
786 views
0 votes
1 answer

Hadoop Command to merge edit logs with fsimage

You can use this command to save ...READ MORE

answered May 11, 2018 in Big Data Hadoop by Shubham
• 13,490 points
1,668 views
0 votes
1 answer

How does the HDFS Client knows the block size while writing?

HDFS is designed in a way where ...READ MORE

answered Mar 27, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
835 views
0 votes
1 answer

hadoop: command not found

The reason you are getting hadoop as ...READ MORE

answered Mar 28, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
13,322 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP