Why does my yarn application not have logs even with logging enabled?

0 votes

I have enabled logs using yarn-site.xml. After this I restarted yarn as well.

After running a sample MR application, I was able to see the applicationID using command: 

yarn application -list. 

Now, for checking logs when I issue the command - yarn logs -applicationId <application ID>, and I am get the following prompt instead of logs:

hdfs://<ip address>/var/log/hadoop-yarn/path/to/application/  does not have any log files

Please help me to figure out what went wrong and what is the correct way to check logs.

Thank you.

Mar 15, 2018 in Big Data Hadoop by Ashish
• 2,630 points
350 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

yarn application -list command shows you all the application having the following state: SUBMITED ACCEPTED RUNNING Logs are always collected once the application has been finished into dir set to yarn.nodemanager.remote-app-log-dir.Basically, the Log Aggregation feature aggregates logs across all containers on a worker node and stores them as one aggregated log file per worker node after the application has been finished.

 Hence, the application for which you are trying to get the logs has not been completed yet. So, wait for the application to be finished and then you use the same command i.e.: yarn logs -applicationId <your_application_id>

 To check the state of all application use command:

 yarn application -list -appStates FINISHED ALL

answered Mar 19, 2018 by kurt_cobain
• 9,260 points

Related Questions In Big Data Hadoop

0 votes
1 answer
0 votes
1 answer

Why we are configuring mapred.job.tracker in YARN?

I really dont know the reason behind ...READ MORE

answered Mar 29, 2018 in Big Data Hadoop by Ashish
• 2,630 points
132 views
0 votes
1 answer

Not able to find HDFS directory on my system?

I guess you didn't understand it correctly dfs.datanode.data.dir, ...READ MORE

answered Apr 17, 2018 in Big Data Hadoop by nitinrawat895
• 9,070 points
98 views
0 votes
1 answer

Why am I not able to see the Hadoop daemons that are running?

I guess you are starting the services ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by Shubham
• 12,270 points
38 views
0 votes
1 answer

Hadoop Command to merge edit logs with fsimage

You can use this command to save ...READ MORE

answered May 10, 2018 in Big Data Hadoop by Shubham
• 12,270 points
229 views
0 votes
1 answer

Why Hadoop is not implemented using Message Passing Interface (MPI)?

One of the big features of Hadoop/map-reduce ...READ MORE

answered Sep 20, 2018 in Big Data Hadoop by Frankie
• 9,590 points
45 views
0 votes
1 answer

How does the HDFS Client knows the block size while writing?

HDFS is designed in a way where ...READ MORE

answered Mar 27, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
15 views
0 votes
1 answer

hadoop: command not found

The reason you are getting hadoop as ...READ MORE

answered Mar 27, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
1,928 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.