Where can I find logs in Spark on YARN

Question

I can't find logs in the history server as in a normal MapReduce job. Where can I find the logs/stderr/stdout?

Answer 1

You can access logs through the command

yarn logs -applicationId <application ID> [OPTIONS]​

Examples:

yarn logs -applicationId application_1414530900704_0003                                      
yarn logs -applicationId application_1414530900704_0003 myuserid

// the user ids are different
yarn logs -applicationId <appid> --appOwner <userid>