Spark Yarn Changing maximum number of time to submit application

Question

What is the default value set for the maximum number of times that the application is submitted and how can I change it?

Answer 1

By default, the maximum number of times to submit the application is set to the property yarn.resourcemanager.am.max-attempts

If you want to change this, refer to the below command:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.yarn.maxAppAttempts= <max number of attempt>