Increasing memory to use for Yarn application master

Question

I think the default memory for yarn application is less and I want to increase the memory. The spark application is already running so I want to make this change dynamically. How can I do this?

Answer 1

You can increase the memory dynamically by using the following command in Spark shell:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.yarn.am.memory=<set the memory>