How to limit the cores being used by a cluster

Question

I am running a Standalone deploy cluster and I want to limit the number of cores being from the application across the cluster to 2. How can I do this?

Answer 1

You can set the maximum number of cores that can be requested by the application. Refer to the command below:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.cores.max=2