How to spread applications to less nodes

Question

Hi. I have a Spark application and when on deployment, it is being spread out across nodes. But because there are compute-intensive workloads, I want it to spread across fewer nodes. How can I do this?

Answer 1

You can limit the spread out by setting the spreadout configuration to false. You can do it as follows:

Open spark shell and run the following command:

val sc = new SparkContext(new SparkConf())

./bin/spark-submit <all your existing options> --spark.deploy.spreadOut=false