Dynamic allocation Set minimum number of executor

0 votes
Please tell me how to set minimum number of executors for dynamic allocation. Thanks in advance.
Mar 13, 2019 in Apache Spark by Kumar
574 views

1 answer to this question.

0 votes

You cans et it dynamically like this: 

val sc = new SparkContext(new SparkConf())
./bin/spark-submit <all your existing options> --spark.dynamicAllocation.minExecutors=<number of executors>
answered Mar 13, 2019 by Venu

Related Questions In Apache Spark

0 votes
1 answer

How to set max executors for dynamic allocation?

You can set it by assigning the ...READ MORE

answered Mar 13, 2019 in Apache Spark by Venu
1,795 views
0 votes
1 answer

Delay requesting new executor in dynamic allocation

You can set the duration like this: val ...READ MORE

answered Mar 13, 2019 in Apache Spark by Venu
489 views
0 votes
1 answer

Can the executor core be greater than the total number of spark tasks?

Hi@Rishi, Yes, it is possible. If executor no. ...READ MORE

answered Jun 17, 2020 in Apache Spark by MD
• 95,460 points
2,114 views
0 votes
1 answer

Can number of Spark task be greater than the executor core?

Hi@Rishi, Yes, number of spark tasks can be ...READ MORE

answered Jun 17, 2020 in Apache Spark by MD
• 95,460 points
1,946 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,931 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,460 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
108,357 views
0 votes
1 answer

Increase number of cores in Spark

Now that the job is already running, ...READ MORE

answered Feb 23, 2019 in Apache Spark by Reshma
2,177 views
0 votes
1 answer

How many executors run when dynamic allocation is enabled?

The number of executors running by default ...READ MORE

answered Mar 13, 2019 in Apache Spark by Venu
819 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP