Dynamic allocation Set minimum number of executor

0 votes
Please tell me how to set minimum number of executors for dynamic allocation. Thanks in advance.
Mar 13, 2019 in Apache Spark by Kumar
194 views

1 answer to this question.

0 votes

You cans et it dynamically like this: 

val sc = new SparkContext(new SparkConf())
./bin/spark-submit <all your existing options> --spark.dynamicAllocation.minExecutors=<number of executors>
answered Mar 13, 2019 by Venu

Related Questions In Apache Spark

0 votes
1 answer

How to set max executors for dynamic allocation?

You can set it by assigning the ...READ MORE

answered Mar 13, 2019 in Apache Spark by Venu
1,174 views
0 votes
1 answer

Delay requesting new executor in dynamic allocation

You can set the duration like this: val ...READ MORE

answered Mar 13, 2019 in Apache Spark by Venu
158 views
0 votes
1 answer

Can the executor core be greater than the total number of spark tasks?

Hi@Rishi, Yes, it is possible. If executor no. ...READ MORE

answered Jun 17, 2020 in Apache Spark by MD
• 95,340 points
817 views
0 votes
1 answer

Can number of Spark task be greater than the executor core?

Hi@Rishi, Yes, number of spark tasks can be ...READ MORE

answered Jun 17, 2020 in Apache Spark by MD
• 95,340 points
349 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
8,625 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,522 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
76,127 views
0 votes
1 answer

Increase number of cores in Spark

Now that the job is already running, ...READ MORE

answered Feb 23, 2019 in Apache Spark by Reshma
1,100 views
0 votes
1 answer

How many executors run when dynamic allocation is enabled?

The number of executors running by default ...READ MORE

answered Mar 13, 2019 in Apache Spark by Venu
322 views
webinar REGISTER FOR FREE WEBINAR X
Send OTP
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP