How will you decide whether you need to use the Capacity Scheduler or the Fair Scheduler?

0 votes
I know how fair scheduler will handle more than one job, but doesn't know how to decide the use of Capacity Scheduler. Can you help me out with this.
Dec 20, 2018 in Big Data Hadoop by Neha
• 6,280 points
111 views

1 answer to this question.

0 votes
Fair Scheduling is the process in which resources are assigned to jobs such that all jobs get to share equal number of resources over time. Fair Scheduler can be used under the following circumstances -

i) If you wants the jobs to make equal progress instead of following the FIFO order then you must use Fair Scheduling.

ii) If you have slow connectivity and data locality plays a vital role and makes a significant difference to the job runtime then you must use Fair Scheduling.

iii) Use fair scheduling if there is a lot of variability in the utilization between pools.

Capacity Scheduler allows runs the Hadoop mapreduce cluster as a shared, multi-tenant cluster to maximize the utilization of the hadoop cluster and throughput. Capacity Scheduler can be used under the following circumstances -

i) If the jobs require scheduler determinism then Capacity Scheduler can be useful.

ii) CS's memory based scheduling method is useful if the jobs have varying memory requirements.

iii) If you want to enforce resource allocation because you know very well about the cluster utilization and workload then use Capacity Scheduler.
answered Dec 20, 2018 by Frankie
• 9,810 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How will the Fair Scheduler handle more than one Job?

In your case when job B gets ...READ MORE

answered Aug 16, 2018 in Big Data Hadoop by nitinrawat895
• 10,760 points
58 views
0 votes
1 answer

How can I use my host machine’s web browser to check my HDFS services running in the VM?

The sole purpose of the virtual machine ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by Shubham
• 13,350 points
171 views
0 votes
1 answer

Why we need to install "ant -Dhadoopversion=23" ? What is the use of this?

Hi, We install ant to build pig, If the ...READ MORE

answered Apr 26 in Big Data Hadoop by Gitika
• 25,420 points
48 views
0 votes
1 answer
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,760 points
3,551 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
18,191 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
1,318 views
0 votes
1 answer
0 votes
1 answer

How to write examples to test whether the installation is validated or not?

You have some example jars in your ...READ MORE

answered Sep 5, 2018 in Big Data Hadoop by Frankie
• 9,810 points
40 views
0 votes
1 answer

How to format the output being written by MapReduce in Hadoop?

Here is a simple code demonstrate the ...READ MORE

answered Sep 5, 2018 in Big Data Hadoop by Frankie
• 9,810 points
170 views