how to make an existing hadoop cluster self adaptive

0 votes
i have a hadoop cluster of 10 slaves nodes and i want it to be self adaptive. That is, it will either shrink or expand depending on the size of dataset file in master node. how can i proceed with it?

all the ip of slaves nodes are stored in ect/host file.
Feb 10, 2020 in Big Data Hadoop by Sanjana
• 140 points
183 views

Hi@Sanjana,

In Hadoop we don't have the concept of block as a service. So you can't able to do partitions in your cluster.

Can you kindly explain more about your query on shrink or expand?

Thank You

1 answer to this question.

0 votes

Hey, @Sanjana,

You can't shrink or expand nodes when your are using bare-metal servers, You can do that when your cluster is hosted on the cloud but that will not happen automatically, you need to do it manually. Increase/decrease the disk size but this doesn't happen in a productive environment.

answered Jul 16, 2020 by Gitika
• 65,870 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to install and configure a multi-node Hadoop cluster?

I would recommend you to install Cent ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by Shubham
• 13,480 points
1,220 views
0 votes
1 answer

How to programmatically access hadoop cluster where kerberos is enable?

Okay,here's the code snippet to work in the ...READ MORE

answered Mar 27, 2018 in Big Data Hadoop by coldcode
• 2,080 points
4,227 views
0 votes
1 answer

How to delete a directory from Hadoop cluster which is having comma(,) in its name?

Just try the following command: hadoop fs -rm ...READ MORE

answered May 7, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
938 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
6,862 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
1,099 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
48,500 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
2,602 views
0 votes
1 answer

How to configure the Hadoop cluster with proxyuser for the Oozie process?

Hey, The following two properties are required in ...READ MORE

answered Jun 10, 2019 in Big Data Hadoop by Gitika
• 65,870 points
202 views
0 votes
1 answer

How to import Flat files in Hadoop using sqoop?

Sqoop is a tool which we use ...READ MORE

answered Apr 9, 2019 in Big Data Hadoop by Gitika
• 65,870 points
714 views