if HDFS threshold has been reached. What is the our approach to resolve this issue?

0 votes
Jun 21 in Big Data Hadoop by Manish
66 views

2 answers to this question.

0 votes

Datanodes fill the disk unevenly. Most of the time, certain disks are full while others still have space in them. You can use the tool called DiskBalancer to solve this issue. The Disk Balancer lets administrators rebalance data across multiple disks of a DataNode. To know more, refer to this: https://issues.apache.org/jira/browse/HDFS-1312

answered Jun 24 by Rishi
0 votes

This was a fundamental issue in HDFS for a long time, but there is a new tool called DiskBalancer. It essentially allows you to create a plan file -- That describes how data will be moved from disk to disk and then you can ask a datanode to execute it.

 If one disk is over-utilized, some writes will fail, that is when the datanode picks that disk. So you need to make sure data is similarly distributed in each of the disks. That is what DiskBalancer does for you, it computes how much to move based on each disk type.

answered Jun 24 by Gitika
• 25,360 points

Related Questions In Big Data Hadoop

0 votes
5 answers
0 votes
1 answer

What is the command to navigate in HDFS?

First of all there is no command ...READ MORE

answered Apr 27, 2018 in Big Data Hadoop by Shubham
• 13,310 points
521 views
0 votes
1 answer

What is the command to find the free space in HDFS?

You can use dfsadmin which runs a ...READ MORE

answered Apr 29, 2018 in Big Data Hadoop by Shubham
• 13,310 points
241 views
0 votes
1 answer

What is the standard way to create files in your hdfs file-system?

Well, it's so easy. Just enter the below ...READ MORE

answered Sep 22, 2018 in Big Data Hadoop by Frankie
• 9,810 points
128 views
0 votes
1 answer

Apache Hadoop Yarn example program

You can go to this location $Yarn_Home/share/hadoop/mapreduce . You'll ...READ MORE

answered Apr 4, 2018 in Big Data Hadoop by nitinrawat895
• 10,730 points
244 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,730 points
3,382 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,730 points
407 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
16,894 views
0 votes
1 answer

Why we need to install "ant -Dhadoopversion=23" ? What is the use of this?

Hi, We install ant to build pig, If the ...READ MORE

answered Apr 26 in Big Data Hadoop by Gitika
• 25,360 points
42 views
0 votes
1 answer

What will happen if the OOZIE_URL environment variable has not been set?

Hey, If the Oozie_URL environment variable has not ...READ MORE

answered Jun 24 in Big Data Hadoop by Gitika
• 25,360 points
37 views