How does decreasing the block size make more space on Datanode?

0 votes
I need to move a file titled weblogs into HDFS. When I try to copy the file, I can't because I have ample space on the DataNodes. I was asked decrease the block size on your remaining files to relieve this situation and store more files in HDFS. How does this help?
Jul 5 in Big Data Hadoop by Vishnu
20 views

1 answer to this question.

+1 vote
When there is space in data nodes and the file write to hdfs is still failing, the size of the block should be reduced.

The reason behind recommending to try a lower block size is that the write success depends on the completion of individual blocks and if the space is less on each data node but collectively high on the overall cluster combining all the data nodes free spaces, then a small block write to each data node has a higher chance of success in evading a full disc condition than when trying to write fewer large contiguous blocks.
answered Jul 5 by Krish

Related Questions In Big Data Hadoop

0 votes
1 answer

How does the HDFS Client knows the block size while writing?

HDFS is designed in a way where ...READ MORE

answered Mar 27, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
25 views
0 votes
1 answer

Where does HDFS stores data on the local file system?

First find the Hadoop directory present in ...READ MORE

answered May 8, 2018 in Big Data Hadoop by Shubham
• 13,190 points
751 views
0 votes
1 answer

How to analyze block placement on datanodes and rebalancing data across Hadoop nodes?

HDFS provides a tool for administrators i.e. ...READ MORE

answered Jun 21, 2018 in Big Data Hadoop by nitinrawat895
• 10,110 points
93 views
0 votes
1 answer

What metadata is stored on a DataNode when a block is written to it?

Let me explain you step by step.  Each ...READ MORE

answered Jul 23, 2018 in Big Data Hadoop by nitinrawat895
• 10,110 points
138 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,110 points
2,047 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,110 points
196 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,476 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
764 views
0 votes
1 answer

Unable to run Name node and datanode on Single Node cluster: Does not contain a valid host:port

Add the following properties in the core-site.xml file, it ...READ MORE

answered May 22 in Big Data Hadoop by Sanam
17 views
0 votes
1 answer

How Hadoop scalabality linear or proportional depends on the number of nodes?

Hey, These jobs are often IO based not ...READ MORE

answered May 28 in Big Data Hadoop by Gitika
• 19,720 points
28 views