How does decreasing the block size make more space on Datanode?

0 votes
I need to move a file titled weblogs into HDFS. When I try to copy the file, I can't because I have ample space on the DataNodes. I was asked decrease the block size on your remaining files to relieve this situation and store more files in HDFS. How does this help?
Jul 5 in Big Data Hadoop by Vishnu
37 views

1 answer to this question.

+1 vote
When there is space in data nodes and the file write to hdfs is still failing, the size of the block should be reduced.

The reason behind recommending to try a lower block size is that the write success depends on the completion of individual blocks and if the space is less on each data node but collectively high on the overall cluster combining all the data nodes free spaces, then a small block write to each data node has a higher chance of success in evading a full disc condition than when trying to write fewer large contiguous blocks.
answered Jul 5 by Krish

Related Questions In Big Data Hadoop

0 votes
1 answer

How does the HDFS Client knows the block size while writing?

HDFS is designed in a way where ...READ MORE

answered Mar 27, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
47 views
0 votes
1 answer

Where does HDFS stores data on the local file system?

First find the Hadoop directory present in ...READ MORE

answered May 8, 2018 in Big Data Hadoop by Shubham
• 13,350 points
2,208 views
0 votes
1 answer

How to analyze block placement on datanodes and rebalancing data across Hadoop nodes?

HDFS provides a tool for administrators i.e. ...READ MORE

answered Jun 21, 2018 in Big Data Hadoop by nitinrawat895
• 10,760 points
144 views
0 votes
1 answer

What metadata is stored on a DataNode when a block is written to it?

Let me explain you step by step.  Each ...READ MORE

answered Jul 23, 2018 in Big Data Hadoop by nitinrawat895
• 10,760 points
267 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,760 points
3,551 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,760 points
441 views
+1 vote
11 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
18,202 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,280 points
1,319 views
0 votes
1 answer

Unable to run Name node and datanode on Single Node cluster: Does not contain a valid host:port

Add the following properties in the core-site.xml file, it ...READ MORE

answered May 22 in Big Data Hadoop by Sanam
60 views
0 votes
1 answer

How Hadoop scalabality linear or proportional depends on the number of nodes?

Hey, These jobs are often IO based not ...READ MORE

answered May 28 in Big Data Hadoop by Gitika
• 25,420 points
44 views