How does decreasing the block size make more space on Datanode

0 votes
I need to move a file titled weblogs into HDFS. When I try to copy the file, I can't because I have ample space on the DataNodes. I was asked decrease the block size on your remaining files to relieve this situation and store more files in HDFS. How does this help?
Jul 5, 2019 in Big Data Hadoop by Vishnu
670 views

1 answer to this question.

+1 vote
When there is space in data nodes and the file write to hdfs is still failing, the size of the block should be reduced.

The reason behind recommending to try a lower block size is that the write success depends on the completion of individual blocks and if the space is less on each data node but collectively high on the overall cluster combining all the data nodes free spaces, then a small block write to each data node has a higher chance of success in evading a full disc condition than when trying to write fewer large contiguous blocks.
answered Jul 5, 2019 by Krish

Related Questions In Big Data Hadoop

0 votes
1 answer

How does the HDFS Client knows the block size while writing?

HDFS is designed in a way where ...READ MORE

answered Mar 27, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
834 views
0 votes
1 answer

Where does HDFS stores data on the local file system?

First find the Hadoop directory present in ...READ MORE

answered May 8, 2018 in Big Data Hadoop by Shubham
• 13,490 points
15,658 views
0 votes
1 answer

How to analyze block placement on datanodes and rebalancing data across Hadoop nodes?

HDFS provides a tool for administrators i.e. ...READ MORE

answered Jun 21, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
823 views
0 votes
1 answer

What metadata is stored on a DataNode when a block is written to it?

Let me explain you step by step.  Each ...READ MORE

answered Jul 23, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
2,723 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,555 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
2,184 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,185 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,260 views
0 votes
1 answer

Unable to run Name node and datanode on Single Node cluster: Does not contain a valid host:port

Add the following properties in the core-site.xml file, it ...READ MORE

answered May 22, 2019 in Big Data Hadoop by Sanam
1,061 views
0 votes
1 answer

How Hadoop scalabality linear or proportional depends on the number of nodes?

Hey, These jobs are often IO based not ...READ MORE

answered May 28, 2019 in Big Data Hadoop by Gitika
• 65,910 points
372 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP