HDFS usage for small files

0 votes
Suppose we have two situations and they are having a small amount of data distributed across different files and a large amount of data in one file. In which situation we should use HDFS?
Little confused over here
Jun 12, 2018 in Big Data Hadoop by Data_Nerd
• 2,360 points
24 views

1 answer to this question.

0 votes
Well, the HDFS works more reliably with the large data when the same is stored on a single file. In Name Node, the concerned information is available in the RAM and thus it cannot deal with a large number of files. In case files are more, there will be a large amount of Metadata it needs to deal with. It is almost impossible to store such a large volume of Metadata in the RAM.
Hope this clears everything
answered Jun 12, 2018 by BigData_is_my_ex

Related Questions In Big Data Hadoop

0 votes
1 answer
0 votes
1 answer

Files for Configuring HDFS in Hadoop 2.2.0?

By default these Hadoop configuration files are ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 13,290 points
24 views
0 votes
1 answer

Small files and HDFS blocks

Multiple files are not stored in a ...READ MORE

answered Nov 16, 2018 in Big Data Hadoop by Frankie
• 9,810 points
50 views
0 votes
1 answer

How to create a FileSystem object that can be used for reading from and writing to HDFS?

Read operation on HDFS In order to read ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by nitinrawat895
• 10,670 points

edited Mar 21, 2018 by nitinrawat895 287 views
0 votes
5 answers
0 votes
1 answer

What are the various ways to import files into HDFS?

There are various tools and frameworks available ...READ MORE

answered Apr 13, 2018 in Big Data Hadoop by nitinrawat895
• 10,670 points
297 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,670 points
2,686 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
13,371 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
980 views
0 votes
1 answer