What is the use of fsimage in hadoop?

0 votes
In hadoop eco-system, edit Logs holds all the information about Metadata, then what's the need of FSimage?
Dec 20, 2018 in Big Data Hadoop by digger
• 27,620 points
586 views

1 answer to this question.

0 votes

The NameNode stores modifications to the file system as a log appended to a native file system file, edits

When a NameNode starts up, it reads HDFS state from an image file, fsimage, and then applies edits from the edits log file.

It then writes new HDFS state to the fsimage and starts normal operation with an empty edits file.

FsImage is a file stored on the OS filesystem that contains the complete directory structure (namespace) of the HDFS with details about the location of the data on the Data Blocks and which blocks are stored on which node.

EditLogs is a transaction log that recorde the changes in the HDFS file system or any action performed on the HDFS cluster such as addtion of a new block, replication, deletion etc., It records the changes since the last FsImage was created, it then merges the changes into the FsImage file to create a new FsImage file.

When we are starting namenode, latest FsImage file is loaded into "in-memory" and at the same time, EditLog file is also loaded into memory if FsImage file does not contain up to date information.

Namenode stores metadata in "in-memory" in order to serve the multiple client request(s) as fast as possible. If this is not done,  then for every operation , namenode has to read the metadata information from the disk to in-memory. This process will consume more disk seek time for every operation.

answered Dec 20, 2018 by Omkar
• 67,120 points

Related Questions In Big Data Hadoop

0 votes
1 answer

What is the use of sequence file in Hadoop?

Sequence files are binary files containing serialized ...READ MORE

answered Apr 5, 2018 in Big Data Hadoop by Ashish
• 2,630 points
630 views
0 votes
12 answers

What is Zookeeper? What is the purpose of Zookeeper in Hadoop Ecosystem?

Hey, Apache Zookeeper says that it is a ...READ MORE

answered Apr 29 in Big Data Hadoop by Gitika
• 19,720 points
3,130 views
0 votes
1 answer

What is the meaning of Write Ahead Log in Hadoop?

Write Ahead Log (WAL) is a file ...READ MORE

answered Nov 20, 2018 in Big Data Hadoop by Sunil
74 views
0 votes
1 answer

What is the usage of Configured class in Hadoop programs?

Configured is a default implementation of the Configurable interface - ...READ MORE

answered Nov 22, 2018 in Big Data Hadoop by Frankie
• 9,810 points
96 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,030 points
2,026 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,030 points
188 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,242 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
751 views
0 votes
1 answer

What is the command to count number of lines in a file in hdfs?

hadoop fs -cat /example2/doc1 | wc -l READ MORE

answered Nov 22, 2018 in Big Data Hadoop by Omkar
• 67,120 points
191 views
0 votes
1 answer

What is the difference between MapReduce and YARN in Hadoop?

MapReduce: MapReduce is an algorithm used to store ...READ MORE

answered Dec 19, 2018 in Big Data Hadoop by Omkar
• 67,120 points
231 views