Hdfs cluster and Master/standby Machine

0 votes
The master machine which is used to store Metadata is a part of Hadoop distributed file system. What is the exact difference between hdfs cluster and Master/standby Machine? How these HDfS and Master machines are interrelated?
Jun 19 in Big Data Hadoop by Reshma
13 views

1 answer to this question.

0 votes

Client submits job to Namenode.

To achieve this, Client should have the same copy of Hadoop Distribution and configuration which is present at Namenode.

Then the Only Client will come to know on which node Job tracker is running, and IP of Namenode to access HDFS data.

The three major categories of machine roles in a Hadoop deployment are Client machines, Masters nodes, and Slave nodes.  

The Master nodes oversee the two key functional pieces that make up Hadoop: storing lots of data (HDFS), and running parallel computations on all that data (Map Reduce).  The Name Node oversees and coordinates the data storage function (HDFS), while the Job Tracker oversees and coordinates the parallel processing of data using Map Reduce.  

Slave Nodes make up the vast majority of machines and do all the dirty work of storing the data and running the computations.  Each slave runs both a Data Node and Task Tracker daemon that communicates with and receives instructions from their master nodes.  The Task Tracker daemon is a slave to the Job Tracker, the Data Node daemon a slave to the Name Node.

Client machines have Hadoop installed with all the cluster settings but are neither a Master or a Slave.  Instead, the role of the Client machine is to load data into the cluster, submit Map Reduce jobs describing how that data should be processed, and then retrieve or view the results of the job when it's finished.  In smaller clusters (~40 nodes) you may have a single physical server playing multiple roles, such as both Job Tracker and Name Node.  With medium to large clusters, you will often have each role operating on a single server machine.

answered Jun 19 by Raj

Related Questions In Big Data Hadoop

0 votes
1 answer

Difference between root of hdfs and local machine

The command hdfs dfs -put command is used to ...READ MORE

answered Jun 19 in Big Data Hadoop by Mahi
15 views
0 votes
1 answer

How can I download only hdfs and not hadoop?

No, you cannot download HDFS alone because ...READ MORE

answered Mar 15, 2018 in Big Data Hadoop by nitinrawat895
• 10,110 points
68 views
0 votes
1 answer

How to install and configure a multi-node Hadoop cluster?

I would recommend you to install Cent ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by Shubham
• 13,190 points
545 views
0 votes
1 answer

How to create a FileSystem object that can be used for reading from and writing to HDFS?

Read operation on HDFS In order to read ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by nitinrawat895
• 10,110 points

edited Mar 21, 2018 by nitinrawat895 213 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,110 points
2,053 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,110 points
197 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,499 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
765 views
+2 votes
10 answers

Is there any difference between “hdfs dfs” and “hadoop fs” shell commands?

Yes, there's a difference between hadoop fs and ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Kunal
8,714 views
0 votes
2 answers

Difference between HDFS and NFS

In NFS, the data is stored only ...READ MORE

answered Feb 4 in Big Data Hadoop by Kumar
1,936 views