Hadoop Cluster Node Setup.

0 votes

I am setting up hadoop on a multinode cluster, and I have a few questions:

  1. Will it be ok to have NameNode and ResourceManager on the same machine?

  2. Which will be the best role for a master system, NameNode, ResourceManager Or DataNode/NodeManager?.

  3. I have a master and 3 slave machines. The slaves file on the master machine has the following entries:

    master
    slave1
    slave2
    slave3
    

Do I have to place this same slaves file in all of the slave machines? Or should I remove the first line (master) and then place it in the slave machines?

Oct 15, 2018 in Big Data Hadoop by Neha
• 6,280 points
25 views

1 answer to this question.

0 votes
  1. Yes, at least in small clusters those two should be running in the master node.
  2. Check answer 1. Master node can have also, for example, SecondaryNamenode and JobHistoryServer
  3. No, the slaves file is only on the master node. If you have the master node in the slaves file, it means that the master node acts also as a datanode. Especially in small clusters that's totally fine. The slaves file essentially tells which on nodes the datanode processes are started.

Slave nodes should only run DataNode and NodeManager. But this is all handled by Hadoop if the configurations are correct - you can just check which processes are running after starting the cluster from the master node. Master node basically takes care of everything and you "never" need to manually connect to the slaves for any configurations.

My answer is meant for small clusters, probably in bigger "real" clusters the server responsibilities are even more separated.

answered Oct 15, 2018 by Frankie
• 9,810 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to install and configure a multi-node Hadoop cluster?

I would recommend you to install Cent ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by Shubham
• 13,190 points
545 views
0 votes
1 answer

Hadoop single node cluster set up issues

As far as the error, there's a ...READ MORE

answered Apr 19, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
63 views
0 votes
1 answer

Can anyone help me in installing and configuring a Multi-Node Hadoop Cluster?

To install Hadoop setup on the 4-node ...READ MORE

answered Jun 4 in Big Data Hadoop by ravikiran
• 3,560 points
25 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,110 points
2,053 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,500 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
765 views
0 votes
1 answer
0 votes
1 answer

Which Windows client is used for Cloudera Hadoop Cluster?

You can very well use VM linux ...READ MORE

answered Sep 4, 2018 in Big Data Hadoop by Frankie
• 9,810 points
54 views
0 votes
1 answer

Understanding Hadoop Master Node, Slave Node and Data Node.

The NameNode and the JobTracker are single ...READ MORE

answered Sep 6, 2018 in Big Data Hadoop by Frankie
• 9,810 points
318 views