Hadoop Cluster Node Setup

0 votes

I am setting up hadoop on a multinode cluster, and I have a few questions:

  1. Will it be ok to have NameNode and ResourceManager on the same machine?

  2. Which will be the best role for a master system, NameNode, ResourceManager Or DataNode/NodeManager?.

  3. I have a master and 3 slave machines. The slaves file on the master machine has the following entries:

    master
    slave1
    slave2
    slave3
    

Do I have to place this same slaves file in all of the slave machines? Or should I remove the first line (master) and then place it in the slave machines?

Oct 15, 2018 in Big Data Hadoop by Neha
• 6,300 points
573 views

1 answer to this question.

0 votes
  1. Yes, at least in small clusters those two should be running in the master node.
  2. Check answer 1. Master node can have also, for example, SecondaryNamenode and JobHistoryServer
  3. No, the slaves file is only on the master node. If you have the master node in the slaves file, it means that the master node acts also as a datanode. Especially in small clusters that's totally fine. The slaves file essentially tells which on nodes the datanode processes are started.

Slave nodes should only run DataNode and NodeManager. But this is all handled by Hadoop if the configurations are correct - you can just check which processes are running after starting the cluster from the master node. Master node basically takes care of everything and you "never" need to manually connect to the slaves for any configurations.

My answer is meant for small clusters, probably in bigger "real" clusters the server responsibilities are even more separated.

answered Oct 15, 2018 by Frankie
• 9,830 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to install and configure a multi-node Hadoop cluster?

I would recommend you to install Cent ...READ MORE

answered Mar 22, 2018 in Big Data Hadoop by Shubham
• 13,490 points
2,454 views
0 votes
1 answer

Hadoop single node cluster set up issues

As far as the error, there's a ...READ MORE

answered Apr 19, 2018 in Big Data Hadoop by kurt_cobain
• 9,350 points
937 views
0 votes
1 answer

Can anyone help me in installing and configuring a Multi-Node Hadoop Cluster?

To install Hadoop setup on the 4-node ...READ MORE

answered Jun 4, 2019 in Big Data Hadoop by ravikiran
• 4,620 points
1,393 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
11,072 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
109,058 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,350 points
4,639 views
0 votes
1 answer
0 votes
1 answer

Which Windows client is used for Cloudera Hadoop Cluster?

You can very well use VM linux ...READ MORE

answered Sep 4, 2018 in Big Data Hadoop by Frankie
• 9,830 points
918 views
0 votes
1 answer

Understanding Hadoop Master Node, Slave Node and Data Node.

The NameNode and the JobTracker are single ...READ MORE

answered Sep 6, 2018 in Big Data Hadoop by Frankie
• 9,830 points
2,679 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP