Hadoop Cluster Node Setup.

0 votes

I am setting up hadoop on a multinode cluster, and I have a few questions:

  1. Will it be ok to have NameNode and ResourceManager on the same machine?

  2. Which will be the best role for a master system, NameNode, ResourceManager Or DataNode/NodeManager?.

  3. I have a master and 3 slave machines. The slaves file on the master machine has the following entries:

    master
    slave1
    slave2
    slave3
    

Do I have to place this same slaves file in all of the slave machines? Or should I remove the first line (master) and then place it in the slave machines?

Oct 15, 2018 in Big Data Hadoop by Neha
• 6,140 points
17 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes
  1. Yes, at least in small clusters those two should be running in the master node.
  2. Check answer 1. Master node can have also, for example, SecondaryNamenode and JobHistoryServer
  3. No, the slaves file is only on the master node. If you have the master node in the slaves file, it means that the master node acts also as a datanode. Especially in small clusters that's totally fine. The slaves file essentially tells which on nodes the datanode processes are started.

Slave nodes should only run DataNode and NodeManager. But this is all handled by Hadoop if the configurations are correct - you can just check which processes are running after starting the cluster from the master node. Master node basically takes care of everything and you "never" need to manually connect to the slaves for any configurations.

My answer is meant for small clusters, probably in bigger "real" clusters the server responsibilities are even more separated.

answered Oct 15, 2018 by Frankie
• 9,570 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to install and configure a multi-node Hadoop cluster?

I would recommend you to install Cent ...READ MORE

answered Mar 21, 2018 in Big Data Hadoop by Shubham
• 12,150 points
450 views
0 votes
1 answer

Hadoop single node cluster set up issues

As far as the error, there's a ...READ MORE

answered Apr 19, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
51 views
0 votes
1 answer

How to programmatically access hadoop cluster where kerberos is enable?

Okay,here's the code snippet to work in the ...READ MORE

answered Mar 27, 2018 in Big Data Hadoop by coldcode
• 1,980 points
901 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,030 points
1,659 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
8,044 views
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,260 points
560 views
0 votes
1 answer
0 votes
1 answer

Which Windows client is used for Cloudera Hadoop Cluster?

You can very well use VM linux ...READ MORE

answered Sep 4, 2018 in Big Data Hadoop by Frankie
• 9,570 points
38 views
0 votes
1 answer

Understanding Hadoop Master Node, Slave Node and Data Node.

The NameNode and the JobTracker are single ...READ MORE

answered Sep 6, 2018 in Big Data Hadoop by Frankie
• 9,570 points
253 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.