Which Operating system is more preferable for data node

0 votes
Since I am using Cloudera CDH4 VM in Pseudo distributed mode. My question is, in actual hdfs cluster do we want to install hadoop on the datanode? Can we see the data split in the datanode drive by logging to datanode?.
Sep 4, 2018 in Big Data Hadoop by Frankie
• 9,830 points
505 views

1 answer to this question.

0 votes
In a real installation (1 active namenode, many datanodes) hadoop must be installed on each of the nodes. CDH (and most other vendors) provide software to help with the distributed installation.

You can see file metadata (and generally browse hdfs) via webhdfs, by enabling webhdfs (set property dfs.webhdfs.enabled to true in hdfs-site.xml, and restart hdfs), directing your browser to localhost:50070, and browsing to a file of interest.

File metadata can also be retrieved programmatically in Java via the hadoop FileInputFormat API. e.g, for file splits, you can use getSplits(). It will return the location of each split of the file of interest. A more straight forward solution can be to use the FileSystem API, specifically FileSystem.listFiles() which returns block location information. The latter may be only included in later hadoop 2.x versions though, I'm not sure.
answered Sep 4, 2018 by Neha
• 6,300 points

Related Questions In Big Data Hadoop

0 votes
1 answer

Which Data Structure is used in case of Map Reduce?

In case of Hadoop, HDFS is used ...READ MORE

answered May 4, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
1,273 views
0 votes
1 answer

Which is the Real Time Monitoring tool/API for Hadoop?

If you're using Yarn, there's a rest ...READ MORE

answered Sep 4, 2018 in Big Data Hadoop by Frankie
• 9,830 points
1,051 views
0 votes
1 answer

Which Windows client is used for Cloudera Hadoop Cluster?

You can very well use VM linux ...READ MORE

answered Sep 4, 2018 in Big Data Hadoop by Frankie
• 9,830 points
681 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,555 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,199 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,260 views
0 votes
1 answer
0 votes
1 answer

Which is the most preferable language for Hadooop Map-Reduce programs?

MapReduce is a programming model to perform ...READ MORE

answered Aug 4, 2018 in Big Data Hadoop by Neha
• 6,300 points
3,244 views
0 votes
1 answer
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP