Access DFS through Java API

Question

I m trying to access DFS through Java API so for that I created one Configuration object by using 

Configuration conf = new Configuration(); 

It is not working and not able to find DFS. I've tried adding core-site.xml,mapred-site.xml,yarn-site.xml,and hdfs-site.xml to the Configuration as resources, but it doesn't change anything. What should I do now? 

Thanks in advance.

Answer 1

It is because, may be you are not adding the core-site.xml, hdfs-site.xml correctly to the configuration. 

And no need to add mapred-site.xml and yarn-site.xml. 

Try the below code, hope it will work 

Configuration config = new Configuration(); 

config.addResource(new Path("file:///etc/hadoop/conf/core-site.xml")); // Put your actual core-site.xml path 

config.addResource(new Path("file:///etc/hadoop/conf/hdfs-site.xml")); // Put your actual hdfs-site.xml path 

Path p = new Path("."); // put HDFS path 

FileSystem fs1 = p.getFileSystem(config); 

System.out.println("Home directory :"+fs1.getHomeDirectory()); 

Alternatively you can also try: 

Instead of adding configuration files using addResource method, use set method. Open your core-site.xml file and find the value of fs.defaultFS. Use set method instead of addResource method. config.set("fs1.defaultFS","hdfs://<Namenode-Host>:<Port>"); // Refer your core-site.xml file and replace <Namenode-Host> and <Port> with your cluster namenode and Port (default port number should be `8020`).