How to get absolute path of files in a directory in Hadoop?

0 votes
I have a directory with files, directories, subdirectories, etc.

How I can get the list of absolute paths to all files and directories using the Apache Hadoop API?
Dec 4, 2018 in Big Data Hadoop by Neha
• 6,280 points
557 views

1 answer to this question.

0 votes

You can use HDFS API like the below code

package org.myorg.hdfsdemo;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;



public class HdfsDemo {

    public static void main(String[] args) throws IOException {

        Configuration conf = new Configuration();
        conf.addResource(new Path("/Users/edureka/hadoop/hadoop-1.1.2/conf/core-site.xml"));
        conf.addResource(new Path("/Users/edureka/hadoop/hadoop-1.1.2/conf/hdfs-site.xml"));
        FileSystem fs = FileSystem.get(conf);
        System.out.println("Enter the directory name :");
        BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
        Path path = new Path(br.readLine());
        displayDirectoryContents(fs, path);
    }

    private static void displayDirectoryContents(FileSystem fs, Path rootDir) {
        // TODO Auto-generated method stub
        try {

            FileStatus[] status = fs.listStatus(rootDir);
            for (FileStatus file : status) {
                if (file.isDir()) {
                    System.out.println("This is a directory:" + file.getPath());
                    displayDirectoryContents(fs, file.getPath());
                } else {
                    System.out.println("This is a file:" + file.getPath());
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

answered Dec 4, 2018 by Frankie
• 9,810 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to change the replication factor of specific directory in Hadoop?

Yes, you can change the replication factor ...READ MORE

answered May 10, 2018 in Big Data Hadoop by Shubham
• 13,300 points
486 views
0 votes
1 answer

How to check the size of a file in Hadoop HDFS?

You can use the  hadoop fs -ls command to ...READ MORE

answered Nov 21, 2018 in Big Data Hadoop by Omkar
• 67,610 points
934 views
+1 vote
2 answers

What does hadoop fs -du command gives as output?

du command is used for to see ...READ MORE

answered Jul 23 in Big Data Hadoop by Lokesh Singh
961 views
0 votes
1 answer

How can I write text in HDFS using CMD?

Hadoop put & appendToFile only reads standard ...READ MORE

answered Apr 27, 2018 in Big Data Hadoop by Shubham
• 13,300 points
85 views
0 votes
1 answer

What is the command to find the free space in HDFS?

You can use dfsadmin which runs a ...READ MORE

answered Apr 29, 2018 in Big Data Hadoop by Shubham
• 13,300 points
200 views
0 votes
1 answer

How to find the used cache in HDFS

hdfs dfsadmin -report This command tells fs ...READ MORE

answered May 4, 2018 in Big Data Hadoop by Shubham
• 13,300 points
234 views
0 votes
1 answer

Can I have a list of property files used in Hadoop Framework?

Here is a complete list of configuration ...READ MORE

answered Aug 14, 2018 in Big Data Hadoop by Frankie
• 9,810 points
66 views
0 votes
1 answer