How to read HDFS and local files with the same code in Java?

0 votes

I want to use my Java code to read the hdfs and local files. Now I use separate functions for it because hdfs comes with hdfs:// prefix. Is there a way I can use the same code to read both files?​

Nov 22, 2018 in Big Data Hadoop by slayer
• 29,040 points
83 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

You can try something like this:

    Configuration conf = new Configuration();
    conf.addResource(new Path("/hadoop/projects/hadoop-1.0.4/conf/core-site.xml"));
    conf.addResource(new Path("/hadoop/projects/hadoop-1.0.4/conf/hdfs-site.xml"));
    file_path = //assign path to your file
    Path path = new Path(file_path);
    FileSystem fs = path.getFileSystem(conf);
    FSDataInputStream inputStream = fs.open(path);
    System.out.println(inputStream.available());
    fs.close();
answered Nov 22, 2018 by Omkar
• 65,850 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to write a file in hdfs with Java?

You could pass the URI when getting ...READ MORE

answered Sep 26, 2018 in Big Data Hadoop by digger
• 27,620 points
42 views
0 votes
1 answer

How to create new directory in hdfs using java code?

Try this: { Configuration config ...READ MORE

answered Dec 6, 2018 in Big Data Hadoop by Omkar
• 65,850 points
25 views
0 votes
1 answer

How to copy files in hdfs with wildcards?

You can use the hdfs command: hdfs fs ...READ MORE

answered Dec 11, 2018 in Big Data Hadoop by Omkar
• 65,850 points
55 views
0 votes
1 answer

How to read multiple files in hdfs?

If you want to read files with ...READ MORE

answered Feb 7 in Big Data Hadoop by Omkar
• 65,850 points
105 views
0 votes
1 answer

Moving files in Hadoop using the Java API?

I would recommend you to use FileSystem.rename(). ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 12,270 points
537 views
0 votes
1 answer

Hadoop giving java.io.IOException, in mkdir Java code.

I am not sure about the issue. ...READ MORE

answered May 3, 2018 in Big Data Hadoop by Shubham
• 12,270 points
266 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,070 points
1,679 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 9,070 points
132 views
0 votes
1 answer

Hadoop: How to copy directory from local system to hdfs using Java code?

Just use the FileSystem's copyFromLocalFile method. If the source Path ...READ MORE

answered Nov 14, 2018 in Big Data Hadoop by Omkar
• 65,850 points
167 views
0 votes
1 answer

How to concatenate hdfs files and store in output file?

You can use a combination of cat and put command. Something ...READ MORE

answered Dec 5, 2018 in Big Data Hadoop by Omkar
• 65,850 points
172 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.