How to read HDFS and local files with the same code in Java?

0 votes

I want to use my Java code to read the hdfs and local files. Now I use separate functions for it because hdfs comes with hdfs:// prefix. Is there a way I can use the same code to read both files?​

Nov 22, 2018 in Big Data Hadoop by slayer
• 29,050 points
442 views

1 answer to this question.

0 votes

You can try something like this:

    Configuration conf = new Configuration();
    conf.addResource(new Path("/hadoop/projects/hadoop-1.0.4/conf/core-site.xml"));
    conf.addResource(new Path("/hadoop/projects/hadoop-1.0.4/conf/hdfs-site.xml"));
    file_path = //assign path to your file
    Path path = new Path(file_path);
    FileSystem fs = path.getFileSystem(conf);
    FSDataInputStream inputStream = fs.open(path);
    System.out.println(inputStream.available());
    fs.close();
answered Nov 22, 2018 by Omkar
• 67,460 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to write a file in hdfs with Java?

You could pass the URI when getting ...READ MORE

answered Sep 26, 2018 in Big Data Hadoop by digger
• 27,640 points
117 views
0 votes
1 answer

How to create new directory in hdfs using java code?

Try this: { Configuration config ...READ MORE

answered Dec 6, 2018 in Big Data Hadoop by Omkar
• 67,460 points
72 views
0 votes
1 answer

How to copy files in hdfs with wildcards?

You can use the hdfs command: hdfs fs ...READ MORE

answered Dec 11, 2018 in Big Data Hadoop by Omkar
• 67,460 points
130 views
0 votes
1 answer

How to read multiple files in hdfs?

If you want to read files with ...READ MORE

answered Feb 7 in Big Data Hadoop by Omkar
• 67,460 points
325 views
0 votes
1 answer

Moving files in Hadoop using the Java API?

I would recommend you to use FileSystem.rename(). ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 13,290 points
797 views
0 votes
1 answer

Hadoop giving java.io.IOException, in mkdir Java code.

I am not sure about the issue. ...READ MORE

answered May 3, 2018 in Big Data Hadoop by Shubham
• 13,290 points
399 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,670 points
2,704 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,670 points
284 views
0 votes
1 answer

Hadoop: How to copy directory from local system to hdfs using Java code?

Just use the FileSystem's copyFromLocalFile method. If the source Path ...READ MORE

answered Nov 14, 2018 in Big Data Hadoop by Omkar
• 67,460 points
512 views
0 votes
1 answer

How to concatenate hdfs files and store in output file?

You can use a combination of cat and put command. Something ...READ MORE

answered Dec 5, 2018 in Big Data Hadoop by Omkar
• 67,460 points
424 views