“no such file or directory" in case of hadoop fs -ls

0 votes

I have installed and configured Hadoop 2.7.6 for a 12 node cluster. 1 is acting as masternode and other nodes as slavenodes.

I am facing a problem in executing hadoop fs commands. 
hadoop fs -ls command is working fine with HDFS URI. But it gives me error message "ls: `.': No such file or directory" when used without HDFS URI

ubuntu@sam-master:~$ hadoop fs -ls
15/01/30 17:03:49 WARN util.NativeCodeLoader: Unable to load native-
hadoop 
ibrary for your platform... using builtin-java classes where applicable
ls: `.': No such file or directory
ubuntu@sam-master:~$ 

Whereas, executing the same command with HDFS URI

ubuntu@sam-master:~$ hadoop fs -ls hdfs://sam-master:50000/
15/01/30 17:14:31 WARN util.NativeCodeLoader: Unable to load native-hadoop      
library for your platform... using builtin-java classes where applicable
Found 3 items

drwxr-xr-x   - ubuntu supergroup          0 2015-01-28 12:07 hdfs://sam-master:50000/hvision-data
-
rw-r--r--   2 ubuntu supergroup   15512587 2015-01-28 11:50 hdfs://sam-master:50000/testimage.seq
 
drwxr-xr-x   - ubuntu supergroup          0 2015-01-30 17:03 hdfs://sam-master:50000/wrodcount-in
 ubuntu@sam-master:~$ 

I am getting exception in MapReduce program due to this behavior. jarlib is referring to the HDFS file location, whereas, I want jarlib to refer to the jar files stored at the local file system on the Hadoop nodes.

What is happening with me. I don't understand, please help!

Thanks in advance.

May 9, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
1,950 views

1 answer to this question.

0 votes

The behaviour that you are seeing is expected, let me explain what's going on when you are working with hadoop fs commands.

The exact command's syntax is this: hadoop fs -ls [path]

By default, when you don't specify [path] for the above command, hadoop expands the path to /home/[username] in hdfs; where [username] gets replaced with linux username who is executing the command.

So, when you execute this command:

ubuntu@xadsam-master:~$ hadoop fs -ls
the reason you are seeing the error is ls: '.': No such file or directory because hadoop is looking for this path /home/ubuntu, it seems like this path doesn't exist in hdfs.

The reason why this command:

ubuntu@sam-master:~$ hadoop fs -ls hdfs://101-master:50000/
is working because, you have explicitly specified [path] and is the root of the hdfs. You can also do the same using this:

ubuntu@sam-master:~$ hadoop fs -ls /
which automatically gets evaluated to the root of hdfs.

Hope, this clears the behaviour you are seeing while executing hadoop fs -ls command.

Hence, if you want to specify local file system path use file:/// url scheme.

Hope this will clear all your doubts.

answered May 9, 2018 by nitinrawat895
• 10,030 points

edited May 9, 2018 by nitinrawat895

Related Questions In Big Data Hadoop

0 votes
1 answer

Hadoop Pipe: shared libraries, No such file or directory error

Add -rpath my.zip/lib to LDFLAGS_HDP. For compilation to work, you ...READ MORE

answered Aug 30, 2018 in Big Data Hadoop by Frankie
• 9,810 points
41 views
0 votes
1 answer

Hadoop: `.' no such file or directory while installing

Try hadoop fs -mkdir -p /user/[Username] and then run ...READ MORE

answered Nov 2, 2018 in Big Data Hadoop by Omkar
• 67,120 points
53 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
757 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,030 points
2,037 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,030 points
191 views
0 votes
10 answers

hadoop fs -put command?

copy command can be used to copy files ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Sujay
10,320 views
0 votes
7 answers

How to run a jar file in hadoop?

I used this command to run my ...READ MORE

answered Dec 10, 2018 in Big Data Hadoop by Dasinto
3,967 views
0 votes
12 answers

What is Zookeeper? What is the purpose of Zookeeper in Hadoop Ecosystem?

Hey, Apache Zookeeper says that it is a ...READ MORE

answered Apr 29 in Big Data Hadoop by Gitika
• 19,720 points
3,170 views