Hadoop error Input path does not exist

Question

I am trying to specify In folder but i get this error:

18/11/28 22:05:27 ERROR security.UserGroupInformation: PriviledgedActionException as:USER cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://localhost:9100/user/USER/In

How to solve this?

Answer 1

I think you forgot to upload the input files to hdfs. Try this:

bin/hadoop fs -mkdir In

Answer 2

I was able to fix the issue. There was nothing wrong with my code.

I had two terminals open. In one terminal I moved the input file to HDFS.
[edureka@localhost Desktop]$ hdfs dfs -put sample1.txt ponezhil/sample.txt

In another terminal I executed the MapReduce program with this command.
[edureka@localhost ~]$ hadoop jar /home/edureka/Desktop/WordcountMR.jar hdfs:ponezhil/sample.txt hdfs:out

I got error. I referred this command from LMS in a video "How to run MapReduce Program in Edureka VM"

Then I used this command.
[edureka@localhost ~]$ hadoop jar /home/edureka/Desktop/WordcountMR.jar com.edureka.WordCountDriver ponezhil/sample.txt out

We were thought in class to fire the command in this format.
hadoop jar jarFileName className I/Ppath O/Ppath.
But this also ended up in error. After a lot of trial and error, I happened to notice that every time after executing this command, my input file was diappearing from the HDFS.

And finally I used this command which worked, again copying my input file to the HDFS.
[edureka@localhost ~]$ hadoop jar /home/edureka/Desktop/WordcountMR.jar ponezhil/sample.txt out

Comments

Hey @Ponezhil Ramu!
What error did you get when you ran the command with input and output path?