How to list files in hdfs that contains a specific string

Question

I have stored few files in hdfs (CSV files). Some of these files have passwords and usernames stored. I am doing it on Single node cluster and just for practice. I can not find in which files I have stored passwords because all the CSV files are named in ascending Natural numbers. There are over 50 files and I can’t manually check. Is there a way I can use pattern match to do this?

Answer 1

Yes, you can do this. You can use hdfs command to list the files and then use grep to find the pattern in those files. Example:

hadoop fs -ls pat/to/dir | grep password

This will filter the files and list only those files that have the word you are looking for.