What is the HDFS command to list all the files in HDFS according to the timestamp

Question

What is the command to list the directories in HDFS as per timestamp? I tried hdfs dfs ls -l which provides the list of directories with their respective permissions. I tried a workaround with hdfs -dfs -ls /tmp | sort -k6,7. Is there an inbuilt hdfs command for this?

Answer 1

The following arguments are available with hadoop ls command:

Usage: hadoop fs -ls [-d] [-h] [-R] [-t] [-S] [-r] [-u] <args>

Options:

-d: Directories are listed as plain files.
-h: Format file sizes in a human-readable fashion (eg 64.0m instead of 67108864).
-R: Recursively list subdirectories encountered.
-t: Sort output by modification time (most recent first).
-S: Sort output by file size.
-r: Reverse the sort order.
-u: Use access time rather than modification time for display and sorting.

You can sort the files using following command:

hdfs dfs -ls -t -R (-r) /tmp 

For details, You can even check out Hadoop Ecosystem tools with the Training for big data.

Answer 2

ls -R -t (-r) should do the job

Answer 3

hdfs dfs -ls /test | sort -k6,7 is simple and easy. I dont see a reason why you want to use something else.

Answer 4

hdfs dfs -ls -t1

Comments

This does not list files by timestamp. It only lists files by time

Answer 5

You can try filter using value in ls.

ls -ltr | awk '$6 == "<Month of the year>" && $7 == <day of the month> {print $9}