How can we list files in HDFS directory as per timestamp?

0 votes

Is there a HDFS command that can list files in HDFS directory as per timestamp in ascending or descending order? 

By default, hdfs dfs -ls command gives unsorted list of files.

When I searched around, I found something of this sort i.e. 

hdfs dfs -ls /tmp | sort -k6,7 

Can this be done using any inbuilt hdfs dfs commandline? Or is there any better way?

May 8, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
800 views

1 answer to this question.

0 votes

No, there is no other option to sort the files based on datetime.
If you are using hadoop version < 2.7, you will have to use sort -k6,7 as you are doing:

hdfs dfs -ls /tmp | sort -k6,7
And for hadoop 2.7.x ls command , there are following options available :

Usage: hadoop fs -ls [-d] [-h] [-R] [-t] [-S] [-r] [-u] <args>

Options:
-d: Directories are listed as plain files.
-h: Format file sizes in a human-readable fashion (eg 64.0m instead of 67108864).
-R: Recursively list subdirectories encountered.
-t: Sort output by modification time (most recent first).
-S: Sort output by file size.
-r: Reverse the sort order.
-u: Use access time rather than modification time for display and sorting.
So you can easily sort the files:

hdfs dfs -ls -t -R (-r) /tmp 

Hope this will answer your query to some extent.

answered May 8, 2018 by nitinrawat895
• 10,670 points

Related Questions In Big Data Hadoop

0 votes
5 answers
0 votes
5 answers

Hadoop hdfs: list all files in a directory and its subdirectories

Hi, You can try this command: hadoop fs -ls ...READ MORE

answered Aug 1 in Big Data Hadoop by Dinish
1,666 views
0 votes
1 answer

How to list files in hdfs that contains a specific string?

Yes, you can do this. You can ...READ MORE

answered Jan 27 in Big Data Hadoop by Omkar
• 67,480 points
555 views
0 votes
1 answer
0 votes
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,240 points
993 views
0 votes
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 10,670 points
2,736 views
0 votes
10 answers

hadoop fs -put command?

put syntax: put <localSrc> <dest> copy syntax: copyFr ...READ MORE

answered Dec 7, 2018 in Big Data Hadoop by Aditya
13,561 views
0 votes
1 answer
0 votes
1 answer

How can we transfer a PDF file to HDFS?

You can easily upload any file to ...READ MORE

answered Apr 13, 2018 in Big Data Hadoop by nitinrawat895
• 10,670 points
75 views
0 votes
1 answer

How can I download only hdfs and not hadoop?

No, you cannot download HDFS alone because ...READ MORE

answered Mar 15, 2018 in Big Data Hadoop by nitinrawat895
• 10,670 points
83 views