How can we list files in HDFS directory as per timestamp

+1 vote

Is there a HDFS command that can list files in HDFS directory as per timestamp in ascending or descending order? 

By default, hdfs dfs -ls command gives unsorted list of files.

When I searched around, I found something of this sort i.e. 

hdfs dfs -ls /tmp | sort -k6,7 

Can this be done using any inbuilt hdfs dfs commandline? Or is there any better way?

May 8, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
11,210 views

1 answer to this question.

+1 vote

No, there is no other option to sort the files based on datetime.
If you are using hadoop version < 2.7, you will have to use sort -k6,7 as you are doing:

hdfs dfs -ls /tmp | sort -k6,7
And for hadoop 2.7.x ls command , there are following options available :

Usage: hadoop fs -ls [-d] [-h] [-R] [-t] [-S] [-r] [-u] <args>

Options:
-d: Directories are listed as plain files.
-h: Format file sizes in a human-readable fashion (eg 64.0m instead of 67108864).
-R: Recursively list subdirectories encountered.
-t: Sort output by modification time (most recent first).
-S: Sort output by file size.
-r: Reverse the sort order.
-u: Use access time rather than modification time for display and sorting.
So you can easily sort the files:

hdfs dfs -ls -t -R (-r) /tmp 

Hope this will answer your query to some extent.

answered May 8, 2018 by nitinrawat895
• 11,380 points

Related Questions In Big Data Hadoop

0 votes
5 answers
0 votes
5 answers

Hadoop hdfs: list all files in a directory and its subdirectories

Hi, You can try this command: hadoop fs -ls ...READ MORE

answered Aug 1, 2019 in Big Data Hadoop by Dinish
17,300 views
0 votes
1 answer

How to list files in hdfs that contains a specific string?

Yes, you can do this. You can ...READ MORE

answered Jan 27, 2019 in Big Data Hadoop by Omkar
• 69,210 points
10,902 views
0 votes
1 answer

How many files can hive create in HDFS duringbMapReduce job?

There is a limit on how many ...READ MORE

answered May 21, 2019 in Big Data Hadoop by Charuni
1,025 views
–1 vote
1 answer

Hadoop dfs -ls command?

In your case there is no difference ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by kurt_cobain
• 9,390 points
4,260 views
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
10,559 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
104,215 views
0 votes
1 answer
0 votes
1 answer

How can we transfer a PDF file to HDFS?

You can easily upload any file to ...READ MORE

answered Apr 13, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
842 views
0 votes
1 answer

How can I download only hdfs and not hadoop?

No, you cannot download HDFS alone because ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
1,094 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP