How to find previous records from a data set in Pig

Question

Hi everyone

I'm trying to find previous records from a large dataset in Pig. Say I want to find the data of last 1 month.

Which command will be used for that.

Can anyone please answer me earliest.

Thank you

Answer 1

hi @Nadeem,

Convert your date field to DateTime data type using the ToDate() function. Use the CurrentTime() and get the difference between the two dates using DaysBetween() and filter accordingly.

Hope this helps :)

Comments

Thanks for your reply.

---

Can you please share the structure of your dataset or your dataset. I'll help you with it.

Answer 2

Hi,

You can use ToDate() and SubtractDuration() function to find the previous records.

Say, You want to find the previous 100 days records from a file. 

$ Filter data by ToDate(Inspection_Date,'MM/dd/yyyy') > SubtractDuration(CurrentTime(),'P100D');

Hope this will work.

Thank You