How to traverse from a file to parse another file

Question

I have a central location to store the logs, I have created a directory a  and in this place, I unzip all the logs:

After unzipping it created directory b and 123/567 under a

I am looking in the file /a/b/file1.txt

if there is an err-1 in /a/b/file1.txt

then I get the file name where the error occurred from file1.txt

/u01/prod/123/567/123_apply_2018Oct17_12_28_12.log  - shows directory structure on the sysytem where the error occured 

After unzipping the file in the directory a 123/567 sub directories are created directory and files will be stored in the location :

/a/123/567/123_apply_2018Oct17_12_28_12.log

From the python script how to I traverse from /a/b/file1.txt  to parse the file /a/123/567/123_apply_2018Oct17_12_28_12.log

Do I need to use subprocess in Linux and perform manually like this:

pwd 
/a/b
cd ..
cd 123/567 

How do i extract this from the string  /u01/prod/123/567/123_apply_2018Oct17_12_28_12.log and then parse the file 123_apply_2018Oct17_12_28_12.log?

I cannot do a string substitution because placeholder directory name may change, like it may be a1 or a2, a is variable but the directory structure after that is constant.

Answer 1

You use os.walk() module of python for traversing through directories and use regex to match directory names.