tWaitForFile component insufficiency in Talend

Question

Please suggest how can I read the files from a directory which are added dynamically by a producer process using Talend tWaitForFile component. While I was trying to perform this, I was facing following two issues:

1.   tWaitForFile component can read the files which are already present in the directory, so the files added dynamically after the job has started, remains invisible to it.

2.   There is also a possibility that Talend starts reading the files, even before producer process has finished writing the files. I am not able to confirm it as _wait_release_ parameter of the tWaitForFile is not working on the Linux system.

Answer 1

Regarding your first issue, tWaitForFile component provides various trigger options (created, modified and deleted). You can select the “created” option to read the files when they are added into the directory.

Regarding the 2^nd issue, you can try giving your file mask as “.OK” in tWaitForFile and search and retrieve the completed files. Masking your file with “.OK”  makes it easier for Talend to search for the relevant files. Then using an iterate link you can connect the delimited or excel or whichever format file input component you want. In my case, I am using a tFileInputDelimited component to fetch the delimited files. Finally, in the “File Name” section you can add the below code:

((String)globalMap.get("tWaitForFile_1_CREATED_FILE")).substring(0,((String)globalMap.get("tWaitForFile_1_CREATED_FILE")).length()-3) + ".txt"

Hope this was what you were looking for. 

Get your Talend certification today to become certified professional.

Thanks.