tWaitForFile component insufficiency in Talend

0 votes

Please suggest how can I read the files from a directory which are added dynamically by a producer process using Talend tWaitForFile component. While I was trying to perform this, I was facing following two issues:

1.   tWaitForFile component can read the files which are already present in the directory, so the files added dynamically after the job has started, remains invisible to it.

2.   There is also a possibility that Talend starts reading the files, even before producer process has finished writing the files. I am not able to confirm it as _wait_release_ parameter of the tWaitForFile is not working on the Linux system.

Apr 10, 2018 in Talend by code.reaper12
• 3,450 points
313 views

1 answer to this question.

0 votes

Regarding your first issue, tWaitForFile component provides various trigger options (created, modified and deleted). You can select the “created” option to read the files when they are added into the directory.

image

Regarding the 2nd issue, you can try giving your file mask as “.OK” in tWaitForFile and search and retrieve the completed files. Masking your file with “.OK”  makes it easier for Talend to search for the relevant files. Then using an iterate link you can connect the delimited or excel or whichever format file input component you want. In my case, I am using a tFileInputDelimited component to fetch the delimited files. Finally, in the “File Name” section you can add the below code:

((String)globalMap.get("tWaitForFile_1_CREATED_FILE")).substring(0,((String)globalMap.get("tWaitForFile_1_CREATED_FILE")).length()-3) + ".txt"

image

answered Apr 10, 2018 by geek.erkami
• 2,320 points

Related Questions In Talend

0 votes
1 answer

Looping through all the schemas in Talend

It is really simple to make tOracleInput ...READ MORE

answered Apr 10, 2018 in Talend by code.reaper12
• 3,450 points
107 views
0 votes
1 answer

Parsing JSON in Talend

While working with JSON files, one thing ...READ MORE

answered Apr 2, 2018 in Talend by code.reaper12
• 3,450 points
517 views
0 votes
1 answer
0 votes
1 answer
0 votes
2 answers

Closed loop in Talend

Hi, Talend specifies that tUnite cannot be ...READ MORE

answered Aug 22 in Talend by vinceall
335 views
0 votes
1 answer

Implement Common Logging in Talend

Well, you can try using following 2 ...READ MORE

answered Apr 11, 2018 in Talend by geek.erkami
• 2,320 points
139 views
0 votes
1 answer

Using Java Classes in Talend

While working with routines, the very 1st ...READ MORE

answered Apr 14, 2018 in Talend by geek.erkami
• 2,320 points
415 views