How to load a set of files into database using Talend

Question

I have a set of delimited files stored in a particular directory. Now I want to read all these files at once and then load them into a database. Does Talend provide any specific component to deal with this or do we need to create any specific job?

Can anyone suggest me how can I perform this?

code.reaper12 · Answer 1 · Apr 17, 2018

Using tFileList component, you can read all the files present in the specified directory. In the component tab of tFileList, you can specify the directory name and the file mask using which Talend will browse through the files matching the pattern. To read through each and every file in the directory you need to make use of an “iterate” link and connect it to a tFileInputDelimited component.

In the component tab of tFileInputDelimited component, specify the file name using the tFileList current file expression. Something like: ((String)globalMap.get("tFileList_1_CURRENT_FILE"))

You can find this expression from the outline window in the Talend Open Studio.

Then from the tFileInputDelimited component take a Row Main link and connect to the database output component in order to load it to the tMysqlOutput component. Since you want all of your data in a single table, you can leave the action as default, which will automatically append all the data.

Hope this helps!

To know more about Talend, join Talend for big data certification training today.

Thanks.