Comparing 2 files and keeping the unique entries

Question

Can someone help me in creating a Job where I can take data from a database table and an excel file, map them to keep only the records unique to the database table and finally load the resultant data into a delimited file? Also, I don’t want any duplicates in my final data. I have tried many ways of doing it but still no success.

Answer 1

Hi,

To get the unique records from the database table you can try the following data flow:

Make sure to keep your database input as main flow and the excel iput as a lookup.

Once you are done setting up the components, go to the tMap Editor and perform the following operations:

1. Click on tMap settings of row2 table.

2. Set the “Match model” and “Join Model” as “Unique Match” and “Inner Join” respectively.

3. Click on tMap settings of output table.

4. Set the “Catch look up inner join reject” as “true”

5. Map the rows from the database incoming flow to the output table

And there you go.

Execute the job, you will get the required output with all unique records.