Filtering Out the Unique records and populating the records with Max count using Talend

0 votes

I have a real-time data table in which data is updated in every 24 hours. From this table, I am pulling in the data using an excel sheet. Sometimes with regular updates, it leads to duplication of data along with null values at some places. Can someone help me in filtering out the unique rows with most number of columns?

Apr 27, 2018 in Talend by misc.edu04
• 1,420 points
407 views

1 answer to this question.

0 votes

Hi...I think in your case you can make use of tUniqRow component. In order to get the required output, you can make use of a tJavaRow component to specify the conditions and then sort the incoming rows and then pass them to the tUniqRow component.

Your job flow should look somewhat like below:

image

In the tJavaRow component, you need to add the below code to filter out the required data:

// Code generated according to input schema and output schema
output_row.id = input_row.id;
output_row.name       = input_row.name;
output_row.address = input_row.address;
// End of pre-generated code

int i = 0;
if (input_row.id == null) { i++; }
if (input_row.name       == null) { i++; }
if (input_row.address  == null) { i++; }
output_row.priority = i;

Hope this helps!!

answered Apr 27, 2018 by geek.erkami
• 2,640 points

Related Questions In Talend

0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

Looping through all the schemas in Talend

It is really simple to make tOracleInput ...READ MORE

answered Apr 10, 2018 in Talend by code.reaper12
• 3,450 points
123 views
0 votes
1 answer

tWaitForFile component insufficiency in Talend

Regarding your first issue, tWaitForFile component provides ...READ MORE

answered Apr 10, 2018 in Talend by geek.erkami
• 2,640 points
343 views
0 votes
1 answer
0 votes
2 answers

Closed loop in Talend

Hi, Talend specifies that tUnite cannot be ...READ MORE

answered Aug 22 in Talend by vinceall
360 views
0 votes
1 answer

Comparing 2 files and keeping the unique entries

Hi, To get the unique records from the ...READ MORE

answered Apr 20, 2018 in Talend by geek.erkami
• 2,640 points
118 views