Filtering Out the Unique records and populating the records with Max count using Talend

0 votes

I have a real-time data table in which data is updated in every 24 hours. From this table, I am pulling in the data using an excel sheet. Sometimes with regular updates, it leads to duplication of data along with null values at some places. Can someone help me in filtering out the unique rows with most number of columns?

Apr 27, 2018 in Talend by misc.edu04
• 1,450 points
3,060 views

1 answer to this question.

0 votes

Hi...I think in your case you can make use of tUniqRow component. In order to get the required output, you can make use of a tJavaRow component to specify the conditions and then sort the incoming rows and then pass them to the tUniqRow component.

Your job flow should look somewhat like below:

image

In the tJavaRow component, you need to add the below code to filter out the required data:

// Code generated according to input schema and output schema
output_row.id = input_row.id;
output_row.name       = input_row.name;
output_row.address = input_row.address;
// End of pre-generated code

int i = 0;
if (input_row.id == null) { i++; }
if (input_row.name       == null) { i++; }
if (input_row.address  == null) { i++; }
output_row.priority = i;

Hope this helps!!

Join the Talend course today to know more about and become expert.

Thanks.

answered Apr 27, 2018 by geek.erkami
• 2,680 points

Related Questions In Talend

0 votes
1 answer
0 votes
1 answer
+1 vote
1 answer
0 votes
0 answers

How can I use talend to extract data from Oracle and import the data into a MSSQL database?

We currently have around 100 Oracle tables ...READ MORE

Nov 4, 2020 in Talend by umesh
• 150 points
2,157 views
0 votes
1 answer

tWaitForFile component insufficiency in Talend

Regarding your first issue, tWaitForFile component provides ...READ MORE

answered Apr 11, 2018 in Talend by geek.erkami
• 2,680 points
2,148 views
+1 vote
1 answer
0 votes
2 answers

Closed loop in Talend

Hi, Talend specifies that tUnite cannot be ...READ MORE

answered Aug 23, 2019 in Talend by vinceall
1,867 views
0 votes
1 answer

Comparing 2 files and keeping the unique entries

Hi, To get the unique records from the ...READ MORE

answered Apr 20, 2018 in Talend by geek.erkami
• 2,680 points
1,522 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP