I have a job where I need to update data from sql rds to redshift on a daily basis.
Below is my Job flow:
![image](https://lh3.googleusercontent.com/Wx_tG5IQdhNOAo7y85kixKUr3srbUW7KW8Ta64tpopegHHDMpadhhKFvQCT7YqsteecG5DIO_rGVngsfkPc3_y2nUwTcW1EsmHNyeidQgf1-jL1QMplPzXKa_08VkntUlyo1YVpC)
And below is the query I am using to retrieve the data:
SELECT * FROM abc.xyz
WHERE id > (SELECT COALESCE(max(id), 0) as id
FROM abc.demo)
Here:
abc→ myrds Table
Demo → redshift table
The job executes file but the rate of data transfer is very slow.
Can someone help me in speeding up my Job?