AWS kinesis producer writing same record multiple times

0 votes

I am writing data to AWS kinesis stream using putRecordsRequest method to write a batch of putRecordsRequestEntryList. Hence writing is successful but each record is sent multiple times as bellow:

Fetched Record: shardId: shardId-000000000001 partitionKey: 1a9473be8d0ea31f7f79a571bf0db79b sequenceNumber: 49582947864364362927706152534174639194775987182159003666 data: 1522044380916 =>Fetched Record: shardId: shardId-000000000001 partitionKey: 1a9473be8d0ea31f7f79a571bf0db79b sequenceNumber: 49582947864364362927706152534197608785348665205197897746 data: 1522044380916

But when I checked producer log, it was sent for only once. Any help please...

Oct 29, 2018 in AWS by findingbugs
• 3,260 points
516 views

1 answer to this question.

0 votes

"Fetched Record" implies you're reading the stream. If so, are you checkpointing? Please add more detail including sample code for your producer and your consumer. I am inclined to believe you are only writing the record to the stream once, but you're fetching it multiple times because you're not checkpointing.

answered Oct 29, 2018 by Priyaj
• 58,090 points

Related Questions In AWS

+1 vote
1 answer

How many times can one receive the AWS free tier?

Yes, it is possible to create a ...READ MORE

answered Jul 27, 2018 in AWS by Priyaj
• 58,090 points
1,316 views
0 votes
1 answer

How to switch between multiple accounts with AWS SDK

You can define multiple profiles in ~/.aws/credentials ...READ MORE

answered Aug 24, 2018 in AWS by Priyaj
• 58,090 points
1,855 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

Creating multiple sub accounts in IAM AWS!

you can check this link and proceed ...READ MORE

answered Jul 11, 2018 in AWS by Priyaj
• 58,090 points
2,087 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP