AWS Lambda read from SQS without concurrency

0 votes

My requirements are as follows.

Every two hours, all the messages in a SQS are read, collected, and processed.
A file containing information from SQS messages is created as part of processing, and it is then uploaded to a sftp server.

I used an AWS Lambda to implement step 1 of the process. My Lambda contains a sqs trigger. I've set the batch timeframe to 2 hours and the batch size to 50. My presumption was that Lambda would be activated every two hours, that it would receive 50 messages all at once, and that I would create a file for every 50 records.

But I observed that my lambda function is triggered with varied number of messages(sometimes 50 sometimes 20, sometimes 5 etc) even though I have configured batch size as 50.
After reading some documentation I got to know(I am not sure) that there are 5 long polling connections which lambda spawns to read from SQS and this is causing this behaviour of lambda function being triggered with varied number of messages.

My question is

  1. Is my assumption on 5 parallel connections being established correct? If yes, is there a way I can control it? I want this to happen in a single thread / connection
  2. If 1 is not possible, what other alternative do I have here. I do not want to have one file created for every few records. I want one file to be generated every two hours with all the messages in sqs.
Jan 3, 2023 in AWS by Tejashwini
• 3,820 points
256 views

No answer to this question. Be the first to respond.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.

Related Questions In AWS

+1 vote
2 answers

Want my AWS s3 Bucket to read Name from CloudWatch Event

CloudTrail events for S3 bucket level operations ...READ MORE

answered May 28, 2018 in AWS by Cloud gunner
• 4,670 points
1,745 views
+1 vote
3 answers

Not able to pass params POST to AWS Lambda from Amazon API Gateway

For this template : application/x-www-form-urlencoded  This line below will ...READ MORE

answered Jun 13, 2018 in AWS by Cloud gunner
• 4,670 points
8,382 views
0 votes
1 answer
+1 vote
2 answers

How to use jenkinsfile to get content from AWS SQS?

You can import standard AWS SDK to ...READ MORE

answered Jul 20, 2018 in AWS by Flying geek
• 3,280 points
2,120 views
0 votes
1 answer

Publishing messages from AWS SNS to SQS

Unless you are using SQS FIFO queues, ...READ MORE

answered Aug 24, 2018 in AWS by Priyaj
• 58,090 points
1,436 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP