IOPS or Throughput?

0 votes
If IOPS is the bottleneck, I can easily upgrade to Provisioned IOPS. But if network latency is the issue, I will need to redesign our load jobs to load raw data from EC2 instances instead of our remote servers, which will take some time to implement.

Determining Write Bottleneck in Amazon RDS Instance. We have nightly load jobs that writes several hundred thousand records to an Mysql reporting database running in Amazon RDS.

The load jobs are taking several hours to complete, but I am having a hard time figuring out where the bottleneck is.

The instance is currently running with General Purpose (SSD) storage. By looking at the cloudwatch metrics, it appears I am averaging less than 50 IOPS for the last week. However, Network Receive Throughput is less than 0.2 MB/sec.

Is there anyway to tell from this data if I am being bottlenecked by network latency (we are currently loading the data from a remote server...this will change eventually) or by Write IOPS?

Any advice is appreciated.
Nov 14, 2018 in AWS by Jino
• 5,520 points
32 views

1 answer to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes
Configure your database driver to print statistics about the database workload. This might inform you that you are issuing far more queries than you would expect. A very quick test is to buy provisioned IOPS, but be careful as you may get fewer than you do currently during a burst.

Another quick means to determine your bottleneck is to profile your job execution application with a profiler that understands your database driver. If you're using Java, JProfiler will show the characteristics of your job and it's use of the database.
answered Nov 14, 2018 by Theodor
• 740 points

Related Questions In AWS

+4 votes
5 answers

Usage of Amazon Cloudfront or S3

When to use S3? S3 is like many ...READ MORE

answered Apr 3, 2018 in AWS by brat_1
• 7,080 points
59 views
0 votes
1 answer
0 votes
1 answer

if instance is a part of some AutoScaling Group in AWS or not ?

You can run this command below on ...READ MORE

answered Jun 1, 2018 in AWS by Cloud gunner
• 4,240 points
21 views
0 votes
1 answer

Which cloud hosting is best ( AWS or Azure or Google)

Hello @Hammer, This totally depends on your needs, ...READ MORE

answered Aug 10, 2018 in AWS by Priyaj
• 56,120 points
35 views
0 votes
1 answer

AWS S3 or DynamoDB ?

I have used both Dynamodb and S3. ...READ MORE

answered Jun 28, 2018 in AWS by Flying geek
• 3,130 points
63 views
0 votes
1 answer

How can I install curl 7.34 or later on Amazon linux old instance

got answer from this link https://www.digitalocean.com/community/questions/how-to-upgrade-curl-in-centos6 worked for ...READ MORE

answered Aug 9, 2018 in AWS by Priyaj
• 56,120 points
419 views
+1 vote
2 answers

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.