Inserting 40 50 records to Azure table storage sometimes taking more than 30 seconds hence throwing timeout exception

0 votes

Here is my code

ThreadPool.SetMinThreads(1024, 256);
ServicePointManager.DefaultConnectionLimit = 256;
ServicePointManager.UseNagleAlgorithm = false;
ServicePointManager.Expect100Continue = false;
client.DefaultRequestOptions = new TableRequestOptions
            {
                MaximumExecutionTime = TimeSpan.FromSeconds(30), //Timeout requests after 30 seconds
                RetryPolicy = new ExponentialRetry(TimeSpan.FromSeconds(3), 4),
                LocationMode = LocationMode.PrimaryThenSecondary
            };  
var tableEntityGroups = histories.Select(h => new TrackHistoryTableEntity(h)).GroupBy(e => e.PartitionKey).ToDictionary(g => g.Key, g => g.ToList());
            List<Task> tasks = new List<Task>();
            foreach (var kvp in tableEntityGroups)
            {
                //Merge Track history records with the same FixTaken second into one, taking the average 
                var mergedHistories = kvp.Value.GroupBy(v => v.RowKey).Select(g => new TrackHistoryTableEntity()
                {
                    PartitionKey = g.First().PartitionKey,
                    RowKey = g.First().RowKey,
                    A = g.Select(v => v.A).Average(),
                    N = g.Select(v => v.N).Average(),
                    V = g.Select(v => v.V).Average(),
                    B = g.Select(v => v.B).Average(),
                    D = g.Select(v => v.D).Sum()
                });
                TableBatchOperation batchOperation = new TableBatchOperation();
                foreach (var v in mergedHistories)
                {
                    batchOperation.Add(TableOperation.InsertOrReplace(v));
                    if (batchOperation.Count >= 100)
                    {
                        tasks.Add(TrackHistoryTable.ExecuteBatchAsync(batchOperation));
                        batchOperation = new TableBatchOperation();
                    }
                }
                if (batchOperation.Count > 0)
                {
                    tasks.Add(TrackHistoryTable.ExecuteBatchAsync(batchOperation));
                }

                var splitKey = kvp.Value[0].PartitionKey.Split('_');
                tasks.Add(TrackHistoryTracksTable.ExecuteAsync(TableOperation.InsertOrReplace(new TableEntity(splitKey[0], Int32.Parse(splitKey[1]).ToString()))));

                if (trackPartitionUpdates)

I've got a long-running application that's supposed to insert data every 2/3 seconds or so. The majority of the time, it works perfectly. However, I occasionally receive a time-out exception. Every time I checked, it was inserting roughly 50 records. I checked with a higher load, such as over 2000 rows. It performs admirably. It only throws a timeout exception a few times every day.

Mar 9, 2022 in Azure by Edureka
• 13,620 points
1,298 views

1 answer to this question.

0 votes

Here are a couple of things to think about:

[CAUTION] Batch table operations have a maximum processing time SLA of 30 seconds, compared to 2 seconds for single entity operations. More information can be found at https://azure.microsoft.com/en-us/support/legal/sla/storage/v1 5/. .
[BEST PRACTICE] Formalized paraphrase Implement a policy of retries (e.g. preferably exponential retry batching use cases and considering your SLA). More information is available at: https://docs.microsoft.com/en-us/azure/architecture/best-practices/retry-service-specific#azure-storage.

answered Mar 24, 2022 by Edureka
• 12,690 points

Related Questions In Azure

0 votes
1 answer

azure table storage doesn't add new fields

Add new properties in the class that ...READ MORE

answered Mar 4, 2022 in Azure by Edureka
• 13,620 points
886 views
0 votes
1 answer

How to retrieve View definition on Synapse (Azure SQL DW)?

To authenticate with Azure SQL Database or ...READ MORE

answered Mar 4, 2022 in Azure by Edureka
• 13,620 points
1,685 views
0 votes
1 answer

TCP in Azure IoT Hub

The default Protocol Gateway samples are indeed ...READ MORE

answered Oct 11, 2018 in IoT (Internet of Things) by DataKing99
• 8,250 points
2,255 views
0 votes
1 answer

connecting to Power BI API using non-interactive authentication?

Try changing the resource URI: var result = ...READ MORE

answered Oct 30, 2018 in Power BI by Shubham
• 13,490 points
2,148 views
0 votes
1 answer

How to install C#

Hi @Akanksha, C# is a programming language ...READ MORE

answered Feb 28, 2019 in Others by Pratibha
• 3,690 points
1,063 views
0 votes
1 answer

How to run C# code on Ubuntu

Hey @Abha, you can create and run ...READ MORE

answered Mar 1, 2019 in Others by Anvi
• 14,150 points
6,404 views
0 votes
1 answer

Azure AzCopy command fail to copy csv files to Azure Storage

try using the below command: azcopy copy [source] ...READ MORE

answered Mar 24, 2022 in Azure by Edureka
• 12,690 points
3,312 views
0 votes
1 answer
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP