Adding Cache-Control and Expires headers to Azure Storage Blobs

0 votes

Hi all

I'm using Azure Storage to serve up static file blobs but I'd like to add a Cache-Control and Expires header to the files/blobs when served up to reduce bandwidth costs.

Application like CloudXplorer and Cerebrata's Cloud Storage Studio give options to set metadata properties on containers and blobs but get upset when trying to add Cache-Control.

Anyone know if it's possible to set these headers for files? 

TIA

Aug 6, 2018 in Azure by cloudie_crank
• 1,610 points
279 views

2 answers to this question.

Your answer

Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
0 votes

I had the same question in my mind and I figured it out after going through few research and references.

I had to run a batch job on about 600k blobs and found 2 things that really helped:

  1. Running the operation from a worker role in the same data center. The speed between Azure services is great as long as they are in the same affinity group. Plus there are no data transfer costs.
  2. Running the operation in parallel. The Task Parallel Library (TPL) in .net v4 makes this really easy. Here is the code to set the cache-control header for every blob in a container in parallel:

// get the info for every blob in the container
var blobInfos = cloudBlobContainer.ListBlobs(
    new BlobRequestOptions() { UseFlatBlobListing = true });
Parallel.ForEach(blobInfos, (blobInfo) =>
{
    // get the blob properties
    CloudBlob blob = container.GetBlobReference(blobInfo.Uri.ToString());
    blob.FetchAttributes();

    // set cache-control header if necessary
    if (blob.Properties.CacheControl != YOUR_CACHE_CONTROL_HEADER)
    {
        blob.Properties.CacheControl = YOUR_CACHE_CONTROL_HEADER;
        blob.SetProperties();
    }
});

 

answered Aug 6, 2018 by club_seesharp
• 3,450 points
0 votes

Here's an updated version of the previous answer:

Instead of creating a website and using a WorkerRole, Azure now has the ability to run "WebJobs". You can run any executable on demand on a website at the same datacenter where your storage account is located to set cache headers or any other header field.

  1. Create a throw-away, temporary website in the same datacenter as your storage account. Don't worry about affinity groups; create an empty ASP.NET site or any other simple site. The content is unimportant.
  2. Create a console program using the code below which works with the updated Azure Storage APIs. Compile it for release, and then zip the executable and all required DLLs into a .zip file.
  3. Create a WebJob and upload the .zip file from step #2.

​​​

4. Run the WebJob. Everything written to the console is available to view in the log file created and accessible from the WebJob control page.

5. Note the UpdateAzureServiceVersion method. Apparently, by default, Azure storage serves improperly formatted ETags so you may wish to run this code once, for details see: this

The code below runs a separate task for each container, and I'm getting about 70 headers updated per second per container. No egress charges.

using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Auth;
using Microsoft.WindowsAzure.Storage.Blob;

namespace AzureHeaders
{
    class Program
    {
        static StorageCredentials storageCredentials =
            new StorageCredentials("azureaccountname", @"azzureaccountkey");
        private static string newCacheSettings = "public, max-age=7776000"; // 3 months
        private static string[] containersToProcess = { "container1", "container2" };

        static void Main(string[] args)
        {
            var account = new CloudStorageAccount(
                storageCredentials,
                false /* useHttps */);

            CloudBlobClient blobClient = account.CreateCloudBlobClient();

            var tasks = new List<Task>();
            foreach (var container in blobClient.ListContainers())
            {
                if (containersToProcess.Contains(container.Name))
                {
                    var c = container;
                    tasks.Add(Task.Run(() => FixHeaders(c)));
                }
            }
            Task.WaitAll(tasks.ToArray());
        }

        private static async Task FixHeaders(CloudBlobContainer cloudBlobContainer)
        {
            int totalCount = 0, updateCount = 0, errorCount = 0;

            Console.WriteLine("Starting container: " + cloudBlobContainer.Name);
            IEnumerable<IListBlobItem> blobInfos = cloudBlobContainer.ListBlobs(useFlatBlobListing: true);

            foreach (var blobInfo in blobInfos)
            {
                try
                {
                    CloudBlockBlob blockBlob = (CloudBlockBlob)blobInfo;
                    var blob = await cloudBlobContainer.GetBlobReferenceFromServerAsync(blockBlob.Name);
                    blob.FetchAttributes();

                    // set cache-control header if necessary
                    if (blob.Properties.CacheControl != newCacheSettings)
                    {
                        blob.Properties.CacheControl = newCacheSettings;
                        blob.SetProperties();
                        updateCount++;
                    }
                }
                catch (Exception ex)
                {
                    // Console.WriteLine(ex.Message);
                    errorCount++;
                }
                totalCount++;
            }
            Console.WriteLine("Finished container: " + cloudBlobContainer.Name + 
                ", TotalCount = " + totalCount + 
                ", Updated = " + updateCount + 
                ", Errors = " + errorCount);
        }

        // http://geekswithblogs.net/EltonStoneman/archive/2014/10/09/configure-azure-storage-to-return-proper-response-headers-for-blob.aspx
        private static void UpdateAzureServiceVersion(CloudBlobClient blobClient)
        {
            var props = blobClient.GetServiceProperties();
            props.DefaultServiceVersion = "2014-02-14";
            blobClient.SetServiceProperties(props);
        }
    }
}

answered Aug 6, 2018 by null_void
• 3,220 points

Related Questions In Azure

0 votes
1 answer
0 votes
1 answer

Why would you choose Blob Storage over File Storage which also has ability to store file(s) and folder(s)

Azure Files: Azure File storage provides shared ...READ MORE

answered Apr 13, 2018 in Azure by null_void
• 3,220 points

edited Apr 13, 2018 by null_void 25 views
0 votes
1 answer

How can I remove/hide/disable excessive HTTP response headers in Azure/IIS7 without having to use UrlScan?

MSDN published an article on how to ...READ MORE

answered May 21, 2018 in Azure by club_seesharp
• 3,450 points
224 views
0 votes
1 answer

How can i upload to Azure Blob storage with Shared Access key?

For GetBlobReferenceFromServer to work, the blob must be present ...READ MORE

answered Jun 12, 2018 in Azure by club_seesharp
• 3,450 points
486 views
0 votes
1 answer

How to choose between Azure App Service and Azure Service Fabric?

Microsoft has created the document with a comparison for ...READ MORE

answered Jun 13, 2018 in Azure by null_void
• 3,220 points
92 views
0 votes
1 answer

Can we track progress of async file upload to azure storage?

Actually it's not possible because uploading file is ...READ MORE

answered Jun 14, 2018 in Azure by club_seesharp
• 3,450 points
395 views
0 votes
1 answer

How to choose between Azure Webjobs and Azure Functions?

There are a couple options here within ...READ MORE

answered Jul 9, 2018 in Azure by club_seesharp
• 3,450 points
779 views
0 votes
1 answer

Stream uploaded file to Azure blob storage with Node

Using Multiparty(npm install multiparty), a fork of ...READ MORE

answered Sep 24, 2018 in Azure by club_seesharp
• 3,450 points
632 views
0 votes
1 answer

How to upload a file on to Azure Blob storage without writing a code?

You can find the below tools useful ...READ MORE

answered Apr 13, 2018 in Azure by club_seesharp
• 3,450 points
68 views
0 votes
2 answers

How can I download a .vhd image to my local machine from azure and upload the same to a different azure account?

From the Windows Azure Portal you can ...READ MORE

answered Aug 20, 2018 in Azure by Priyaj
• 56,120 points
517 views

© 2018 Brain4ce Education Solutions Pvt. Ltd. All rights Reserved.
"PMP®","PMI®", "PMI-ACP®" and "PMBOK®" are registered marks of the Project Management Institute, Inc. MongoDB®, Mongo and the leaf logo are the registered trademarks of MongoDB, Inc.