Adding Cache-Control and Expires headers to Azure Storage Blobs

Question

Hi all

I'm using Azure Storage to serve up static file blobs but I'd like to add a Cache-Control and Expires header to the files/blobs when served up to reduce bandwidth costs.

Application like CloudXplorer and Cerebrata's Cloud Storage Studio give options to set metadata properties on containers and blobs but get upset when trying to add Cache-Control.

Anyone know if it's possible to set these headers for files? 

TIA

Answer 1

I had the same question in my mind and I figured it out after going through few research and references.

I had to run a batch job on about 600k blobs and found 2 things that really helped:

1. Running the operation from a worker role in the same data center. The speed between Azure services is great as long as they are in the same affinity group. Plus there are no data transfer costs.

2. Running the operation in parallel. The Task Parallel Library (TPL) in .net v4 makes this really easy. Here is the code to set the cache-control header for every blob in a container in parallel:

// get the info for every blob in the container
var blobInfos = cloudBlobContainer.ListBlobs(
    new BlobRequestOptions() { UseFlatBlobListing = true });
Parallel.ForEach(blobInfos, (blobInfo) =>
{
    // get the blob properties
    CloudBlob blob = container.GetBlobReference(blobInfo.Uri.ToString());
    blob.FetchAttributes();

    // set cache-control header if necessary
    if (blob.Properties.CacheControl != YOUR_CACHE_CONTROL_HEADER)
    {
        blob.Properties.CacheControl = YOUR_CACHE_CONTROL_HEADER;
        blob.SetProperties();
    }
});

Hope it helps!

If you want to become Azure solution architect, then join Azure online course today.

Thank you!!

Answer 2

Here's an updated version of the previous answer:

Instead of creating a website and using a WorkerRole, Azure now has the ability to run "WebJobs". You can run any executable on demand on a website at the same datacenter where your storage account is located to set cache headers or any other header field.

1. Create a throw-away, temporary website in the same datacenter as your storage account. Don't worry about affinity groups; create an empty ASP.NET site or any other simple site. The content is unimportant.
2. Create a console program using the code below which works with the updated Azure Storage APIs. Compile it for release, and then zip the executable and all required DLLs into a .zip file.
3. Create a WebJob and upload the .zip file from step #2.

​​​

4. Run the WebJob. Everything written to the console is available to view in the log file created and accessible from the WebJob control page.

5. Note the UpdateAzureServiceVersion method. Apparently, by default, Azure storage serves improperly formatted ETags so you may wish to run this code once, for details see: this

The code below runs a separate task for each container, and I'm getting about 70 headers updated per second per container. No egress charges.

using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Auth;
using Microsoft.WindowsAzure.Storage.Blob;

namespace AzureHeaders
{
    class Program
    {
        static StorageCredentials storageCredentials =
            new StorageCredentials("azureaccountname", @"azzureaccountkey");
        private static string newCacheSettings = "public, max-age=7776000"; // 3 months
        private static string[] containersToProcess = { "container1", "container2" };

        static void Main(string[] args)
        {
            var account = new CloudStorageAccount(
                storageCredentials,
                false /* useHttps */);

            CloudBlobClient blobClient = account.CreateCloudBlobClient();

            var tasks = new List<Task>();
            foreach (var container in blobClient.ListContainers())
            {
                if (containersToProcess.Contains(container.Name))
                {
                    var c = container;
                    tasks.Add(Task.Run(() => FixHeaders(c)));
                }
            }
            Task.WaitAll(tasks.ToArray());
        }

        private static async Task FixHeaders(CloudBlobContainer cloudBlobContainer)
        {
            int totalCount = 0, updateCount = 0, errorCount = 0;

            Console.WriteLine("Starting container: " + cloudBlobContainer.Name);
            IEnumerable<IListBlobItem> blobInfos = cloudBlobContainer.ListBlobs(useFlatBlobListing: true);

            foreach (var blobInfo in blobInfos)
            {
                try
                {
                    CloudBlockBlob blockBlob = (CloudBlockBlob)blobInfo;
                    var blob = await cloudBlobContainer.GetBlobReferenceFromServerAsync(blockBlob.Name);
                    blob.FetchAttributes();

                    // set cache-control header if necessary
                    if (blob.Properties.CacheControl != newCacheSettings)
                    {
                        blob.Properties.CacheControl = newCacheSettings;
                        blob.SetProperties();
                        updateCount++;
                    }
                }
                catch (Exception ex)
                {
                    // Console.WriteLine(ex.Message);
                    errorCount++;
                }
                totalCount++;
            }
            Console.WriteLine("Finished container: " + cloudBlobContainer.Name + 
                ", TotalCount = " + totalCount + 
                ", Updated = " + updateCount + 
                ", Errors = " + errorCount);
        }

        // http://geekswithblogs.net/EltonStoneman/archive/2014/10/09/configure-azure-storage-to-return-proper-response-headers-for-blob.aspx
        private static void UpdateAzureServiceVersion(CloudBlobClient blobClient)
        {
            var props = blobClient.GetServiceProperties();
            props.DefaultServiceVersion = "2014-02-14";
            blobClient.SetServiceProperties(props);
        }
    }
}