Upload and Download Files with Web API and Azure Blob Storage

Before Amazon S3 existed, if your web application needed to handle the uploading and storing of files, you basically had the following options: put them on the web server file system, offload them to a NAS/SAN device, or shove them into the database. Each method had its own issues, but Amazon S3 came along and said, “Screw that, just store all those files in one big bucket in the cloud”, and so that’s now the norm.

Enter Azure Blob Storage on Microsoft’s Azure platform; its rival to Amazon S3. Up until spring of 2014, storing items in Azure Blob Storage was roughly 2x the cost of Amazon S3, but now that the pricing between the two platforms are identical, it makes Azure Blob Storage a much more attractive option.

So let’s say you’re using ASP.NET Web API to build a RESTful API for your app, you need endpoints that support uploading and downloading files, and you’ve decided to use Azure Blob Storage. Great, I have a solution for you.

Add Reference to Windows Azure Storage Library

First things first, use the Nuget package manager to reference the Windows Azure Storage library.

Blob Storage Config Settings

Second thing you need to do is add the following two config entries as <appSettings> in your web.config file – BlobStorageConnectionString and BlobStorageContainerName:

Models for Upload and Download

For this solution you need a couple model classes – BlobUploadModel and BlobDownloadModel – that are used to hold certain pieces of data when uploading and downloading files, respectively. Feel free to put these in the Models folder of your Web API project:

API Routes

With that done, go ahead and add the two routes (one for uploading, one for downloading) to your WebApiConfig.cs file, typically located in your App_Start folder:

BlobsController

Next up is the BlobsController with the actions for uploading and downloading files:

A few things to note about the BlobsController:

  • It calls into a BlobService that does the actual work.
  • Both action methods use async Task, mainly for better support of handling large files.
  • The PostBlobUpload() action:
    • Supports uploading multiple files at once.
    • Returns a list of BlobUploadModel.
    • Has a check to force allowing only multipart form data; meaning, this endpoint only accepts files being posted to it.
  • The GetBlobDownload() action:
    • Takes in a unique blob ID in order to retrieve the blob; this is an ID stored in your database.
    • Returns HttpResponseMessage instead of IHttpActionResult. IMPORTANT!
    • Resets the position of the blob stream that gets returned from the BlobService; otherwise, the download won’t happen.
    • Sets the proper response headers for content type, disposition, and length.

BlobService Interface and Implementation

Below is the IBlobService interface and its BlobService implementation class:

There are a few things to call out here:

  • The async Task approach continues.
  • The UploadBlobs() method:
    • Supports multiple files at once.
    • Creates an instance of the custom class BlobStorageUploadProvider, which gets passed to the ReadAsMultipartAsync() call to perform the actual upload of files.
    • Has a TODO for you to take the data returned from Azure and store it in your own database so that you can retrieve it later through the API properly. You can see an example of this data further down this post.
  • The DownloadBlob() method:
    • Has a TODO for you to implement a helper method that retrieves the blob info from your database, based on the blobId that was passed in.
    • Calls a static BlobHelper class to get the blob container name.
    • Downloads the file asynchronously from Azure into a memory stream that gets returned as part of the BlobDownloadModel result. The closing and disposing of this memory stream is handled by the Web API framework.

Static BlobHelper Class

Here’s the small BlobHelper class, which contains a single static method for getting a reference to your blob container. Notice that it reads the BlobStorageConnectionString and BlobStorageContainerName <appSettings> from config that we defined earlier:

BlobStorageUploadProvider

Now let’s look at the BlobStorageUploadProvider, which performs the actual file uploads into Azure:

Some notes about this class:

  • It inherits from MultipartFileStreamProvider.
  • It exposes a property named Uploads, which contains the list of files that were uploaded.
  • It uses Path.GetTempPath() as the temporary location on disk to store the files before sending them to Azure.
  • Also uses the static BlobHelper to get the blob container.
  • It overrides ExecutePostProcessingAsync() to inject the uploading of the files to Azure, then calls the base to complete the task.

Sample JSON Output for Blob Uploads

With all that in place, you can now use those endpoints to upload and download files into Azure Blob Storage. Here’s a sample JSON output from the API when uploading two files into Azure:

Here you can see the data you should store in your own database to maintain a relationship between your application and Azure.

The download API endpoint works as you’d expect – give it the blobId from your database and hit it from a browser or tool like Postman or Fiddler; you’ll get prompted to download the requested file.

Increase Request Limits

The default request size for ASP.NET is 4MB, so you’ll need to increase that limit if you want to allow uploading and downloading of larger files. You can do this by setting the maxRequestLength and maxAllowedContentLength values in your web.config file.

NOTE: the maxRequestLength value is in kilobytes, but the maxAllowedContentLength value is in bytes.

Here’s an example for increasing request limits to 100MB:

Sequence Diagrams

It probably seems like there’s a lot here, but it’s actually pretty straight forward when you break it down. To help paint the bigger picture, here are two sequence diagrams that show the parts involved for uploading and downloading files through the API into Azure (click each one to open full size image):

Upload Files to Azure Blob Storage

sequence-upload-files-azure-blob-storage

Download File from Azure Blob Storage

sequence-download-file-azure-blob-storage

Summary

I’ve spent a fair amount of time refining this design and think it will prove useful if you have similar needs. One pleasant surprise was Azure’s performance; uploading and downloading files was much quicker than I expected. Enjoy.

Featured Image: All rights reserved by Luigi R. Viggiano

24 comments

  1. Mikki says:

    Thank you for this post. This was very informative and contained new for me to learn.
    I like Azure a lot. It feels a lot more productive development environment than AWS.

  2. Lev says:

    Thank you! There are many Web API + Azure Blob Storage tutorials out there. Most are very outdated and kind of suck. This one is great.

  3. Garima says:

    Can you please provide a link to download this useful sample. It would be really helpful for people like me, who are new to Web API & Azure.

    Thanks in advance :)

  4. Joe says:

    Thanks for the Great Article Dave. Just wondering if there is anyway to incorporate a progress bar during the upload? Thanks.

  5. Glez Kike says:

    Hello, my friend, before all thank you for the article.
    I have a issue when i try to upload in the code:
    blob.UploadFromStream(fs);
    I recieve an error: Remote server (403) Forbidden.
    I am testing in my local machine trying to upload a file from a web to web api. both apps in local enviroment.
    Do you have idea why is this happening?
    Thank you!!

  6. Rich says:

    As much as I really appreciate tutorials and content like this it’s quite frustrating to see them laid out in this manner. As a beginner trying to learn Web API attempting to decipher how to ‘glue’ all of these moving parts together is really difficult.

    Now you might argue I’m trying to run before I can walk but I think uploading files to Azure blob storage is a good, well-rounded, real world task to start my learning adventure with!

    A bit more guidance for the beginner would have been great, but nevertheless thanks for the content.

  7. Salib says:

    Can we use MultipartMemoryStreamProvider instead of MultipartFileStreamProvider. My webapi is hosted as AzureApp and i don’t want to store files locally before storing them to Azure Storage? How UploadProvider should be changed to accomplish that.

    • Salib – Using the MultipartMemoryrStreamProvider is probably possible, I just haven’t tried using it in this scenario. I’m not making any promises, but perhaps I’ll work on it and post the results.

  8. Ricardo Goncalves says:

    Thanks a lot for this article. It was exactly what i was looking for to upload images from my Phonegap application to Webapi using Azure.

    I believe this javascript function can help someone to send formdata to the webapi:

    function UploadImage()
    {
    var str = new String(“__PASTE A BASE64 IMAGE__”);
    var mimeString = “text/plain”; //dataURI.split(‘,’)[0].split(‘:’)[1].split(‘;’)[0]; “data:text/plain;base64,aGVsbG8gd29ybGQ=”;

    var ia = new Uint8Array(str.length);
    for (var i = 0; i < str.length; i++) {
    ia[i] = str.charCodeAt(i);
    }

    var blob = new Blob([ia], { type: mimeString });
    var fd = new FormData(document.forms[0]);
    fd.append("canvasImage", blob);

    $.ajax({
    url: __BASE_URI__ + "/blobs/upload",
    contentType: false,
    processData: false,
    data: fd,
    dataType: 'json',
    method: "POST",
    success: function (jsonData)
    {
    var jsonData = JSON.stringify(jsonData);
    alert(jsonData);
    },
    error: function(jqXHR, textStatus, errorMessage) {
    alert(errorMessage); // Optional
    }
    });
    }

  9. Alberto Donat says:

    Hi Dave

    Thanks for the article. I’m a beginner working with webapi and i need some help.

    At UploadBlobs method, i got two errors:

    var provider = task.Result;

    ‘Task doesn’t contains a definition for Result’

    return provider.Uploads.ToList();

    ‘List doesn’t contains a definition for ToList()’

    Any idea or suggestion ? Maybe i miss a using statement… ?

    Thanks in advance

  10. Swamy says:

    Thanks for this article,

    We have requirement upload encrypted file to Azure blob and download file with decrypted format. Maximum file size will be 2GB.

    We are facing issue in downloading decrypted file, getting out of memory. any way to implement the download decrypted file and download without any memory issue?

  11. Veera says:

    I have requirement to develop application where I need to upload large file of 30 gb. During upload is in progress need to start download on the same file and both should be sync.
    I am looking for file system which provides Api to my requirement

  12. Veron says:

    Hi Dave Donaldson,
    I enjoyed your tutorials. Thanks.
    Am actually a beginner to asp.net api and azure blob storage.

    Can you please give me a heads up on how to save the data in the database.
    At UploadBlobs method.
    // TODO: Use data in the list to store blob info in your
    // database so that you can always retrieve it later.

    And Also at the DownloadBlob(int blobId) method
    there is a method GetBlobName(blobId) which has not been implemented.
    Can I get a help.
    I would much grateful if help me out.
    Thanks in advance

    • Veron, In code exist a TODO comment:

      // TODO: You must implement this helper method. It should retrieve blob info
      // from your database, based on the blobId. The record should contain the
      // blobName, which you should return as the result of this helper method.

  13. Shane says:

    Hi Dave:

    Thank a lot for this article.

    Just wondering in your DownloadBlob implementation, you did not use Using statement. Do you have an example of handling the dispose of the stream in the web API framework?

    Cheers,

Leave a Reply

Your email address will not be published. Required fields are marked *