Accessing Azure Blob Storage from your Apps using S3 API

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

As of PCF 1.10, it became possible to use Azure’s blobstore solution to store files for PCF ERT’s Cloud Controller.

Microsoft Azure partnered with Bounce Storage, the maintainer of S3Proxy, and Apache jclouds, to enable Azure Storage support for S3Proxy as well as provide different deployment options to allow S3Proxy to run as a containerized application anywhere, including Cloud Foundry.

What this implies is that the Cloud Foundry blobs migrator tool will work for Azure ONLY IF you have set up an S3 proxy in front of the Azure blobstore.

Environment

Resolution

There are several options available for this proxy. There are instructions provided in the Microsoft blog on how to bring up the Docker image on Cloud Foundry as an app, but if you do not give your apps direct access to the ERT system component VMs or to the Azure blobstore, you may prefer to set up the proxy directly on the NFS VM, but that means you will have to install and run Docker there.

Additional Information

Microsoft Blog- Access Azure Blob Storage from Your Apps using S3 API: A proxy solution enabling customers to reuse existing code with S3 SDKs against Azure Blob Storage.

Deploy S3Proxydocker to Cloud Foundry Diego: Cloud Foundry with Diego can pull the S3Proxy Docker image from a Docker Registry then run and scale it as containers. 

Cloud Foundry Blobstore Migrator tool: The tool is open-source but we share releases if it with customers via PivNet. The README explains how to use it.