Scenario - Offloading Static Content to Blob Storage

Author: Maarten Balliauw

Date: Tuesday, October 26, 2010, 12:00:00 AM

Tags: Scenario, How to

Table of Contents

Note:This article pertains to the CodePlex SDK initially released late 2009. The Windows Azure team has since then released a newer version of the Azure SDK for PHP on Github. Please refer to the Windows Azure PHP Developer Center for documentation on this more recent version of the SDK.

Please stay tuned and come back here regularly as we are working on refreshing the tutorials to deliver up to date and useful content for our PHP developers.

Recommended pre-read

Code Sample

Scenario description

For a personal online photo album, a website has been developed showing all pictures available.

clip_image001

Figure 1: My photo album application

Due to increasing visits to the website, the decision has been made to host static content (like CSS, and images) on Windows Azure storage, using the content-delivery network. (A content delivery network or content distribution network (CDN) is a system of computers containing copies of data, placed at various points in a network so as to maximize bandwidth for access to the data from clients throughout the network. A client accesses a copy of the data near to the client, as opposed to all clients accessing the same central server, so as to avoid a bottleneck near that server.)

Dynamic pages (PHP scripts) will remain on the current web server, which can be any web server (IIS, Apache, etc.) on any platform (Windows, Linux, etc.).

clip_image002

Figure 2: Scenario - schematic

The above scenario has different advantages:

  • Owned server load & data traffic is reduced because static content no longer has to be served from the own server but instead directly from Windows Azure storage.
  • Owned server can be any OS and any server: Linux + Apache, Windows + IIS, Linux + LigHTTPD, …
  • Web pages and data are on a different (sub)domain name, increasing modern browsers download speed.
  • Windows Azure storage can be expanded with a CDN to increase download speed for static content.

Offloading static content to blob storage

In order to offload static content to blob storage, the photo album application has to be changed:

  • The main page showing pictures should link to the full blob storage URL instead of the local, on-premise picture.
  • An extra page should be developed for automating content copying from on-premise to Windows Azure storage. Note that this step could be omitted: static content can be uploaded to blob storage from various Windows Azure storage clients directly.

Running the application

Viewing the photos

When navigating to the photo application, the main screen showing pictures is presented. This page is served by a PHP script and links images to blob storage:

clip_image003

Figure 3: My photo album application

Uploading static content to blob storage

An extra screen has been developed for uploading static content to blob storage. This screen uploads contents of the styles and images folder to Windows Azure blob storage.

clip_image004

Figure 4: Uploading static content to Windows Azure blob storage

Installing the sample scenario

The sample scenario demo application consists of the following files:

  • scenario_offloading.php
  • scenario_offloading_transfer.php
  • images folder containing photos

These can be deployed and used on any web server (Apache, IIS, …) and platform (Linux, Windows, …).

In order to run the samples out-of-the-box, ensure that development storage is started on your Windows machine by clicking Start, All programs, Windows azure SDK, Development Storage. Refer to the Tutorial - Getting started for installing Windows Azure SDK for PHP and related components.

Running the sample in the Development Storage or in production with a Windows Azure Storage account

Throughout the scenario, development storage will be used for building examples, unless specified differently. If you decided to work with a production Windows Azure storage account, use the production parameters for all constructors of the Microsoft_WindowsAzure_Storage_* objects. If the tutorial connects to the development blob storage service like this:

1: $storageClient = new Microsoft_WindowsAzure_Storage_Blob();
2:// CHANGE THE CODE TO:
3:
4: $storageClient = new Microsoft_WindowsAzure_Storage_Blob(
5: 'blob.core.windows.net',
6: '<;your azure account name, e.g. mystorageplayround',
7: '<;your azure account key>');

Coding the application

Viewing the photos

The implementation for the main screen is very straightforward: it is a PHP script outputting HTML with links to on-premise or cloud hosted images. The following is an extract of the HTML code displaying the main photo application screen:

1: <;div class="main">
2:     <;h2>Pictures</h2>
3:     <;p>Below is an overview of all my pictures per category.</p>
4:     <;h3>General</h3>
5:     <;p style="text-align: center;"><img src="images/general/1.jpg" alt="Image" /></p>
6:     <;p style="text-align: center;"><img src="images/general/2.jpg" alt="Image" /></p>
7:     <;p style="text-align: center;"><img src="images/general/3.jpg" alt="Image" /></p>
8:     <;p style="text-align: center;"><img src="images/general/4.jpg" alt="Image" /></p>
9:     <;p style="text-align: center;"><img src="images/general/5.jpg" alt="Image" /></p>
10:
11:     <;h3>Animals</h3>
12:     <;p style="text-align: center;"><img src="images/animals/1.jpg" alt="Image" /></p>
13:     <;p style="text-align: center;"><img src="images/animals/2.jpg" alt="Image" /></p>
14:     <;p style="text-align: center;"><img src="images/animals/3.jpg" alt="Image" /></p>
15: </div>

Note that image URLs (e.g., images/animals/1.jpg) are linking to the on-premise images. In order to link these to Windows Azure storage, links will have to be updated:

  • Hosted on development storage: http://127.0.0.1:10000/devstoreaccount1/images/animals/1.jpg
  • Hosted on Windows Azure blob storage: e.g., http://mystorageplayground.blob.core.windows.net/images/animals/1.jpg
  • Hosted on Windows Azure CDN: e.g., http://az13167.vo.msecnd.net/images/animals/1.jpg
  • Hosted with a custom domain name: e.g., http://static.example.com/images/animals/1.jpg

Links could be updated manually in source code, however, when hosting on IIS, the URL Rewriting component can be used to do this automatically. Apache mod_rewrite will not be able to do this as it only rewrites incoming URLs, unlike IIS, which modifies HTML output to reflect rewritten URLs.

Uploading static content to blob storage

The actual upload of static content to blob storage is done by a second screen in the photo album application. This is not required: uploading files to blob storage can also be done using various Windows Azure blob storage clients available. By creating this functionality as a web page however, static content can easily be uploaded just from a browser and, optionally, even be automated.

The sample code will be developed as a one-page script. As a side note, no input validation or sanity checks will be done, for the sake of simplicity.

At the top of this one-page script, the PHP code will be located. First of all, start with including and initializing the Microsoft_WindowsAzure_Storage_Blob class:

1: /** Microsoft_WindowsAzure_Storage_Blob */
2: require_once 'Microsoft/WindowsAzure/Storage/Blob.php';
3:
4:// Connect to blob
5: $storageClient = new Microsoft_WindowsAzure_Storage_Blob();
6:// Note that, if you are working with production storage instead of development storage, the storage client will have to be initialized as follows:
7: $storageClient = new Microsoft_WindowsAzure_Storage_Blob(
8: 'blob.core.windows.net',
9: '<;your azure account name, e.g. mystorageplayround',
10: '<;your azure account key>');
11:
12: // Next, two variables are defined:
13:
14: // Settings
15: $foldersToUpload = array('images', 'styles');
16:
17:// Transfer log
18: $transferLog = array();

The $foldersToUpload variable contains the list of directories containing static content. The $transferLog variable contains a log of all the performed uploads.

Once a user requests the current screen, with a request variable of ?action=transfer, the actual upload of static content to blob storage will be performed. For each folder defined in the $foldersToUpload variable, the contents will be transferred. The following is the structure for that:

1:if (isset($_GET['action']) && $_GET['action'] == 'transfer') {
2:// Upload folders
3:foreach ($foldersToUpload as $folderToUpload) {
4:// ...
5:     }
6: }

Within this structure, a blob container for each static folder is created on Windows Azure. Note that you can structure this differently if your application desires a different structure.

1:// Ensure container exists
2:if (!$storageClient->containerExists($folderToUpload)) {
3:     $storageClient->createContainer($folderToUpload);
4: }

By default, a blob container is read/write for the Windows Account owner and access will be denied for any other request made to blob storage. Since the photo application will be serving static content directly from blob storage, the container's ACL has to be set to public. Note that this is read-only public access. Public access can be granted on container level (allowing directory listings) or on blob level (disallowing directory listings).

1:// Ensure blobs can be accessed by anyone
2: $storageClient->setContainerAcl($folderToUpload,
3:                                 Microsoft_WindowsAzure_Storage_Blob::ACL_PUBLIC_CONTAINER);

Next, a list of all the files to upload is generated by calling the recursePath function.

1:// Generate list of files to upload
2: $filesToUpload = recursePath($folderToUpload);
3:
4:// The recursePath function is a recursive function getting the full path to all files within the folder to be uploaded. Here's the code for that function:
5:
6:function recursePath($path){
7:     $files = array_diff(scandir($path), array('.', '..'));
8:     $result = array();
9:
10:foreach($files as $file) {
11:if (is_dir($path . DIRECTORY_SEPARATOR . $file)) {
12:             $result = array_merge($result, recursePath($path . DIRECTORY_SEPARATOR . $file));
13:         } else {
14:             $result[] = $path . DIRECTORY_SEPARATOR . $file;
15:         }
16:     }
17:
18:return $result;
19: }

The array that is generated by the recursePath function looks similar to this:

1:array(2) {
2:     [0]=>string(20) "images\animals\1.jpg"
3:     [1]=>string(20) "images\animals\2.jpg"
4: }

After the list of files is generated, the files will have to be uploaded to blob storage. Here's the code for that:

1:// Upload files
2:foreach ($filesToUpload as $fileToUpload) {
3:     $targetBlobName = str_replace($folderToUpload . DIRECTORY_SEPARATOR, "", $fileToUpload);
4:
5:     $targetBlobName = str_replace(DIRECTORY_SEPARATOR, "/", $targetBlobName);
6:
7:     $storageClient->putBlob($folderToUpload,$targetBlobName,
8:                             $fileToUpload, array(), null, array('x-ms-blob-cache-control' => 3600));
9:
10: }

Let's break down the parameters for the putBlob method:

Parameter

Given value

Description

$containerName

$folderToUpload

A container is created, per folder specified in the settings for this screen.

$blobName

$targetBlobName

A stripped-down version of the local file name: the root path is removed and backslashes are replaced with forward slashes.

$localFileName

$fileToUpload

The local file that should be uploaded to blob storage.

$metadata

array()

An empty array of metadata. This can be a set of key-value pairs describing the blob.

$leaseId

null

An optional lock on the blob, just like file locks on the local operating system.

$additionalHeaders

array('x-ms-blob-cache-control' => 3600)

The amount of time the CDN should cache the file on edge servers. After 1 hour (3600 seconds) the data on edge locations is refreshed from blob storage.

;

This is basically all it takes to upload a set of local files to blob storage and make them publicly available. If you navigate to, for example, http://127.0.0.1:10000/devstoreaccount1/images/general/1.jpg, that image will be served directly from blob storage. Note the structure of the URL: http://127.0.0.1:10000/devstoreaccount1/<container>/ <blob> where container is images and blob is images/general/1.jpg.

Optional: Using the Windows Azure CDN

When an application is used from many locations around the world, using a CDN easily enables better performance and user experience for users who are farther from the source of the content stored in the Windows Azure Blob service. In addition, Windows Azure CDN provides worldwide high-bandwidth access to serve content for popular events.

Windows Azure CDN has locations globally (United States, Europe, Asia, Australia and South America) and continues to expand. It caches your Windows Azure blobs at strategically placed locations to provide maximum bandwidth for delivering your content to users. You may enable CDN delivery for any storage account, via the Windows Azure Developer Portal.

Note that the CDN provides edge delivery only to blobs that are in public blob containers, which are available for anonymous access.

The following steps are required to enable CDN access to a blob storage account:

  1. Navigate to the Windows Azure Developer Portal, click on your Project Name, then on your storage account.
  2. Click on Enable CDN for your storage account.
  3. The portal provides a CDN domain name in the form of http://<guid>.vo.msecnd.net/

clip_image005

Figure 5: Content delivery network enabled

The configuration created for this endpoint is not immediately available; it can take up to 60 minutes for the registration to propagate through the CDN network worldwide. Users who try immediately to use the CDN domain name will get a 400 error until the configuration is updated worldwide.

Optional: Mapping a custom domain name to blob storage and the CDN

Instead of using the default blob storage URL which is long and ugly, you may also register one custom domain name for Windows Azure blob storage or Windows Azure CDN. For example, if you wanted to access blob or CDN content through the domain "static.example.com", you may register that custom domain name for the blob storage or CDN endpoint in the portal.

Follow these steps to register and use a custom storage domain name for your blob or CDN content:

  1. Navigate to the Windows Azure Developer Portal, click on your Project Name, then on your storage account.
  2. Click on Manage for your blob or CDN endpoint.

clip_image006

Figure 6: Manage custom endpoint

3. Enter your custom domain name.

clip_image007

Figure 7: Domain registration

  1. In order to use your custom domain name, you should create a CNAME record for it with your hoster. For example, create a record static.example.com CNAME mystorageplayground.blob.core.windows.net. Or in case of a CDN endpoint, create a record static.example.com CNAME az13167.vo.msecnd.net
  2. Your domain ownership should be verified. Create an additional CNAME record that links to verify.azure.com. This name for this CNAME record is shown on the Windows Azure portal and looks similar to 31cf2967-75f4-4650-9af9-224f5c3eeeda.static.example.com.

Optional: Enabling IIS outbound URL Rewriting

The main screen of the application is built up using a PHP script outputting HTML with links to on-premise or cloud hosted images. The following is an extract of the HTML code displaying the main photo application screen:

1: <div class="main">
2:     <h2>Pictures</h2>
3:     <p>Below is an overview of all my pictures per category.</p>
4:
5:     <h3>General</h3>
6:     <p style="text-align: center;">
7:         <img src="images/general/1.jpg" />
8:     </p>
9:     <p style="text-align: center;">
10:         <img src="images/general/2.jpg" />
11:     </p>
12: </div>

Note that image URLs (e.g. images/animals/1.jpg) are linking to the on-premise images. In order to link these to Windows Azure storage, links will have to be updated to development storage, Windows Azure blob storage, a Windows Azure CDN endpoint or a custom domain name.

Links can be updated manually in source code, however, when hosting on IIS the URL Rewriting component can be used to do this automatically. Apache mod_rewrite will not be able to do this as it only rewrites incoming URLs unlike IIS which modifies HTML output to reflect rewritten URLs.

Installing IIS URL Rewriting component

By using rule templates, rewrite maps, .NET providers, and other functionality integrated into IIS Manager, Web administrators can easily set up rules to define URL rewriting behavior based on HTTP headers, HTTP response or request headers, IIS server variables, and even complex programmatic rules. In addition, Web administrators can perform redirects, send custom responses, or stop HTTP requests based on the logic expressed in the rewrite rules.

The IIS Url Rewriting component can be downloaded and installed from http://www.iis.net/download/urlrewrite or using the Web Platform Installer found at http://microsoft.com/web.

Creating outbound URL Rewriting rules

After installing the IIS URL Rewriting component, a web.config file can be created (or modified) in the root of the website. For the photo application, the web.config file looks like the following. Note that this can be different for your application!

1:<?xmlversion="1.0"encoding="UTF-8"?>
2:<configuration>
3:<system.webServer>
4:<defaultDocument>
5:<files>
6:<addvalue="index.php"/>
7:</files>
8:</defaultDocument>
9:<handlers>
10:<addname="PHP 5.3.2 via FastCGI"path="*.php"verb="*"modules="FastCgiModule"scriptProcessor="C:\PHP\5.3.2-iis\php-cgi.exe"resourceType="Unspecified"/>
11:</handlers>
12:</system.webServer>
13:</configuration>

Rewrite rules for the photo application go in this configuration file and will be used by IIS to perform URL rewriting. IIS URL Rewriting supports both inbound and outbound URL rewriting. The first converts incoming URL requests and maps them to a different URL internally while the latter inspects the HTML response that is sent to the client and rewrites URLs found in tags like <img src=""> or <a href=""> or others.

The following outbound rules will be added to the photo application, mapping static content URLs to development storage:

· /styles/site.css should be rewritten to http://127.0.0.1:10000/devstoreaccount1/styles/site.css

· /images/1.jpg should be rewritten to http://127.0.0.1:10000/devstoreaccount1/images/1.jpg

These rewrite rules can be added in the configuration file, somewhere within <system.webServer> and </system.webServer>:

1:<rewrite>
2:<outboundRules>
3:<rulename="Rewrite static content URL 'styles' to blob storage">
4:<matchfilterByTags="A, Img, Link, Script"pattern="styles/(.*)"/>
5:<actiontype="Rewrite"value="http://127.0.0.1:10000/devstoreaccount1/{R:0}"/>
6:</rule>
7:<rulename="Rewrite static content URL 'images' to blob storage">
8:<matchfilterByTags="A, Img, Link, Script"pattern="images/(.*)"/>
9:<actiontype="Rewrite"value="http://127.0.0.1:10000/devstoreaccount1/{R:0}"/>
10:</rule>
11:</outboundRules>
12:</rewrite>

HTML output for the main screen will now automatically be rewritten to the following on every request:

1:<divclass="main">
2:<h2>Pictures</h2>
3:<p>Below is an overview of all my pictures per category.</p>
4:
5:<h3>General</h3>
6:<pstyle="text-align: center;">
7:<imgsrc=" http://127.0.0.1:10000/devstoreaccount1/images/general/1.jpg "/>
8:</p>
9:
10:<pstyle="text-align: center;">
11:<imgsrc=" http://127.0.0.1:10000/devstoreaccount1/images/general/2.jpg"/>
12:</p>
13:</div>
 
blog comments powered by Disqus

Related Content

No related content was found