Scaling PHP Applications on Windows Azure Part I: Performance Metrics

Author: Ben Lobaugh <ben@lobaugh.net>

Date: Tuesday, May 24, 2011, 11:07:32 AM

Tags: Tutorial, Scaling, Performance Metrics, Service Management API

Table of Contents

Note:This article pertains to the CodePlex SDK initially released late 2009. The Windows Azure team has since then released a newer version of the Azure SDK for PHP on Github. Please refer to the Windows Azure PHP Developer Center for documentation on this more recent version of the SDK.

Please stay tuned and come back here regularly as we are working on refreshing the tutorials to deliver up to date and useful content for our PHP developers.

Recommended pre-read

Synopsis

One of the key features of Windows Azure is its ability to dynamically scale out more role instances to accommodate increased load upon your web applications. Through the course of this three part series I will guide you through connecting to the Windows Azure Service Management API, gathering important site metrics, managing role instance counts programmatically, and finally in part three I will show you how to put it all together into one worker role that will sit in the background monitoring the load state of your application and changing instance counts as load increases and decreases. Part four is optional and will only be applicable to very large scale websites that require more than one worker to help balance load.

The following image is a high level view of our end goal.

image

;

In part I, this tutorial, we will learn how to connect to the Service Management API and how to gather the metrics pertinent to our application. At specified intervals the Web Roles will update the Performance Metrics Storage Table with our metrics.

Get the Windows Azure SDK for PHP

This tutorial relies heavily on the Windows Azure SDK for PHP. If you do not already have the SDK go out to the CodePlex project page and download a copy of the latest release.

http://phpazure.codeplex.com

Setup instructions can be found here: /articles/setup-the-windows-azure-sdk-for-php ;

Note: Versions of the Windows Azure SDK for PHP prior to the version 3 release DO NOT contain the functionality required to interact with the Windows Azure Service Management API. If you are using an older version please update your SDK.

Setup the project structure

In this project we are going to have a web role and a worker role. It is possible to keep both of these roles inside of one document root, however that leads to unnecessary project bloat (worker contains web files and web contains worker files) so we will be separating each role into its own folder, cleverly called 'Web' and 'Worker'. This leads to some code duplication in our example, however it also highlights the ability to run roles completely independent of each other.

Here is the project structure
\ScalingDemo\
\ScalingDemo\Web\
\ScalingDemo\Worker
\ScalingDemo\Worker\worker.php

The file I did not mention is worker.php. This file is the entry point of our worker role and will be run on role startup.

Setup the ServiceConfiguration.cscfg file

In order for your role to connect to the storage service you will need to provide it with your connection details. In your ServiceConfiguration.cscfg file there will be a <ConfigurationSettings> tag. Inside this tag you will need to following line, of course submitting the **** for your connection details:

<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="DefaultEndpointsProtocol=https;AccountName=*****;AccountKey=*****" />

Viewing your Windows Azure Storage data with the Azure Storage Explorer

We will be referencing the Windows Azure Storage quite often and it is awkward to simply toss data around without knowing the results. Neudesic has provided a graphical tool, called Azure Storage Explorer, which allows you to connect to, view, and manipulate data contained within your Windows Azure Storage account. I will be using this tool throughout the series to visually see what is happening in the storage account. If you do not currently have a method of browsing your storage account you can download the Azure Storage Explorer from the Codeplex project page

image

Connect to your Windows Azure Storage Blob

Windows Azure stores configuration information about your roles inside of a blob container called 'wad-control-container'. After you have run your project you will find a blob inside this container corresponding to the id of the running roles.

Because the configuration is kept in a blob inside your own storage account it is easy to make updates to the configuration on the fly. Windows Azure picks up the changes and adjusts your roles accordingly.

Setup some default configuration options and gather required objects

Typically when identical settings could be needed in multiple places or are subject to change it is easiest to create constants or variables in one location to ease the process of updating the settings at a later time. Let's create a file named setup.php that will contain all our settings and setup code. For now we will only be working with the web role, so create all your files in the Web folder.

Tip: You must pass a valid blob storage account to the diagnostics manager. Do not forget to connect to the blob storage first!

To access the Service Management API we need two objects; first a blob connection, and second a connection to the manager to which is passed the blob connection. Open the setup.php file you created in the last section and add the following.

define('PRODUCTION_SITE', false); // Controls connections to cloud or local storage 
define('AZURE_STORAGE_KEY', '<your_storage_key>'); 
define('AZURE_SERVICE', '<your_domain_extension>'); 
define('ROLE_ID', $_SERVER['RoleDeploymentID'] . '/' . $_SERVER['RoleName'] . '/' . $_SERVER['RoleInstanceID']); 
define('PERF_IN_SEC', 30); // How many seconds between times dumping performance metrics to table storage

/** Microsoft_WindowsAzure_Storage_Blob */ 
require_once 'Microsoft/WindowsAzure/Storage/Blob.php';

/** Microsoft_WindowsAzure_Diagnostics_Manager **/ 
require_once 'Microsoft/WindowsAzure/Diagnostics/Manager.php'; 
/** Microsoft_WindowsAzure_Storage_Table */ 
require_once 'Microsoft/WindowsAzure/Storage/Table.php';

if(PRODUCTION_SITE) { 
  $blob = new Microsoft_WindowsAzure_Storage_Blob( 
    'blob.core.windows.net', 
    AZURE_SERVICE, 
    AZURE_STORAGE_KEY 
  ); 
  $table = new Microsoft_WindowsAzure_Storage_Table( 
    'table.core.windows.net', 
    AZURE_SERVICE, 
    AZURE_STORAGE_KEY 
  ); 
} else { 
// Connect to local Storage Emulator 
    $blob = new Microsoft_WindowsAzure_Storage_Blob(); 
    $table = new Microsoft_WindowsAzure_Storage_Table(); 
}

$manager = new Microsoft_WindowsAzure_Diagnostics_Manager($blob);

Save the file and we should be done with it for now.

Subscribe to the metrics needed by your application

By default your Windows Azure role does not store information about such things as processor and memory utilization; however it is trivial to enable such functionality.

Available Subscriptions

There are several metrics to which your application may subscribe, however for this simple tutorial we are only going to look at processor utilization and number of connections.

To see a list of available metrics to which you can read this MSDN article.

Note: Be sure to include the backslashes (\) in the metric strings or you will be unable to gather the metric you intended.

Note: Remember, you are paying for the amount of resources used in your Windows Azure Storage account. Metrics are stored in a table and the more metrics you subscribe to the quicker the size of that table will expand. Try to limit yourself to only those metrics that will be useful to your application in knowing when to scale.

\Processor(_Total)\% Processor Time

Records the amount of processor utilization at the time the diagnostics manager transfers the metric to the storage table.

\TCPv4\Connections Established

Records number of TCPv4 connections established at the time the diagnostics manager transfers the metric to the storage table.

The Code

Here is the code needed to subscribe to the performance metrics counters, save it in the metrics.php file. Be sure you get the current configuration before you try to subscribe to the new counters. If you do not you will overwrite the configuration settings with a bunch of blanks and lose several settings required by your roles in order to properly run.

// Bring in global include file
require_once('setup.php');

// Performance counters to subscribe to
$counters = array(
    '\Processor(_Total)\% Processor Time',
    '\TCPv4\Connections Established',
);

// Retrieve the current configuration information for the running role
$configuration = $manager->getConfigurationForRoleInstance(ROLE_ID);

// Add each subscription counter to the configuration
foreach($counters as $c) {
    $configuration->DataSources->PerformanceCounters->addSubscription($c, PERF_IN_SEC);
}

// These settings are required by the diagnostics manager to know when to transfer the metrics to the storage table
$configuration->DataSources->OverallQuotaInMB = 10;
$configuration->DataSources->PerformanceCounters->BufferQuotaInMB = 10;
$configuration->DataSources->PerformanceCounters->ScheduledTransferPeriodInMinutes = 1;

// Update the configuration for the current running role
$manager->setConfigurationForRoleInstance(ROLE_ID,$configuration);

Code breakdown

$counters = array(
    '\Processor(_Total)\% Processor Time',
    '\TCPv4\Connections Established',
);

I like loops, arrays, and staying DRY so I tend to place things in loops to reduce code whenever I can. It also reduces the number of possible points of failure. Each item in this array corresponds to a metric listed above and will be added to the metrics subscription list.

$configuration->DataSources->OverallQuotaInMB = 10;

This setting is required. If it is not set by the developer it will default to zero and no metrics will be stored. Set it to a value that is realistic for your application.

$configuration->DataSources->PerformanceCounters->BufferQuotaInMB = 10;

This is another quota setting, however this one relates specifically to the performance counters themselves, whereas the previous quota was for the entire diagnostics log storage.

$configuration->DataSources->PerformanceCounters->ScheduledTransferPeriodInMinutes = 1;

This setting controls how often your performance metrics are transferred to the Windows Azure storage table and is also required.

$manager->setConfigurationForRoleInstance($roleid,$configuration);

This line updates the configuration in the Windows Azure storage blob. You do not have to worry about refreshing the package or anything else. Windows Azure will notice your changes a few seconds after you update them and reload the configuration automatically.

Reading the performance metrics

Now that you are getting the metrics it would probably be nice to be able to view them right? Because the metrics are stored in the WADPerformanceCountersTable they may be accessed using standard table access methods.

Create a new file called view_all_metrics.php and insert the code below into the file.

<?php

// Bring in global include file 
require_once('setup.php'); 

// Grab all entities from the metrics table 
$metrics = $table->retrieveEntities('WADPerformanceCountersTable'); 

// Loop through metric entities and display results 
foreach($metrics AS $m) { 
    echo $m->RoleInstance . " - " . $m->CounterName . ": " . $m->CounterValue . "<br/>"; 
}

Save and run that file on your server after building your aplication with the Windows Azure Command Line tools for PHP and you will see the performance metrics that have been collected.

Note: The Windows Azure Storage service will return a maximum of 1,000 entities per call. If you want to see more than that you will need to send back the continuation token to retrieve the next 1,000 entities.

Note: It may be several minutes before you begin to see data in the WADPerformanceCountersTable table. If you set your logging interval to 1 minute it may be more than 1 minute between insertions into the table. During that minute Windows Azure collects the metrics and at the minute mark the metrics are queued to be inserted into the table, but may not insert exactly on the minute.

What's Next?

In part III of this series I will show you how to extrapolate useful bits of information from your performance metrics. Using those bits you will learn how to make your application dynamically scale, but first we need to learn about role management.

 
blog comments powered by Disqus

Related Content

No related content was found