Scaling PHP Applications on Windows Azure Part III: Performance Monitor

Author: Ben Lobaugh <ben@lobaugh.net>

Date: Tuesday, June 14, 2011, 12:50:38 PM

Tags: Tutorial, Article, Scaling

Table of Contents

Note:This article pertains to the CodePlex SDK initially released late 2009. The Windows Azure team has since then released a newer version of the Azure SDK for PHP on Github. Please refer to the Windows Azure PHP Developer Center for documentation on this more recent version of the SDK.

Please stay tuned and come back here regularly as we are working on refreshing the tutorials to deliver up to date and useful content for our PHP developers.

Recommended pre-read

Synopsis

This is part III in a series of articles on scaling PHP applications on Windows Azure. I will be picking up from where I left off at the end of part II of this series. If you have not read parts I and II already I highly encourage you to do so as the referenced code has been built through each article in the series.

In this article I will show you how to build the performance monitor portion of this project. The performance monitor will be a PHP script that monitors the resource usage of a Windows Azure deployment and scales in or out according to need.

New values for setup.php

There is a new value we need to place in our setup.php file for this portion of the tutorial to work properly.

Open your setup.php and add the following:

define('AZURE_WEBROLE_END', '<YOUR WEB ROLE ENDPOINT>'); // Web Endpoint

This defines the endpoint for your web role. It is the name you chose when you created your hosting account. Save that file and let's get cracking on the monitor code.

Initial monitor setup

Before we dig in too deep lets create a new file that will run our performance monitor. Create a new file named monitor.php with the following content:

<?php

require_once('setup.php');

That is all for now. As you follow along this tutorial place all the new code we create in this file. Later we will look at rolling the performance monitoring code into a separate worker role that will continually monitor our performance metrics and automatically scale accordingly.

Retrieving Performance Metrics over time

We want to retrieve our performance metrics over a period of time so we can create charts, calculate averages, and make predictions. In part I of this series you learned how to subscribe to various performance metrics. These metrics are stored in the "WADPerformanceCountersTable". Windows Azure makes it very easy to retrieve performance metrics starting from a specific time with the table's partition key.

Handling C# DateTime.Ticks in PHP

The partition key is stored as a C# DateTime.Ticks timestamp based on when the metric was entered into the table. C# DateTime.Ticks have a resolution of nano seconds, however PHP's datetime functionality only handles down to microsecond. The conversion is not difficult; however we are going to use a pre-built library that already contains the functions we need.

Download C# DateTime.Ticks to Unix timestamp convertor from github

This library provides us with the following functions

  • ticks_to_time()
  • time_to_ticks()
  • str_to_ticks()

For an in-depth explanation of this issue see the article Converting DateTime.Ticks to a Unix timestamp and back in PHP

A note on Performance Metrics gathering

When we setup our performance metrics subscription in part I of this series we specified an interval of one minute between times when Windows Azure transfers our performance metrics to table storage. There are a couple of items to take note of here:

  • Performance metrics may be gathered more often than one minute and queued to be inserted into the WADPerformanceCountersTable
  • Performance metrics are scheduled to be written each minute but may not be in the table every minute, on the minute.

The first point is important to know, however the second point is critical. When your metrics are scheduled to be inserted into the table Windows Azure will queue a write operation. This write operation gathers the metrics you are subscribed to and that are not yet in the table and inserts them into the table. This insertion operation may actually cause the metrics to be written after the minute has passed. It is therefore not always safe to check back only to a range of one minute. The performance monitor may encounter errors or behave in an unexpected manner if the metrics are not available when requested. During my testing I found that three minutes will usually give back decent results, however an interval of five minutes has never failed to give back results. The interval used by your application will vary based on known and anticipated loads, and how your application responds to increased load.

Get the metrics

Now let's finally get some of the metrics from storage so we can finish building our performance monitor! The following function will retrieve the metrics for us over a given period of time.

function get_metrics($deployment_id, $ago = "-15 minutes") {
    global $table;

    // get DateTime.Ticks in past
    $ago = str_to_ticks($ago);

    // build query
    $filter = "DeploymentId eq '$deployment_id' and Role eq 'WebRole' and PartitionKey gt '0$ago'";
    $filter='';

    // run query
    $metrics = $table->retrieveEntities('WADPerformanceCountersTable', $filter);

    $arr = array();
    foreach ($metrics AS $m) {
        // Global totals
        $arr['totals'][$m->countername]['count'] = (!isset($arr['totals'][$m->countername]['count'])) ? 1 : $arr['totals'][$m->countername]['count'] + 1;
        $arr['totals'][$m->countername]['total'] = (!isset($arr['totals'][$m->countername]['total'])) ? $m->countervalue : $arr['totals'][$m->countername]['total'] + $m->countervalue;
        $arr['totals'][$m->countername]['average'] = (!isset($arr['totals'][$m->countername]['average'])) ? $m->countervalue : $arr['totals'][$m->countername]['total'] / $arr['totals'][$m->countername]['count'];

        // Totals by instance
        $arr[$m->roleinstance][$m->countername]['count'] = (!isset($arr[$m->roleinstance][$m->countername]['count'])) ? 1 : $arr[$m->roleinstance][$m->countername]['count'] + 1;
        $arr[$m->roleinstance][$m->countername]['total'] = (!isset($arr[$m->roleinstance][$m->countername]['total'])) ? $m->countervalue : $arr[$m->roleinstance][$m->countername]['total'] + $m->countervalue;
        $arr[$m->roleinstance][$m->countername]['average'] = (!isset($arr[$m->roleinstance][$m->countername]['average'])) ? $m->countervalue : ($arr[$m->roleinstance][$m->countername]['total'] / $arr[$m->roleinstance][$m->countername]['count']);
    }
    return $arr;
}

The array returned by this function contains an array of totals for all instances as well as totals for each individual instance. For now we are going to only look at the total average number of TCPv4 connections, which can be accessed via:

$metrics['totals']['\TCPv4\Connections Established']['average']

Scale Check

Please take note that in this article I will be showing you how to scale based on performance metrics from the number of TCPv4 connections established. This is a very simplistic way of determining when to scale and probably only one facet of what you will want to implement in your production applications.

For more in-depth information on the additional scaling features in Windows Azure see the additional reading links near the end of this article.

To make the code a little cleaner let's create a scale_check function. This function will return

  • 0 if no scaling is needed
  • 1 if more instances are needed
  • -1 if less instances are needed

When we call this function we will do it from the context of a switch statement that will handle the return values for us.

function scale_check($metrics) {
    $ret = 0;

    if(120 > $metrics['totals']['\TCPv4\Connections Established']['average']) {
        $ret = 1;
    } else if(20 > $metrics['totals']['\TCPv4\Connections Established']['average']) {
        $ret = -1;
    }
    return $ret;
}

Note: The numbers (120 and 20) here are representative only. You will need to test your application to determine how many connections it can handle before needing to scale.

The switch statement is going to do the actual work of scaling for us. It will utilize the values passed back from scale_check() to determine our scaling needs.

switch (scale_check($metrics)) {
    case 1:
        // Add an instance
        echo "Scaling Out";
        $client->setInstanceCountBySlot(AZURE_WEBROLE_END, 'production', 'WebRole', get_num_roles('WebRole') + 1);
        break;
    case 0:
        // Do no add/remove an instances
        echo "Perfomance within acceptable range";
        break;
    case -1:
        // Remove an instance
        echo "Scaling in";
        $client->setInstanceCountBySlot(AZURE_WEBROLE_END, 'production', 'WebRole', get_num_roles('WebRole') - 1);
        break;
}

Additional Code

Here are two additional functions that I like to use because it allows my application to be a bit more dynamic, instead of hard coding in values or hoping that a process does not die and cause a loss of information. Please be aware that the functions make calls to you Windows Azure account and therefore will increase the response time accordingly. In general speed should not be an issue.

The first function will pull the currently running deployment information and return the deployment id for you.

function get_deployment_id() {
    global $client;

    $s = $client->getDeploymentBySlot(AZURE_WEBROLE_END, 'production');
    return $s->PrivateId;
}

When adding or removing instances you must specify the total number of instances that you want running. Instead of keeping a variable in memory that could easily loose it's value, it is possible to determine the number of currently running roles via the deployment information that can be retrieved. The following function does just that based on the name of the role you wish to calculate.

function get_num_roles($roleName) {
    global $client;

    $ret = 0;
    $is_role = false;

    $s = $client->getDeploymentBySlot(AZURE_WEBROLE_END, 'production');
    
    $xml = new SimpleXMLElement(mb_convert_encoding($s->configuration, "UTF-16"));

    foreach($xml->Role as $r) {
        foreach($r->attributes() as $a=>$b) {
            if($a == 'name' && $b == $roleName) $is_role = true;
        }

        foreach($r->Instances->attributes() as $a=>$b) {
            if($is_role) return $b;
        }
    }
}

Add both of those functions to your monitor.php file and your performance monitor for this article should be complete. After you have all the new code in monitor.php, rebuild your app and navigate to the monitor.php page in your browser and you should see the result of your scaling operations.

Wrap Up

In part IV of this series I will show you how to wrap everything you have learned in the first 3 parts of this series into a Windows Azure Worker Role that will sit in the background monitoring your performance and automatically scaling your web role according to the scaling triggers that you use for your application.

Additional Reading

Designing scalable applications

What's Next?

 
blog comments powered by Disqus