Scaling PHP applications on Windows Azure Part IV: Using worker roles for automated scaling

Author: Ben Lobaugh <ben@lobaugh.net>

Date: Tuesday, June 14, 2011, 1:13:23 PM

Tags: Tutorial, Article, Scaling, Worker

Table of Contents

Note:This article pertains to the CodePlex SDK initially released late 2009. The Windows Azure team has since then released a newer version of the Azure SDK for PHP on Github. Please refer to the Windows Azure PHP Developer Center for documentation on this more recent version of the SDK.

Please stay tuned and come back here regularly as we are working on refreshing the tutorials to deliver up to date and useful content for our PHP developers.

Recommended pre-read

Synopsis

This is part IV in a series of articles on scaling PHP application on Windows Azure. I will be picking up where I left off at the end of part II of this series. If you have not read parts I, II, and III I highly encourages you to do so as the referenced code has been built through each article in the series.

In this article I will show you how to roll all the code we previously created to monitor and scale our Windows Azure deployment into a worker role. This worker role will sit behind our web role constantly monitoring the performance metrics of the application running in or web role. When scaling thresholds are met the worker role will handle automatically adding and removing instances.

A Word of Caution!

Windows Azure charges you according to the number of instances and hours running. Whenever you are using any tool to automatically scale your instances up you should always set an upper limit to the number of instances that you are willing to pay for. If a malicious user triggers your scaling out code without an upper limit on the number of instances it could wind up costing many more dollars than you intended.

Some Reorganization

You have probably noticed that up until now we have been working out of the Web folder only. In this article we will begin using the Worker folder, but that requires some reorganization of our files. Moving these files also has the added benefit of added security as the certificate will now be hidden inside the Worker with no access from the outside world.

After your reorganization the file structure should look like the following:

  • Web
    • index.php
    • metrics.php
    • setup.php
    • view_all_metrics.php
  • Worker
    • monitor.php - Moved
    • setup.php - Copied
    • ticks_to_time.php - Moved
    • your_certificate.pem - Moved
    • worker.php

From this point forward we will be working inside the Worker folder and it should be assumed that all files referenced or created will be there unless noted otherwise.

Additions to setup.php

Open your setup.php file. The file should be a duplicate of what you have in the setup.php in the Web directory. At first it is tempting to assume there is a redundancy that can be easily eliminated by simply moving the setup.php up one directory and referencing it there in both the Web and Worker roles, however both roles are entirely separate and when your project is running they will not have access to the resources inside each other or in the main project folder.

As mentioned previously, we want to setup the maximum number of instances we are willing to have running. Also we are going to set the minimum number of instances. If we do not have a minimum number of instances it is possible that when the lower threshold is triggered the performance monitor could scale so low as to remove all running web roles. Not a good thing! The minimum number of instances you need to keep running at all times will be dependent upon your specific application, however it should be noted that 2 instances is the minimum to ensure SLA coverage. We will use 2 in this article.

define('MIN_WEBROLES', 2); 
define('MAX_WEBROLES', 20);

Let's also add a setting for how long the worker loop should pause between scaling checks in seconds. Set this at a reasonable. Windows Azure charges for the number of connections and also has a limit on how many connections can be handled per second.

define('LOOP_PAUSE', 10); // in seconds

The Worker

Because a worker does not provide public access to its files we will need to tell Windows Azure to run a file for us when the worker is initialized. This file is worker.php and we will be putting the logic for our worker in it.

Setup

As with every other file we have created we first must include the setup.php file. Your new file should currently look like the following:

<?php
require_once('setup.php');
require_once('ticks_to_time.php');

The Loop

The Loop is the most important aspect of our worker. After deployment Windows Azure will only call our worker.php file once so in order to ensure that we are able to continuously run the performance monitoring code we must create an infinite loop. This loop needs to pause so we do not overwhelm our application and also because the performance counters are not continuous it does not make much sense to constantly poll the counters.

The loop itself is quite simple, what is put into it may be quite complex:

while(1) {
    // performance monitoring code goes here
    sleep(LOOP_PAUSE); // this should be the last line in the loop
}

The functions

The functions we created in monitor.php in part III of this series I am going to copy into the worker.php file. The following function should be copied:

  • get_deployment_id
  • get_metrics
  • get_num_roles
  • scale_check

The Logic

The logic we created in the monitor.php file can be copied and used without change. Copy the code into The Loop and you should wind up with the following.

while(1) {
    $metrics = (get_metrics(get_deployment_id()));
    $num_roles = get_num_roles('WebRole');

    // check to see if we need to scale and scale accordingly
    switch (scale_check($metrics)) {
        case 1:
            // Add an instance
            if($num_roles < MAX_WEBROLES)
                $client->setInstanceCountBySlot(AZURE_WEBROLE_END, 'production', 'WebRole', $num_roles + 1);
            break;
        case 0:
            // Do no add/remove an instances
            break;
        case -1:
            // Remove an instance
            if($num_roles > MIN_WEBROLES)
                $client->setInstanceCountBySlot(AZURE_WEBROLE_END, 'production', 'WebRole', $num_roles - 1);
        break;
    }
    sleep(LOOP_PAUSE); // This should be the last line in the loop
}

It should be noted here that I have removed all the echo statements. You will be unable to view the output of any echo calls since the worker role does not have any sort of interface with which you can view your application.

Creating the package

Note: As of the writing of this tutorial I am using the May 23 2011 release of the Windows Azure Command Line Tools for PHP. A significantly revised version of the tools is set to be released in the near future. This article will be updated to reflect those changes when the new tool is available.

Because we are not creating a package for just a web role the build process is a little more complex. There are two more parameters you must provide package.php with during the build process:

  • --worker-role-startup-script - This is the file Windows Azure will call when the worker role has deployed
  • --worker-role - This is the folder containing the files to be packaged for the worker role.

During the development of this article I used the following command which you can adapt to your specific scenario:

php.exe package.php --source="C:\Projects\scaling_demo\Web" --project="Scaling Demo" --runDevFabric -f --target="C:\Projects\scaling_demo\deploy" --worker-role-startup-script="worker.php" --worker-role="C:\Projects\scaling_demo\Worker"

After you run this command and upload the package files to your Windows Azure account you should see two roles running, WebRole and WorkerRole. With the Portal open to your deployment for reference, generate many connections to your web role. When the connection threshold is reached you should see your deployment going into "Updating" status in the Portal and in a few minutes a new instance of your web role will be running and helping to process the increased load. Continue to place a load of connections on your web role and you should see as many instances spin up as needed until the upper limit of running instances is reached.

Because I always use the same structure for my Windows Azure PHP projects I have developed a nice build script that takes care of a lot of the overhead for me. If you build packages often it might be worth taking the time to create a build script instead of remembering and typing all the package parameters. My build script is available for reference in a blog post here.

Wrap Up

Now that you successfully made it through this series you should be equipped with the tools you need to begin writing PHP application that scale on Windows Azure! As you are building your next million dollar application be sure to browse through the Windows Azure PHP website and consult with the many great articles on scalable architecture available on MSDN.

Additional Reading

Storage

Designing scalable applications

Useful Blogs

Tools

 
blog comments powered by Disqus

Related Content

No related content was found