Tag Archives: Zend-server

Enter Deployment: Zend Server 5.5

Today Zend Server 5.5 was released.  This is actually a pretty good sized deal.  The big addition to the Zend Server feature list is deployment.  Now, you might be saying “come on, Kevin, there are many PHP deployment solutions already”.  To which I would say “yes, there are.  But there are NO solutions that work out of the box“.  Ladies and gentlemen, welcome to the box.  If you have Zend Server 5.5, you have deployment.

So what I’m going to do with this post is give you a quick run-through on getting a simple application up and running.  There are a lot of features I am going to skip over (primarily deployment scripts) and show you a simple workflow so you can see how quickly and easily an application can be deployed.

Step 1: Install The Zend SDK

First of all, you need the Zend SDK (and, of course, Zend Server) and set up the tools directory to be in your path.  Actually, you don’t need the SDK since there is a GUI interface.  But I’m showing you a simple workflow that will get your application running very quickly.  We’ll get into more details in a different blog post that demonstrates the full GUI interface.

Step 2: Create an API Key

In the Zend Server UI you need to name and create a new API key so that the SDK can communicate with your instance of Zend Server.

If you click on the “Show full key” link you will get a popup of the full key so you can copy it into your terminal.  The name of the key is also important.

Step 3: Connect the SDK to Zend Server

Once you have your API key you can then associate it with your local SDK instance.  You will need the API key that you generated earlier along with the name of that key.  Provide the hostname and you’re good to go.

D:\workspace>zend add target -s 8a29cd9333ca366af986df69c32b44ed84aff6af0e741209a82cf4a444e64187
   -k deployment -h http://192.168.0.248
 
Target http://192.168.0.248 was added successfully, with id 0

The ID number is important if you are going to have multiple Zend Server clusters to talk to, such as staging and production.  Then you will need to add the -t argument with the target ID.  However, if you have only one installation then the SDK will automatically use that target.

Step 4: Create a deployment scenario for your PHP project

Once your target is set up you need to provide a deployment descriptor file for your project.  For my example project I’m just going to use a blank Hello World application with a /public application/document root.

D:\workspace>zend create project -t simple -n HelloWorld
 
Project resources were created successfully for HelloWorld under D:\workspace\HelloWorld

The -t option is the template of the application.  You have simple, quickstart and zend.  Because my application already existed I specified that it was a simple application so no additional files would be created.  If you are creating a new project from scratch you can quite easily use one of those templates.  If you have an existing application, using the simple template will give you a good starting point.

The deployment.xml file for my application looks something like this (thanks WordPress for not allowing me to post XML, even the code formatters barf on it)

package
 ---> name: HelloWorld
 ---> version
    ---> release: 1.0.0
 ---> appdir: public

Step 5: Deploy!

The next step is to deploy the application.  You can, again use the SDK to do this from the command line.

D:\workspace\HelloWorld>zend deploy application -b /
 
Application HelloWorld (id 9) is deployed to http://<default-server>/

The -b tells the SDK the base URL that we want to install the application at.  If we wanted it at /test, it would be -b /test.

If we want to remove the application we simply tell the SDK to do it.

D:\workspace\HelloWorld>zend remove application -a 9
 
Application HelloWorld (id 9) was removed from http://<default-server>/

The -a tells the SDK the application ID of the application that we want to remove.  We got the application ID from the return message when we originally deployed it.

Step 6 (optional): Rollback

Every once in a while you deploy an application that wasn’t quite as baked as you thought it was.  With the new deployment feature you can log in to your Zend Server GUI and do a one-click rollback of your application (well, two clicks once you click the “are you sure button”).

Conclusion

That is how easy it is to deploy a (very) simple application using the new deployment techniques.  This works both for Zend Server and Zend Server Cluster Manager installations, so you can deploy to a single machine or let ZSCM push your application to your entire cluster.

Later on I will go through using the Zend Server GUI to set up more complex applications requiring configuration parameters or deployment scripts and also go over the zdpack command which is the package builder that you use if you are not using command line based deployments.

Where to go from here?

It’s simple.  Go to our download site, or set up your test machine to access our yum or deb repositories.  In other words, try it out.

Zend Server 5.1 Web API

This weekend we soft launched the Zend Server 5.1 update.  So soft, in fact, that I didn’t even know it was launched.  But I did know it was coming.  Most of the improvements were either bug fixes or performance improvements.  Some of the performance improvements have actually been pretty significant, particularly under load.

But we did add a new feature that works both with Zend Server and Zend Server Cluster Manager.  This release for Cluster Manager is actually really nice.  I have a video demo that I recorded last week on scaling web applications that I could not have pulled off without the cluster manager.  One of the things that a scalable app does is “find” resources.  This can be application level, or in this case, infrastructure level, as I was using load balancing.  I had an error that I could not find because I did not know which server the logic ran on.  People often add loads of logging to store all of that information so they can go and retrace the steps that caused it.  I just said “screw it” and installed the Cluster Manager.  It took one request to reproduce it and I had all of the information I needed.

Seriously, if you have more than 2 or 3 servers and have enough budget to cover some basic employee benefits you NEED to look at Cluster Manager.  I knew the selling points of Cluster Manager before this, but I was sold on it when I needed it to do its thing when I recorded that video.

But I have digressed a little.

The new feature we added in both Zend Server and Zend Server Cluster Manager 5.1 was a new web API.  It allows you to programmatically access a subset of Zend Server’s provisioning features.  Some of them are available only to the Cluster Manager, but others are available for the regular version of Zend Server, i.e. the “single instance” version.

The feature list is as follows.

  • Get System Info
  • Get Server Status
  • Add Server
  • Remove Server
  • Disable Server
  • Enable Server
  • Restart PHP
  • Configuration Export
  • Configuration Import

The web API is RESTful and has an authentication method that calculates a request signature based off of a secret key that you can generate with either full access or read-only rights in the Zend Server GUI.

But I personally found that while the API was quite useful I could see that people would end up building their own adapters or scripting it in a way that would make it look like spaghetti.

So I wrote my own API that plugs in to the web API and makes your life easier.  I have included it as part of my Job Queue API up on GitHub and it requires PHP 5.3

There are a few simple steps to using it.

  1. Instantiate an API key object
  2. Instantiate a method object that represents the method you wish to call
  3. Instantiate the manager which controls the communication
  4. Call the method by passing it into the controller
  5. Read the response (which will either be raw data, a response object of a defined type (com\zend\api\response\*) or an array of response objects).

It’s really easy to use.  Take, for example, this code that will restart PHP on an individual server.

1
2
3
4
5
6
7
8
9
10
$mgr = new Manager();
$apiKey = new ApiKey(
'kevin',
'b562c7a6a46a9495df03e0b2c083f3ccfdb0de606b1abab05d056898fd6f410b'
);
$mgr->setApiKey($apiKey);
$mgr->setHost('192.168.0.246');
 
$restart = new RestartPhp();
$mgr->call($restart);

That’s it.  Or this code to export the current configuration.

1
2
3
4
5
6
7
8
9
10
11
$mgr = new Manager();
$apiKey = new ApiKey(
'kevin',
'b562c7a6a46a9495df03e0b2c083f3ccfdb0de606b1abab05d056898fd6f410b'
);
$mgr->setApiKey($apiKey);
$mgr->setHost('192.168.0.246');
 
$export = new ConfigurationExport();
$data = $mgr->call($export)->getResult();
// Store $data somewhere

Those are examples of API calls that work on the regular version of Zend Server.  Here are a few that work with Cluster Manager.

Say you want to add a server to your Zend Server cluster.  All you need to do is put this code in a provisioning script that runs on the node.

1
2
3
4
5
6
7
8
9
10
11
$mgr = new Manager();
$apiKey = new ApiKey(
'kevin',
'59455fb2e117e00bdcf04b37f9942da105e6bfd6a6cccf330fe79c79e89ddd4c'
);
$mgr->setApiKey($apiKey);
$mgr->setHost('192.168.0.242');
$addServer = new ClusterAddServer(
new ServerInfo('local', 'http://192.168.0.254:10081', 'password')
);
$mgr->call($addServer);

Done with the server and you want to remove it from the cluster (including graceful handling of session data)?

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$mgr = new Manager();
$apiKey = new ApiKey(
'kevin',
'59455fb2e117e00bdcf04b37f9942da105e6bfd6a6cccf330fe79c79e89ddd4c'
);
$mgr->setApiKey($apiKey);
$mgr->setHost('192.168.0.242');
 
$serverInfo = new ServerInfo();
$serverInfo->setId(get_cfg_var('zend.node_id'));
 
$addServer = new ClusterRemoveServer(
$serverInfo
);
$mgr->call($addServer);

As you can see with the new web API you now have the ability to automate a bunch of the more brainless tasks that you’d rather let a machine take care of.

Get Zend Server (trial available)

New Zend Server Job Queue Library

This will probably be one of the last posts I do on the Zend Server Job Queue functionality.  From this point on they will probably be less frequent, though I’m sure several posts will allude to it.  The reason I’m going to put it on the back burner is because I have written a library which is now available on GitHub.  What it does is encapsulate all of the functionality that I think needs to be there to be able to implement asynchronous functionality.  I’ve been working on it off and on for at least a year, trying out different things.

As I was working through it there were a few things that I wanted to accomplish.

  1. It needed to be easy to use
  2. It needed to be installed in a few minutes
  3. It needed to hide a lot of the implementation details.

That said, I think that I’ve been able to accomplish a lot of that.  With that, let’s take a quick run through.

Creating the job class

There is only one class that you need to know.  It is called JobAbstract and it is in the com\zend\jobqueue namespace.  Yes, the library requires PHP 5.3, though it works with PHP 5.2 code as I’ve already built a Magento extension for it.  In order to implement the job queue, on the code side, you simply create objects that represent your tasks, extending the JobAbstract class and implementing the job() method.  So if you had a job that you wanted to have send out an email it would look something like this.

namespace org\eschrade\job;
use com\zend\jobqueue\JobAbstract;
 
class SendEmail extends JobAbstract
{
    private $to;
    private $message;
 
    public function __construct($to, $message)
    {
        $this->to = $to;
        $this->message = $message;
    }
 
    public function job()
    {
        $mail = new \Zend_Mail();
        $mail->addTo($this->to);
        $mail->setBodyHtml($this->message);
        $mail->send();
    }
}

Then when you want to execute this job you simply call the job

$job = new org\eschrade\job\SendEmail('you@yourdomain', 'Welcome here!');
$response = $job->execute();

This will invoke the Manager object (which we’ll look at in a bit) which will then send the request to the specified job queue server, or to a load balancer.  The response is important if you want to get the results of the job.  If the request is serviced through a load balancer you don’t necessarily have any knowledge about where your job is going to execute.  Thus, if you want to get the result of the job you need to know how to get the job.  That information is stored in the result.  It’s a very simple, serializable object that is intended for persistent storage, such as a session or database.

Setting up the Manager

The manager is mostly self-contained.  The only thing it requires is that you provide a URL for the backend job queue end point.  The end point needs to only contain the following code.

1
2
3
4
use com\zend\jobqueue\Manager;
 
$mgr = new Manager();
$mgr->invoke();

Now, your application may require more at the end point to bootstrap it, but that’s all it needs to interact with your front end servers.

On the front end you need to tell the Manager what the URL is to connect to a job queue server.  As I said before, it can be an individual server or a load balancer in front of a hundred job queue servers, it really doesn’t matter, except that you can scale quite easily without any configuration changes.  To set the URL simply tell the Manager what the URL is.

1
Manager::setDefaultUrl('http://lb/queue.php');

And that’s pretty much it.

Communicating with the backend

Once you’ve sent the job off to the backend to execute you might want to interact with the results of the job.  Once the job object has finished executing, the Manager will serialize the job so you can get the results.  This would typically be done via getters and setters.  I have a good example of this in the library code on GitHub.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
class GetRemoteLinks extends JobAbstract
{
 
    private $url;
    private $links = array();
 
    public function __construct($url)
    {
        $this->url = $url;
    }
 
    public function getLinks()
    {
        return $this->links;
    }
 
    public function getUrl()
    {
        return $this->url;
    }
 
    public function job()
    {
 
        $http = new \Zend_Http_Client($this->url);
        $response = $http->request();
        $selector = new \Zend_Dom_Query();
        $selector->setDocumentHtml(
                $response->getBody()
         );
        foreach ($selector->query('a') as $result) {
            if ($result->getAttribute('href')) {
                $this->links[] = $result->getAttribute('href');
            }
        }
    }
}

To execute this job you would call

1
2
$job = new GetRemoteLinks('http://www.eschrade.com/');
$result = $job->execute();

Then later on

1
2
3
4
5
6
$mgr = new Manager();
if (($job = $mgr->getCompletedJob($result)) !== null) {
    foreach ($job->getLinks() as $link) {
        echo $link . '<br />';
    }
}

 

That’s pretty much it.

Pre-caching FTW

I just had an epiphany.  I’ve talked about pre-caching content before and the benefits thereof before.  But this is the first time I realized not only that there are benefits, but that doing it is BETTER than caching inline.  Let me sum up… no, there is to much.  Let me explain.

Typically caching is done like this (stolen from the ZF caching docs):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$id = 'myBigLoop'; // cache id of "what we want to cache"
 
if ( ($data = $cache->load($id)) === false ) {
    // cache miss
 
    $data = '';
    for ($i = 0; $i < 10000; $i++) {
        $data = $data . $i;
    }
 
    $cache->save($data);
 
}
 
echo $data;

Pretty easy.  But what happens if you have code like this:

1
2
3
4
5
6
$options = $app->getOption('google');
$client = Zend_Gdata_ClientLogin::getHttpClient(
       $options['username'],
       $options['password'],
       Zend_Gdata_Analytics::AUTH_SERVICE_NAME
);

What’s so important about this code?  Is it because it is of a remote nature?  Is it because it uses GData?  Nope.  It’s because it has a username and a password.  Given the previous caching what happens if that password changes (like mine did)?  Your site is down.

So, why do I now think that pre-caching is better than inline caching?  Look at my front page.  You would never know that I’m currently having a problem because it’s still reading from the same cache key (with non-expiring data).

THAT is why I’m forming the opinion that pre-caching/asynchronous caching not only has benefits over inline caching, but that it may actually be better.  I’m not one to make blanket statements, and I’m not going to.  But I am toying with the idea of using pre-caching as the default mechanism for caching instead of the other way around.

Tapping millions of non-PHP PHP developers. (A manager’s short guide for migrating from Java to PHP)

I was getting ready to post the Powerpoint presentations from my last two user group meetings when I fat-fingered my "My Documents" folder and accidentally opened up a Word doc that had an old article that I had written for a Zend newsletter a while ago.  So for this case of serendipity I decided that I would repost this article, which I had written long before I had a blog.  The basic premise is that once you get past a few minor differences, an organization can move from Java development to PHP development in a very short amount of time, saving time, money and headaches.

The Article

Every once in a while you are asked to write an article for a Zend newsletter. You come up with a couple of good ideas (all of which require a lot of work) and then one comes to you that is rather simple.  Then, to top it off, one of your customers sends you an email that confirms everything you were thinking. 

I have had the benefit of being able to program in many different languages for many different environments over the years.  Throughout that experience there are 2 languages that have risen to the top of my favorites list.  PHP is one, obviously.  But Java is the other.  Sometimes, though by no means always, there is some degree of derision towards Java in the PHP community among others. The usually points are made about Java performance, resource utilization and complexity.  And usually the points are true.  But Java, in my personal experience, is still an interesting language to write code in and it has an extremely diverse range of abilities that are not matched in most other languages.

But this is also a bit of a problem.  To access all of the functionality in Java, there is a tradeoff with simplicity.  To access all of the functionality for the web in Java, there is a massive tradeoff with simplicity.

In most cases, complexity costs more money and is more difficult to manage than simplicity.  That is not to say that complexity is a bad thing, or that maintaining software shouldn’t cost money.  But the more complexity you have in your application, the more complexity you have to maintain, which is in turn managed by complex tools that manage the complexity of your complex application.  This prompts me to ask 2 questions

  1. How much complexity (cost) your organization can absorb. 
  2. How much complexity (cost) your organization wants to absorb.

For a web-based environment, PHP is simply less complex and less costly than Java.  Java was never sold on the basis of cost of development and deployment, but in its write-once-run-anywhere nature and its vast array of abilities.  Moreso the latter.

At this point, you may be wondering how this fits in with the topic of non-PHP PHP developers.  And why should the manager of a Java shop even be considering PHP in the first place?  It is because if you have web-based applications and have Java developers, you already have PHP developers.  This is because there are a lot of similarities in terms of syntax between Java and PHP code.  Yes, there are differences, but not the sort of differences that stop your Java developers from writing PHP code.

This takes me to the customer whose comment I received.  The customer was a Java developer for 10 years, and in his words, took 2 days to get familiar with PHP.

2 days.

Let that sink in for a little.  In order for you to take advantage of the primary programming language of the World Wide Web, your Java developers could be writing usable PHP code in a matter of a few days (though probably a few more than just 2).  There are only a few important differences that your Java developers would need to take into consideration.  So let’s look at a few.

  1. The entry point is a script file, not a main(String[] args) definition or another kind of method call.  It is embedded in an HTML file, typically with the extension .php and starts with the delimiter
    <?php
    // Now I write code
  2. Variables start with a dollar sign.

    $ar = new ArrayObject();

  3. Variables are loosely typed, though not un-typed.

This means that unlike variable types can be acted upon as if they were like variable types.  For example, an array can be acted upon as a Boolean.

$ar = array(1,2,3,4,5);
if ($ar) {
            // The array was evaluated as a boolean
}

  1. Class methods are defined using the function keyword, not the return type.  There is no defined return type for PHP functions/methods.

    class User {
         public static function getUser($id) // no return type
         {
              return new User($id);
         }
    }

  2. Arrays are primitive PHP types which can contain ANY type of variable

You will typically not have different types of arrays like you would have LinkedList, ArrayList, etc.  An array (keyed or not) is an array.  However, if you wanted use arrays in an OO fashion you can use the ArrayObject class from the SPL library.

In terms of the bare minimum of differences, these 5 items are primary for Java developers.  There are definitely other differences, but to get started, this is virtually a complete list.  And it is reasonable to state that it is likely that your Java developers already know about points 1-3.  Most of the other rules of object oriented programming and web-based design that they have already been taught still apply.

Another way of saying it is that by moving to PHP, you retain all of the training and knowledge that you have invested in your developers while at the same time gaining the tremendous productivity and cost benefits of building your web-based applications in PHP. 

We can also take this a step further, which is the point of this entire article.  Say you have rebuilt your application in PHP and the CEO is inviting you to weekend trips to Pebble Beach because of all of the money you have saved the company and the reduced turn-around time of application development and deployment.  The CEO asks you to start moving other company applications onto PHP as well because it has been working so well.  Your peers may start questioning your decision to move to PHP because now you have to find more PHP developers.  But you have an edge.  You can hire any number of the millions of talented Java developers out there and, with a little guidance, have them writing PHP code in a matter of days.

How to (properly) evaluate Zend Server – Event Monitoring

So, you just received instructions to download and try out Zend Server.  Or, you heard that Zend Server is a “PHP Application Server”, but you have no idea what that means and you want to find out.  What do you do?

What I have often seen is that people will download and install Zend Server, try a PHP application on it and see if it works.  They see that it does work.  Then they ask “ok, it works but it’s not worth the price so I’ll just go back to what I was using before.”  The problem here is that more often than not, and I’m sure it’s a MUCH more often than not, “worth” is not defined.

The first thing thing that people do is look at cost.  Price, but also performance.  In other words, the first thing people often do to test Zend Server’s Event Monitoring is check to see what the performance overhead is.  What they’ll do is take JMeter, run a test session against Zend Server, and then become concerned because Event Monitoring has overhead.  Yes, Event Monitoring has overhead.  Yes, it will probably be a little more than what the marketing material states but we’re still only talking a few percentage points.  That may make a difference for Facebook or Google but it will probably not make that much of a difference to you.

But what if that few percent of overhead showed you problems that you either a) didn’t know existed, or b) were unable to replicate?

For example, one of the hardest problems to diagnose is a slow page request.  There could be literally hundreds of thousands of expressions that could be the culprit.  Or it could be that you HAVE hundreds of thousands of expressions.  But you don’t know.  In fact, it’s just as likely that you do not even know that you’re having a performance problem, or to what level you’re having it.  And if you do, your only production measurement is external.  In other words, all you know is what the elapsed time is.  There is also a relatively good chance that a lot of the information you need to reproduce the issue is not going to be readily available.  Things like, $_POST, $_SERVER, environment variables and such are often not seen in a PHP log.

I’ve worked in support before there is something I know; when the alarms start going off it’s too late.  When the alarms go off your customers are already being affected.  Most monitoring systems do not do real time monitoring.  At least none of the ones I’ve worked with.  And most systems monitor the OS or HTTP responses to determine if there are issues, and they do it at periodic intervals and they don’t record the cause of the error condition.  That’s what Event Monitoring does.

Why am I saying all of this stuff?  Note my statement at the beginning of this posting. I made the claim that people make the error of testing the performance of event monitoring when what you should be doing is measuring the cost of not having monitoring.  In other words, what does it cost you to have a 5 minute lag time before you know something is wrong.  What does it cost to have your customers start calling you asking you why your site is running slowly.  What does it cost for your developers to spend a week finding a performance issue when it could take them 4 hours.  I’m not joking on that last one.  It happened to me once.  It doesn’t happen often but it does happen.

But let me make a controversial statement.  At least it will be controversial if our sales people read it.  While Zend Server is about benefit and not cost, I would venture to say that when you first install it, it will actually end up costing you more.  I remember talking with someone once who looked at Event Monitoring and specifically stated they were not going to use Zend Server because it threw too many errors.  Let me translate that for you.  They were not going to use Zend Server because it was going to show them how much their application sucked.  I was absolutely flabbergasted at that.  I had no response.  Nothing.  Not a word.  How does one respond to that? … nicely?

I tell that story to emphasize a point.  Zend Server will not immediately begin to save you money, unless you are an absolute top-notch PHP programmer and you know exactly what’s going on in your app, all the time.  What it will immediately do is bring to the forefront what level of effort is going to be required to make your application compliant to the PHP language.  This cost will help you to write PHP code of a much higher quality than you did before.  Too many developers write their PHP code so-as to be utilizing the type juggling and forgiveness of the engine as much as possible.  This is not good PHP code.  Applications like that will definitely cause a lot of errors to be thrown with Event Monitoring.  However, applications like that are also much more likely to exhibit unexpected behaviors that you do not want in a production application.

Evaluating Event Monitoring (for the admins)

As an admin you will be primarily responsible for managing the system in your production environment and you would also be the one signing off on it saying that it will run.

When testing, and you want to see what the overhead is, do not use the defaults unless you have an extremely well-written and high performance application. The way to test for this is to do some kind of a load test (I use JMeter myself) and record the response times. The reason why you want to do this is because there is a series of events that you can monitor for. Having your values set to low will cause an inordinate amount number of events to be thrown. Remember, the purpose here is not to set the thresholds at what you would like the application to run at, or what your SLA says. The purpose here is to find out what your typical response time is, not to find out what requests are problematic. If your SLA for individual requests is lower than your average request time then you need to modify your application, not your thresholds. Here are a list of events that can be used to discover performance problems.

  • Severe Slow Request Execution (Absolute)
  • Slow Request Execution (Absolute)
  • Severe Slow Request Execution (Relative)
  • Slow Request Execution (Relative)
  • Severe High Memory Usage (Absolute)
  • High Memory Usage (Absolute)
  • Severe High Memory Usage (Relative)
  • High Memory Usage (Relative)
  • Inconsistent Output Size

Start by forcing events to be thrown by dropping your thresholds to 5-10ms, for performance, and 10-20k for memory issues. Run your test. Wait. Run again. Look at your second bunch of numbers. As I’m sure you’re aware, a cold cache is no cache, so give it time to warm up.

Assuming a relatively constant response time (+-50% or so) take the mean time and double it for your thresholds for Slow Request Execution (Absolute). Then double it again for the Severe level. These are not hard and fast numbers, but they can help you start to tune it. Remember, you’re tuning for realistic application performance, not for your SLA. If you can’t deliver the app within your SLA you either get better servers or better programmers. :-)

Once you’ve reset the monitoring numbers run the test again and see if the number of events have dropped off significantly.  If they have not, up your times.  If they have, then you can start to examine the events that you collected from the first run and start asking yourself questions like

  1. What is the actual performance impact?
    1. Will it significantly affect throughput?
  2. Will this reduce the time it takes to open a ticket and start working on a resolution?
    1. If so, by how much? If you say that it won’t reduce the time to start working on a resolution
  3. Does this give the developers better information than they get right now?

After asking these questions run the tests again with the tuned Event Monitoring settings and see if you get some events. Examine these and ask yourself, right off the top, “am I learning something about how my application behaves?” If you are, you are getting to see the benefit of working with Event Monitoring.

So, when testing Event Monitoring

  1. Set your thresholds low to force events to occur
  2. Use the timings from those events to tune the thresholds
  3. Run the test again
  4. If the event counts are decently lower, go to the next step, otherwise go to step 2
  5. Examine the data collected from the events to see if you can use it to reduce your Mean Time to Resolution

But there’s even more. However, we’ll look at that at a later article.

Evaluating Event Monitoring (for the developers)

Developers are of a different breed than administrators. I believe that is because developers don’t have pagers that go off in the middle of the night. Administrators like things that keep the pager from going off. Developers like cool things, that inevitably cause the pager to go off. When it comes to event monitoring there’s not much you need to do.  Believe it or not, that’s actually kind of a good thing.  Event Monitoring is intended to run external to your code.  In other words, you don’t need to hook into it.  It’s mostly an operational thing.

That said, there are some hooks that you can use and you do have an important part to play in the evaluation process.  We claim that Event Monitoring will help you reduce your MTtR (Mean Time to Resolution).  Test that.  Install Zend Server on your local machine.  Take an old version of your code, one that has an error to reproduce, and test it.  The best type of error to test is one that is difficult to reproduce. Slow page requests with no specific issue are a good place to start.

One of the things you want to do is write an application that does not fatally error out. Ideally you want to catch every error. Practically, that’s not possible, but that’s the goal. The problem with catching any error, like if you’re building a ZF MVC application, is that you can’t automatically catch errors. You can catch slow execution requests and such, but application errors are more difficult because, technically, the application hasn’t errored out. It caught the error.

However, all is not lost! There is an API call you can make. Let me show you in the error controller for my blog.

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
class ErrorController extends Zend_Controller_Action
{
    public function errorAction()
    {
        $errors = $this->_getParam('error_handler');
        switch ($errors->type) {
            case Zend_Controller_Plugin_ErrorHandler::EXCEPTION_NO_CONTROLLER:
            case Zend_Controller_Plugin_ErrorHandler::EXCEPTION_NO_ACTION:
            // 404 error -- controller or action not found
                $this->getResponse()->setHttpResponseCode(404);
                $this->view->message = 'Page not found';
                break;
            default:
                // application error
                $this->getResponse()->setHttpResponseCode(500);
                $this->view->message = 'Application error';
                $exception = $errors->exception;
                /* @var $exception Exception */
                if (function_exists('zend_monitor_set_aggregation_hint')) {
                    zend_monitor_set_aggregation_hint($exception->getMessage());
                    zend_monitor_custom_event(
                        'Internal Server Error Caught',
                        $exception->getMessage(),
                        array(
                            'trace' => $exception->getTraceAsString()
                        )
                    );
            }
            break;
        }
    $this->view->exception = $errors->exception;
    $this->view->request = $errors->request;
    }
}

 

There are two API calls here. The first one sets an aggregation hint. The way Event Monitoring works is that rather than generate multiple events it will aggregate similar events. Given that my title is “Internal Server Error Caught” any error that is caught will be aggregated according to that. So we aggregate according to title of the exception by calling zend_monitor_set_aggregation_hint() prior to calling zend_monitor_custom_event().

zend_monitor_custom_event() takes the following parameters.

  • Class – The nature of the event (aggregated)
  • Text – The description of the event (not aggregated)
  • User data – Any data that you want to store along with the event

When your admin is doing their testing make sure you check out the events that are being generated and check this out.

When an event is caught, it will also catch a lot of the information that you need to do a debug.  So with this button you can start a debug session in your browser that is kicked off using the same context as what caused the issue.  Things like GET and POST variables.  Additionally, if you click on the Settings button you can take that context and initiate it on a different server from where the event was collected.  So you could have an event kick off in production, but debug it in development with the same context.

Like with the admins, there is more, but we’ll look at that in a later article.

Conclusion

At this point we’ve really only scraped the surface of what we’re going to be looking at.  Event Monitoring is actually a two-part process.  This is the nifty part.  Next we’re going to look at the freaking cool part.

How to (properly) evaluate Zend Server – Introduction

So, today kind of got away from me and I was trying to think of what I could do to salvage this day when I came across an idea that I have had in the past.  As you all probably know, Zend has salespeople.  Those sales people have sales engineers who show how to use our products.  However, I personally hate being on the phone for a canned presentation when all I really want to do is tinker.  So, in an effort to produce something of benefit today I decided to start a series of blog posts on how to evaluate Zend Server if you are a tinkerer, like me.

What's it got?

The first thing we need to cover is what Zend Server has as part of its product offering.  Let me steal a list from our product page.

  • Native installation
  • Certified PHP
  • Zend Framework
  • Apache or IIS integration
  • MySQL (on Windows and Mac)
  • Built-in DB connectivity (Oracle, DB2, MySQL,..)
  • Java connector
  • Web-based administrator console
  • Debugger interface
  • Bytecode acceleration
  • Caching API
  • PHP 5.3 support
  • Clustering support with Zend Server Cluster Manager
  • Page caching
  • Application monitoring (alerting)
  • Application problem diagnostics
  • Code tracing
  • Job queue
  • Zend Download Server (Linux only)
  • Software updates and hot fixes
  • Technical support

However, for the purpose of this discussion we are going to focus on only 3 things.  Things are better examined in threes, anyway.  Those three things are Application Monitoring, Code Tracing and the Zend Server Job Queue.  Why only these three?  Because these three, IMHO, are the best three features here.  We have the fastest opcode accelerator (at least it was fastest the last time I checked), but that really doesn't matter because competing on performance is almost pointless because performance, in and of itself, is relatively easy to improve upon.  Cache it.  So why not compete on caching?  Because caching is boring.  To paraphrase Chef Gusteau, "Anyone can cache".  The reason I chose these three is because there are virtually no competitors for them, probably because they require a fair amount of work.  The one exception to that could be the Job Queue, which would be competing against Gearman, but I do prefer the Zend Server Job Queue.  Partially because I work for Zend but also, and not insignificantly, because I like the way it works.  But I'm getting ahead of myself.

I'm actually getting well ahead of myself. What I'm going to do is break this into several blog posts.  You should see each one as I do it linked up on the side under "Related Stuff".

Zend Server is proprietary. NNNOOOOO!!!!

In my Do You Queue article there was a comment posted about open source alternatives to Zend Server's Job Queue.  Being somewhat of an open source afficionado (I can spell that thanks to my love of fine tobacco products) that is a position that I can relate to.  I have heard several times "I like what Zend Server offers but I only want to use open software, or I don't think the features are worth the money."  I understand why.  "Open" is the new black, and has been for a while now.  So let's explore these two issues.  1) Proprietary software, and 2) the price of the software.

There are many places where I have no doubt that "open" is beneficial.  Chief among those is communication.  An open communication protocol can be easily integrated with.  TCP is a good example of this.  Without an open TCP communication protocol it would be difficult to have the Internet.  In fact we couldn't.  Same thing with HTML.  In fact we've seen with HTML the problems that can be associated with taking something that should be open and making it not so open. But as we've moved forward "open" and "standards" are becoming more and more important.

But does that mean that everything needs to be open?  From a purely utilitarian perspective, no.  I'm listening to Creed right now on proprietary speakers on a proprietary computer running a proprietary operating system.  Would I prefer that it was open?  Yeah.  Absolutely I would.  Do I really care?  Well, not really.  Why not?  Because I want to listen to Creed.  I record music using Cakewalk Sonar Producer.  Would I prefer that it was open?  Absolutely I would.  Do I really care?  Well, not really.  Why not?  Because I want to record music.  When I code I use Zend Studio.  Would I prefer that it was open?  Absolutely I would.  Do I really care?  Well, not really.  Why not?  Because I want to write code.  When I deploy my application I deploy using Zend Server.  Would I prefer that it was open?  Absolutely I would.  Do I really care?  Well, not really.  Why not?  Because I want things like monitoring of my PHP application, asynchronous execution, code acceleration and code tracing. 

I don't think that this position is overly controversial.  That is because the purpose of software is to solve a problem or do work.  That's it.  (Ok, software can also be written for fun too.)  But code written to be poetic it is pointless code.  If you are looking for existentialism in an "if" statement you won't find it.  If you are looking for joy in an architecture design you won't find it, unless that architecture solves the problem you are trying to solve.  Otherwise you are going to spend all of that joy trying to justify why you designed your system that way when it breaks or doesn't do the job it was supposed to.

So what if you are writing software for an open source platform (like PHP)?  Should you use proprietary software (like Zend Server)?  That's a question I can't answer.  The reason I can't answer that is because it depends on who your customer is.  Is your application going to deployed to a VHosted LAMP setup that is intended to be as cheap as possible?  Then, no, Zend Server is not a good solution to build for. 

But here's a question.  What if your application is going to be used in an operation where it's important to the bottom line of the customer?  What if your developer time is extremely important, and not cheap?  Going back two paragraphs; what if "open" really isn't truly all that important.  Let me make a strong and, perhaps, controversial statement.  If an application is either important to the bottom line or the developers are not cheap, there is no reason to not use Zend Server.  And in fact, it may be in your best interests.

Now, I know that the open source folks are may start getting hot under the collar at this point.  But let me explain myself first.  I have now used Zend Server since it came out.  That was originally when I was working in Zend's Global Services organization.  Often I would be brought in to do performance audits for people's applications.  People would spend 5 figures to get someone like me onsite and one of the first things I would do is install Zend Server and turn on Monitoring.  Once Zend Server 5 came out the first thing I would do is install Code Tracing.  Within a day I would have a pretty good idea of what was wrong and within 2-3 days I had a solution that I was working on.  Was it because I was hot stuff?  Hell no.  It was because I had the tools to diagnose the issue.

And the funny thing about it was that in many cases had the customer spend a few grand on Zend Server licenses they might not have needed to spend several thousand more on having me come out there.

"But!" I hear you say. "The likes of Yahoo and Facebook don't need Zend Server.  Why should I?".  That's a good, and a very valid, question.  The way I look at it is that there is a relationship between the number of developers and the number of servers you have that help you to determine that.  Do you have a thousand servers?  Zend Server might be too expensive, though I'd be VERY willing to bet that you could get a good deal if you had a thousand servers.  The question isn't whether or not Yahoo or Facebook need Zend Server, the question is whether or not you do.

Let's go back to what I had said about software.  It is there to do a job, or to help you with yours.  You, like me, might prefer that it's open.  But at the same time, if it means that you can go home at a decent hour do you mind that it's proprietary?  Maybe you do.  I dunno.  I like solving problems faster.  I also like having the ability to run things asynchronously out of the box, clustering out of the box and so on.

The way I look at Zend Server is that it is not about cost.  It is about benefit.  Would Zend Server cost you $10,000 a year?  Big deal!!  The real question is does it save you $10,001.  It might not, but when you factor in things like developer costs and such $10k in a developer organization of 3 people only requires a 4% increase in productivity, or less.  So, if you spend 5 hours a week fixing bugs, if you can reduce that by 1.5 hours per person you are already in the black.  Add to that reduced mean time to resolution through the use of Monitoring to notify you of problems that could be causing a material impact to business operations and the benefit starts to increase very quickly.  Even on this site, I know within a few seconds that an issue has occured in the code, which is something that typical monitoring, such as nagios and the like, cannot do.  For a blog like this Zend Server is overkill.  But unless I (personally) were heading up a business that was unprofitable and self-funded (meaning that it would be cash-strapped), Zend Server would be at the core of my operations.  Which brings me to my next point.

There is one thing that developers are not very good at.  In fact they tend to be really bad at it.  That is: operations.  I had a year and a half spat of working in a large operations department and you know what I found out?  Developers know squat about what actually needs to be done to manage their applications.  It's kind of interesting because the people who I hear from who are less convinced about Zend Server are generally developers and not sys admins.  That is partially because I fancy myself more a dev than a sys admin and I tend to run into developers more than sys admins.  But in my experience, system administrators tend to be more pragmatic and developers more poetic.  I think there's a very good reason for this.  It is the sys admins who are paged at 4:00 in the morning when the server goes down.  Being philosophical about software drops rapidly after a week of no sleep because of an unstable application.

So, coming back to my original reason for writing this article.  The question was asked if there was an open source alternative for the Zend Server Job Queue.  My response is that one should not look at the Zend Server Job Queue in isolation from the rest of Zend Server.  Could you write an application that uses Gearman and does asynchronous processing even better than Zend Server?  Maybe.  I haven't used Gearman yet, but I am willing to aknowledge that it might be true.  But that would be missing my point.  My point is that looking only at individual features is not the best way to look at Zend Server.  Can you get debugging, asynchronous execution, cluster-like functionality, caching and code acceleration elsewhere?  Of course you can.  But will it cost more to maintain your application in an environment of multiple disparate services than with Zend Server?  That question is not so easy to answer.

So, here is a summary of my thoughts.

  1. Proprietary isn't as evil is it's often made out to be – I am 90% convinced that for 90% of the people out there that proprietary is an easy objection to overcome because most people would simply like to either get more done or spend less time working.  Open is great for interoperability.  While, for developers, it would be nice for everything to be open, in practically, it is probably less important.
  2. Price is not the correct terminology to judge software on.  Cost/benefit is.  If the price is above the benefit then don't buy.
  3. Don't look only at development features.  Look at operational costs too.  Can you set up something with asynchronous execution in under 5 minutes?  I can.  And I can fully harness it in another 10 minutes. (That's how long it took me to re-implement that code in another application)
  4. Don't just look at operational costs.  Look at mean time to resolution.  Replicating and fixing bugs takes time.  Using Code Tracing and Monitoring you can drastically reduce that.  How do I know that?  Because I have only had one situation in what is now approaching 4 years at Zend where those two features did not provide significant benefit when dealing with bugs or production issues.

Given that some people will find this posting contentious, do consider that there are a lot of other considerations that I had made when writing this that I did not include.  Don't misunderstand me.  This article is not meant to be "pro-proprietary".  My purpose in writing this was to show that just because something is proprietary does not mean that it's as big of a problem as you might thing.

Zend Server Cluster Manager

On June 23rd Zend announced the release of Zend Server Cluster Manager.  Which means what, exactly.

PHP is designed using a shared-nothing architecture.  What that means is that nothing is shared.  What that really means is that each individual PHP request is isolated from every other PHP request.  That's great!  It makes for a very stable, very easy to use architecture.  But what happens when you go beyond one server?

Not much, in actuality.  How about a hundred?  Same thing (yes, that is to some degree a hyperbolous statement).  You might need to worry about scaling your database now, but the PHP side of things will continue to chug along.

But what about management?  Managing a hundred servers is quite different from managing one.  Heck, managing three servers is different from managing one.  As a side note, you should never have two servers.  If you have the budget for two you have the budget for three.  Why three?  Because when you take one down for maintenance you still have redundancy.  If you only have two you do not.

One of the primary ways that Zend Server (not Zend Server Cluster Manager) benefits you is in the monitoring of your PHP application.  Checking for long running requests, excessive memory usage, PHP errors and the like, storing that information so you can either replay them in the testing environment or, with Code Tracing, seeing the individual function calls with timing and memory usage.  Additionally, you have the ability to use the Job Queue to run tasks asynchronously.  You have the ability to manage your configuration via an easy to use GUI. You have the ability to run Session Clustering.

But you don't have the ability to look at your Zend Server servers as a holistic environment.  That is what Zend Server Cluster Manager does for you.

  • Centralized settings
  • Centralized Monitoring (including notification of unauthorized configuration changes)
  • Centralized Session Clustering management
  • Centralized everything

Got a hundred servers?  Need to make a change to php.ini?  Piece of cake.  Need to add a server and have sessions automatically distributed to the rest of the cluster?  Piece of cake.  Need to aggregate monitoring data from those hundred servers?  Piece of cake.

Setting up Zend Server Cluster Manager is really easy.  First, make sure that you have the Zend repository settings in your configuration.  In my case I'm using Yum.

[Zend]
name=Zend Server
baseurl=http://repos.zend.com/zend-server/rpm/$basearch
enabled=1
gpgcheck=0

[Zend_noarch]
name=Zend Server - noarch
baseurl=http://repos.zend.com/zend-server/rpm/noarch
enabled=1
gpgcheck=0

After that "yum install zend-server-cluster-manager".  The next thing you need to do is make sure that a MySQL server is available somewhere.  This MySQL server needs to be accessible to the individual Zend Server nodes so simply doing root@localhost won't be sufficient.  Have a non-standard user and make a good password.  Keep the server behind a firewall, but make it available to the nodes.

After you have installed Zend Server Cluster Manager you need to go to the GUI page which is on http://(server):10081/.  There you will agree to the terms and conditions and enter in the GUI password.  After the first two pages you will come upon a page that asks you for two different licenses.  The first license key is for the cluster manager itself.  The second is the license key that the cluster manager will use on all of the nodes.  After you have entered the license keys you will be asked to enter in the database information.  It will not accept the localhost as the database.  It must be an outward facing network address so the nodes can connect.

After you've done that you are ready to start adding nodes.  You can actually add existing Zend Server instances (which is kinda how it works anyway), but make sure that you have updated them to the most current revision first.  I didn't do that and submitted a bug report 3 days before the launch because a bunch of things were broken.  Hilarity did not ensue.

Adding a new server to the cluster is relatively easy.  In the Cluster Setup window click on the "Add Server" button.  Enter in the information.  Done.  When setting up an individual node you will need to at least go through the setup in the GUI once (again, server:10081).  You can enter in the license key if you want, but the only thing that's needed on the node is the password so that ZSCM can authenticate against it.  Once a server is part of a cluster you cannot manage it except through the ZSCM GUI.  Well, I take that back.  You can manage it by making changes locally via the CLI, but then ZSCM will throw a warning stating that a server is out of sync.  It won't remove it or anything like that, just tell you about it.

The benefit of Zend Server Monitoring

I was sitting at my computer this morning when an email came in notifying me that there was an error indexing my Lucene content.  This error was generated by the Zend Server event monitoring system.  I went to the site and tried doing a search but saw that the results were coming up double.

I re-ran the indexing task and the searches started coming up the way they should.  Mean time to resolution? 5 minutes.  Customer impact?  Zero.  Why?  Because I knew there was a problem before anyone coming to the site did.  THAT is the benefit of Zend Server Event Monitoring.

Web Analytics