Optimizing your (ZF) web application

Over at IDG, we're currently working on moving all our applications from a php4 codebase to php5 (as everyone should). In the process we're also porting all of our applications to Zend Framework (talking about migrations ;)). After finishing our first project, and moving it over the performance testing phase and all, we encountered quite a few issues. There are some typical things you can avoid, so bare with me.

A few days ago, I found this very interesting blogpost by Till, which shows a lot of common pitfalls for optimizing your ZF application. While we don't have the visitor numbers he has for this particular project, we also have a lot less hardware to work with. Let me emphasize that. We have a lot less hardware to work with.

For this particular application, we have two, dualcore, 2ghz machines available, with 2GB of memory, and some default scsi hd hardware (nothing fancy). Above on that, the 2GB memory is shared with an apache 1.3 instance, using up about 1.4GB of memory. So we end up with about 600MB for our second apache2 and php5 instance. Not much I can tell you

This post is divided in a few topics:

• Calculating what you need
• The actual benchmarking
• General optimizations
• Apd and php profiling
• Zend Framework tips

Calculating what you need

Before optimizing your application, you might want to calculate the amount of throughput you actually need for the application. In this case, we had about 3.000.000 pageviews, and about 250.000 unique visitors per month. You want to optimize for the peak performances you are going to need. We divided the pageviews by 28 (least amount of days), which got us about 108.000 pageviews per day, dividing that by 24, 60 and 60, gives us the least amount of throughput you are going to need for your application:

3.000.000 / 28 / 24 / 60 / 60 = 1.24 request/sec

The same calculation can be done for the concurrent users:

250.000 / 28 / 24 / 60 / 60 = 0,1 concurrent user

So, not a whole lot But. We need the application to outperform the current stats. And we want to have the application primed for higher traffic. On top of that, these numbers are (again), the least amount of traffic you are going to get. On peekmoments (look into your site analytics for those), you'll get much higher demands. Figure out what your timescale is in which you are receiving your visitors. For this particular project, the most active hours are between 11.00 (11 am) and 22.00 (11 pm). So that's 11 hours instead of 24:

3.000.000 / 28 / 11 / 60 / 60 = 2.7 request/sec
250.000 / 28 / 11 / 60 / 60 = 0,2 concurrent user

So that's the actual number you want to go for. Because we wanted to scale up for larger traffic, we've set our goal higher: 10 req/s and 100 concurrent users.

The actual benchmarking

As you can read in Till's blogpost, you can easily test with apacheBench (ab) on the commandline. Be sure to test this on a line you know isn't capped or cached. The best way to test is to benchmark on the local machine. But you might not be in the position to actually run it locally. @mathieuk pointed me to Siege, which can do a lot more in-depth testing (multiple url's and such), which might come in handy. I'll stick with ApacheBench for this blogpost, but be sure to check it out.

To test the performance, we start off with an easy test, to get a baseline result. 1000 pageviews on the homepage, by 100 concurrent users:

(don't forget the trailing slash)

You'll get a lot of info about the test. Be sure to run the same test every time after you optimized some code, so you know if you've already reached your goal, and you can actually compare the statistics. The number you want to look for is:

Or whatever your number is. (btw: that's the actual output for webwereld.nl)

General Optimizations

Right, let's get to business. but before that, there are a couple of really simple things you should do for every web application you built:

Make sure AllowOverride None is set

Setup the ZF rewrite rules in your virtual host config (or httpd config). You don't want to parse a simple .htaccess file for every request. It shouldn't change that much, so no need to have it in your application anyway. (or setup your virtualhosts over svn and deploy those as well, works like a charm for us)

Make sure all your images are served via a CDN

Setting up a DNS alias for static files is the easiest. Serve all your images, stylesheets and all through that static host, you can even have server[1-4].static.* for your hostname, and serve them over the same server. Your browserclients will love you for that. (no more then 4 cdn servers though, as Yahoo describes).
Be sure to serve your static files through a small webserver instance, look at lighttpd, nginx or similair, or setup a very tiny small apache binary for serving the static files.
Don't forget to add caching and gzip-ing. On a linux/apache machine, we use the following rules for our static domains:

Install Yslow for firebug

Yahoo has lots of interesting suggestions that can teach you a lot about client side optimizing your code, which is usually a lot less work then optimizing a backend

Install APC (or similair)

Really.

The Real Stuff (tm)

After setting up these issues, we can finally get into the ZF optimizations. We'll use php's apd module for that, and use pprofp to test the performance of our application.

As you can read on the APD page, you need to setup the apd_set_pprofp_trace(); from the point you want to start parsing. In our ZF application, you might want to put on the very first line (after the opening tag ofcourse), of the index.php file in your public dir.

After that, call the page once, a pprofp trace file will be written to disk with all the output you are going to need. To see a quick overview of the calls that are the most expensive in your application, you can call pprofp with the -R option, and you'll get something like this:

From this view, you can easily distinguish which methods are called the most often, and where you need to look for the best optimizations you can make. (you can actually see more lines with the -n option). You can also get a complete listing of the complete application tree. This might be easy to get a quick overview of where code gets repeated a lot. If so, you can try to make sure to optimize those pieces at once.

The first time we ran pprofp, our tree listing was a staggering 229.602 lines long (woops!). This means that for every homepage request, the parses needs to parse and read through those 229.602 lines. After a few runs, we got the total number of lines down to 36.810 lines, a lot better, but it's a lot of manual labour. Reading through the dumps afterwards, we filtered out the following things to keep in mind:

Zend Framework tips

Take a good look at your bootstrap.

Do you really need that ridiculously long include path? Be sure to setup the most used paths in front of the include path, not in the back. Use plugins with caution, don't set them up, if you don't need them.

Zend_Loader is (slightly) faster then require_once

Not much, but it can become quite significant if you have a lot of includes. As Till also mentions, it might be good to write your own xyz_Loader, and use that for your inclusion paths. Although your mileage may vary.

Zend_Db

It's an awesome piece of code, but it can be very slow. Try to avoid it for larger, queries, especially if your doing joins over multiple tables. We've had instances were we had over 120 lines in our call tree, for a query which only outputs a total amount of something.
If you don't need fancy prepare and execute functionality, you can get a simple query value really easy like this (yes, it does have it's downsides):

In this way, you can save up a lot of redundant calls, which aren't used anyway (and you cache the output really easy with Memcache).

Zend_Controller_Router_Route

Be sure to use Zend_Controller_Router_Route if you can, try to use Zend_Controller_Router_Route_Regex as little as possible, but you already figured that one, right?

Zend_Filter

We have a UBB filter, which parses all our user comments for UBB tags ([url][/url], etc). We've built our own filter for that, but be sure to make it as efficient as possible. As the php doc mentions, you should use str_replace whenever possible. But be sure to still check if your pushing it to the max.

Zend_Config

We have a application config (ini) file for every application. This config file is written in our deployment process. We access these values in our application via:

That's fine and works like a charm. But every time you call the $config->application->title, array_key_exists is called, which can end up in something like this:

Not very efficient now, is it? So if you have a value which you are going to use more often, make sure to set it up as a constant (which it actually is), like so:

then you can just call the constant, while still using the parsed config file.

Other stuff

Some other important issues to take into account when optimizing code:

Avoid shell_exec

really, it's slooooooow, (up to 77 msec per call in our situation). Which can really add up if you have it in multiple places. We had a simple:

for every thumbnail (don't ask). which took a lot of resources, use alternatives when available.

Optimize Smarty

Be sure to unload the smarty modules you don't need

Ofcourse there are a lot more tips for optimizing code out there. In this blogpost I wanted to show you how you can look through your (ZF) code, and show you which common pitfalls there are. Every application is different, so your mileage may vary. If you have any more tips, please let me know in the comments.