When you are benchmarking a web server, you may fall into the trap of benchmarking against a small file (maybe the debian default “It’s working!” index.html file). I decided to write about this pitfall, so that my friends & readers will get a more realistic benchmark.
I’ve found the following general guidelines are a good idea to follow when running a benchmark on a web server:
- You should benchmark with gzip enabled, since that will more realistically simulate what’s going on between most browsers and your web server.
- You should benchmark from another machine (preferably remote).
- You should benchmark against a real page on your site, not a very small test file (for example benching with robots.txt is a bad idea).
With Apache Bench (“ab”), you would do this by adding the following switch: -H “Accept-Encoding: gzip,deflate”
An example “ab” command would look like this:
ab -k -n 1000 -c 100 -H "Accept-Encoding: gzip,deflate" "http://test.com/page/"
This command will simulate 100 concurrent users, each one performing 10 requests for a total of 1000 requests. The -k switch will instruct Apache Bench to use KeepAlive.
If your sample web page weighs 250k uncompressed (just for the HTML body), that’s a lot of data to transfer between your web server and the machine from which you are performing the benchmark tests. The problem is that the network interface (or other transfer medium), will probably choke well before you achieve the maximum requests per second, and you may find yourself confused and spending time trying to tweak nginx or varnish, where in fact you are just hitting the limit of your network card or maybe even an enforced maximum speed policy by your hosting company (some might limit your network card at the Switch level, and some others will use Rate Limiting at the Firewall level).
But in the real world, it would probably be compressed anyway (especially if your config enables gzip, which in most cases it does by default). Such a page might be compressed to 10% of its original size (in this case to 25k). This will give you a much nicer and more realistic “Requests Per Second” number.
A proper, realistic benchmark should test a page that goes all the way through to the database and performs one or more queries. This allows you to make sure that you have proper caching enabled at the various levels of your web applications.
An important note about KeepAlive
With Apache: If you are experiencing very high traffic (many concurrent users), it may be a good idea to keep this value rather small. Say between 3 and 5 seconds in most reasonable cases. You do this so the Web Server can free worker threads for new connections, rather than keeping them alive with a user who may have already finished loading the page. If not enough threads are free for new connections, new users will have to wait for a while until such threads are free. If new users are coming at a pace faster than the pace in which threads are freed up, they will all be blocked and never reach your page. On the other hand, you may not want to turn KeepAlive off entirely, especially if you have a lot of images, css files, javascript files, all imported from your pages, because that would mean that for each such resource, your web server will have to spawn another worker thread, and that might cause a load spike if you experience a lot of traffic. So it’s a fine balance: You want it enabled, but you don’t want it to be too long, especially in the age of broadband, where a properly compressed page could finish transferring to the user in less than 3 seconds.
With Nginx: Threads in nginx work differently, so I personally feel that keepalive can be kept at 60 seconds with nginx, without cost to memory/thread pools.