Benchmark Setup

To compare mod_php to FastCGI with a PHP opcode cache, I used the ab tool on my desktop running PHP 5.3.0, mod_fastcgi 2.4.6 and Apache 2.2.11 on Gentoo linux. Apache had a limited number of modules enabled (actions alias authz_host dav deflate dir expires filter headers log_config mime rewrite setenvif status vhost_alias). The hardware consisted of a Intel Core2 Quad Q6600 (2 x 4MB L2 cache) with 2GB of RAM.

I chose to benchmark against the main page of a stock WordPress 2.8.2 install with no plugins enabled. I chose WordPress because it’s extremely easy to set up and many people are familiar with it. I’ve read that WordPress is very liberal with queries to the database (especially when plugins are involved). Without database queries, I would expect to see a bigger difference between results with an opcode cache and results without one. That being said, most PHP applications today make database queries as part of a typical request and thus I think these results are realistic of what you might experience on your server.

I executed the following command:

in four different environments:

• mod_php with a 30MB APC opcode cache (Apache MPM prefork)
• mod_php without an opcode cache (Apache MPM prefork)
• mod_fastcgi, suEXEC with a 30MB APC opcode cache (Apache MPM worker)
• mod_fastcgi, suEXEC without an opcode cache (Apache MPM worker)

Between each benchmark, I reconfigured and restarted the server. I then loaded the main page in a web browser to prime the APC cache.

Benchmark Results

Requests

Requests per second

The graph above shows that mod_php and FastCGI behaved very similar in terms of requests per second. Using an opcode cache increased the request rate by almost 3 times in both scenarios. As I mentioned earlier, I’d expect to see an even bigger increase with fewer database queries.

I saw similar results when increasing the number of requests to 10,000, or changing the concurrency level. I never saw any failed requests. I was happy to see there wasn’t a significant performance difference moving from mod_php to FastCGI.

Memory Usage

Memory usage

The graph above shows the approximate memory usage for each of the 4 benchmarking scenarios. This data was gathered by running the ps_mem.py script immediately after the ab tool completed. This script outputs the private and shared memory usage for each program running on the system. Because I ran my benchmarks in isolation, with nothing else accessing the host, these results should be pretty accurate.

The results surprised me. From the graph you can see that disabling the opcode cache resulted in more memory usage. I think this is misleading. Typically, Linux keeps memory resident until it is needed by another process. I believe the increased memory usage for the non-opcode cache scenarios is actually discarded memory that the Linux kernel hasn’t reclaimed. Since each process is having to recompile the PHP script on each request, you see a lot more built up memory reported as private but most likely free to be reused. I suspect running this on a low memory server would yield slightly different results.

You can clearly see the difference in memory usage for the Apache processes (green in the graph) between FastCGI and mod_php. The FastCGI Apache processes are much leaner. Since Apache doesn’t need the PHP bloat, it can serve static content much more effectively and quickly.

The scenarios with an opcode cache each used a similar amount of RAM. There is a catch to these results however: I only benchmarked a single user. In a shared environment, each user gets their own PHP instance and opcode cache. With mod_php, all users share the same. Thus I would expect every user to roughly double the RAM used in a FastCGI scenario, while mod_php memory usage would stay relatively constant. This increased memory usage is the main trade-off when going with FastCGI.

Raw Results

For those who want even more detail, below are the raw results from my benchmark tests:

mod_php without an opcode cache:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)

Server Software:        Apache
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        5474 bytes

Concurrency Level:      30
Time taken for tests:   28.507 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      5717000 bytes
HTML transferred:       5474000 bytes
Requests per second:    35.08 [#/sec] (mean)
Time per request:       855.200 [ms] (mean)
Time per request:       28.507 [ms] (mean, across all concurrent requests)
Transfer rate:          195.85 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       2
Processing:   138  849 214.5    842    1608
Waiting:      138  848 214.3    842    1601
Total:        139  849 214.4    843    1608

Percentage of the requests served within a certain time (ms)
  50%    843
  66%    911
  75%    975
  80%   1004
  90%   1112
  95%   1230
  98%   1355
  99%   1426
 100%   1608 (longest request)

mod_php with opcode cache:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)

Server Software:        Apache
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        5474 bytes

Concurrency Level:      30
Time taken for tests:   10.784 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      5717000 bytes
HTML transferred:       5474000 bytes
Requests per second:    92.73 [#/sec] (mean)
Time per request:       323.530 [ms] (mean)
Time per request:       10.784 [ms] (mean, across all concurrent requests)
Transfer rate:          517.70 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       2
Processing:    66  319  62.9    316     580
Waiting:       66  318  62.9    316     580
Total:         67  319  62.8    316     580

Percentage of the requests served within a certain time (ms)
  50%    316
  66%    339
  75%    354
  80%    364
  90%    395
  95%    424
  98%    464
  99%    482
 100%    580 (longest request)

FastCGI without an opcode cache:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)

Server Software:        Apache
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        5474 bytes

Concurrency Level:      30
Time taken for tests:   28.170 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      5695000 bytes
HTML transferred:       5474000 bytes
Requests per second:    35.50 [#/sec] (mean)
Time per request:       845.104 [ms] (mean)
Time per request:       28.170 [ms] (mean, across all concurrent requests)
Transfer rate:          197.43 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       2
Processing:   105  832  82.9    832    1004
Waiting:      105  832  82.9    832    1004
Total:        107  832  82.8    832    1004

Percentage of the requests served within a certain time (ms)
  50%    832
  66%    856
  75%    873
  80%    882
  90%    908
  95%    927
  98%    954
  99%    971
 100%   1004 (longest request)

FastCGI with opcode cache:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)

Server Software:        Apache
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        5474 bytes

Concurrency Level:      30
Time taken for tests:   10.917 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      5695000 bytes
HTML transferred:       5474000 bytes
Requests per second:    91.60 [#/sec] (mean)
Time per request:       327.509 [ms] (mean)
Time per request:       10.917 [ms] (mean, across all concurrent requests)
Transfer rate:          509.44 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       2
Processing:    46  323  36.6    325     426
Waiting:       46  323  36.6    325     426
Total:         47  323  36.5    325     426

Percentage of the requests served within a certain time (ms)
  50%    325
  66%    335
  75%    342
  80%    346
  90%    359
  95%    368
  98%    383
  99%    389
 100%    426 (longest request)

Memory Usage (in MB):

Conclusion

I run my blog on a Linode VPS with a very limited amount of RAM. I also host other applications and a few friends’ sites on this same server. Thus, I am very concerned about memory usage. Still, the security advantages provided by FastCGI far outweigh the increased RAM usage.

I hope you have seen that the performance of FastCGI with an opcode cache is just as good as with mod_php. It’s also evident how important an opcode cache is to running PHP scripts.

If you have any thoughts about these benchmarks, or the performance of FastCGI vs mod_php, feel free to leave a comment or question below!