Home > Server Admin > FastCGI with a PHP opcode cache benchmarks

FastCGI with a PHP opcode cache benchmarks

July 29th, 2009

In my previous post, I described how to implement FastCGI with a PHP opcode cache on an Apache webserver. My primary motivation for moving to FastCGI was to take advantage of the extra security provided by FastCGI with suEXEC over mod_php. In this post, I’ll compare the two environments and provide a few benchmark results.

Benchmark Setup

To compare mod_php to FastCGI with a PHP opcode cache, I used the ab tool on my desktop running PHP 5.3.0, mod_fastcgi 2.4.6 and Apache 2.2.11 on Gentoo linux. Apache had a limited number of modules enabled (actions alias authz_host dav deflate dir expires filter headers log_config mime rewrite setenvif status vhost_alias). The hardware consisted of a Intel Core2 Quad Q6600 (2 x 4MB L2 cache) with 2GB of RAM.

I chose to benchmark against the main page of a stock WordPress 2.8.2 install with no plugins enabled. I chose WordPress because it’s extremely easy to set up and many people are familiar with it. I’ve read that WordPress is very liberal with queries to the database (especially when plugins are involved). Without database queries, I would expect to see a bigger difference between results with an opcode cache and results without one. That being said, most PHP applications today make database queries as part of a typical request and thus I think these results are realistic of what you might experience on your server.

I executed the following command:

$ ab -n 1000 -c 30 http://localhost/

in four different environments:

  • mod_php with a 30MB APC opcode cache (Apache MPM prefork)
  • mod_php without an opcode cache (Apache MPM prefork)
  • mod_fastcgi, suEXEC with a 30MB APC opcode cache (Apache MPM worker)
  • mod_fastcgi, suEXEC without an opcode cache (Apache MPM worker)

Between each benchmark, I reconfigured and restarted the server. I then loaded the main page in a web browser to prime the APC cache.

Benchmark Results

Requests
Graph displaying requests per second of each benchmark scenario

Requests per second

The graph above shows that mod_php and FastCGI behaved very similar in terms of requests per second. Using an opcode cache increased the request rate by almost 3 times in both scenarios. As I mentioned earlier, I’d expect to see an even bigger increase with fewer database queries.

I saw similar results when increasing the number of requests to 10,000, or changing the concurrency level. I never saw any failed requests. I was happy to see there wasn’t a significant performance difference moving from mod_php to FastCGI.

Memory Usage
Graph displaying memory usage of each benchmark scenario

Memory usage

The graph above shows the approximate memory usage for each of the 4 benchmarking scenarios. This data was gathered by running the ps_mem.py script immediately after the ab tool completed. This script outputs the private and shared memory usage for each program running on the system. Because I ran my benchmarks in isolation, with nothing else accessing the host, these results should be pretty accurate.

The results surprised me. From the graph you can see that disabling the opcode cache resulted in more memory usage. I think this is misleading. Typically, Linux keeps memory resident until it is needed by another process. I believe the increased memory usage for the non-opcode cache scenarios is actually discarded memory that the Linux kernel hasn’t reclaimed. Since each process is having to recompile the PHP script on each request, you see a lot more built up memory reported as private but most likely free to be reused. I suspect running this on a low memory server would yield slightly different results.

You can clearly see the difference in memory usage for the Apache processes (green in the graph) between FastCGI and mod_php. The FastCGI Apache processes are much leaner. Since Apache doesn’t need the PHP bloat, it can serve static content much more effectively and quickly.

The scenarios with an opcode cache each used a similar amount of RAM. There is a catch to these results however: I only benchmarked a single user. In a shared environment, each user gets their own PHP instance and opcode cache. With mod_php, all users share the same. Thus I would expect every user to roughly double the RAM used in a FastCGI scenario, while mod_php memory usage would stay relatively constant. This increased memory usage is the main trade-off when going with FastCGI.

Raw Results

For those who want even more detail, below are the raw results from my benchmark tests:

mod_php without an opcode cache:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)


Server Software:        Apache
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        5474 bytes

Concurrency Level:      30
Time taken for tests:   28.507 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      5717000 bytes
HTML transferred:       5474000 bytes
Requests per second:    35.08 [#/sec] (mean)
Time per request:       855.200 [ms] (mean)
Time per request:       28.507 [ms] (mean, across all concurrent requests)
Transfer rate:          195.85 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       2
Processing:   138  849 214.5    842    1608
Waiting:      138  848 214.3    842    1601
Total:        139  849 214.4    843    1608

Percentage of the requests served within a certain time (ms)
  50%    843
  66%    911
  75%    975
  80%   1004
  90%   1112
  95%   1230
  98%   1355
  99%   1426
 100%   1608 (longest request)

mod_php with opcode cache:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)


Server Software:        Apache
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        5474 bytes

Concurrency Level:      30
Time taken for tests:   10.784 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      5717000 bytes
HTML transferred:       5474000 bytes
Requests per second:    92.73 [#/sec] (mean)
Time per request:       323.530 [ms] (mean)
Time per request:       10.784 [ms] (mean, across all concurrent requests)
Transfer rate:          517.70 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       2
Processing:    66  319  62.9    316     580
Waiting:       66  318  62.9    316     580
Total:         67  319  62.8    316     580

Percentage of the requests served within a certain time (ms)
  50%    316
  66%    339
  75%    354
  80%    364
  90%    395
  95%    424
  98%    464
  99%    482
 100%    580 (longest request)

FastCGI without an opcode cache:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)


Server Software:        Apache
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        5474 bytes

Concurrency Level:      30
Time taken for tests:   28.170 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      5695000 bytes
HTML transferred:       5474000 bytes
Requests per second:    35.50 [#/sec] (mean)
Time per request:       845.104 [ms] (mean)
Time per request:       28.170 [ms] (mean, across all concurrent requests)
Transfer rate:          197.43 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       2
Processing:   105  832  82.9    832    1004
Waiting:      105  832  82.9    832    1004
Total:        107  832  82.8    832    1004

Percentage of the requests served within a certain time (ms)
  50%    832
  66%    856
  75%    873
  80%    882
  90%    908
  95%    927
  98%    954
  99%    971
 100%   1004 (longest request)

FastCGI with opcode cache:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)


Server Software:        Apache
Server Hostname:        localhost
Server Port:            80

Document Path:          /
Document Length:        5474 bytes

Concurrency Level:      30
Time taken for tests:   10.917 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      5695000 bytes
HTML transferred:       5474000 bytes
Requests per second:    91.60 [#/sec] (mean)
Time per request:       327.509 [ms] (mean)
Time per request:       10.917 [ms] (mean, across all concurrent requests)
Transfer rate:          509.44 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       2
Processing:    46  323  36.6    325     426
Waiting:       46  323  36.6    325     426
Total:         47  323  36.5    325     426

Percentage of the requests served within a certain time (ms)
  50%    325
  66%    335
  75%    342
  80%    346
  90%    359
  95%    368
  98%    383
  99%    389
 100%    426 (longest request)

Memory Usage (in MB):

Benchmark Apache PHP Total
private shared total private shared total
mod_php, no cache 202.3 7.7 210 210
mod_php, with cache 45.3 15.5 60.8 60.8
FastCGI, no cache 9.2 3.3 12.5 68.3 5.0 73.3 85.8
FastCGI, with cache 9.2 3.6 12.8 23.4 13.8 37.2 50

Conclusion

I run my blog on a Linode VPS with a very limited amount of RAM. I also host other applications and a few friends’ sites on this same server. Thus, I am very concerned about memory usage. Still, the security advantages provided by FastCGI far outweigh the increased RAM usage.

I hope you have seen that the performance of FastCGI with an opcode cache is just as good as with mod_php. It’s also evident how important an opcode cache is to running PHP scripts.

If you have any thoughts about these benchmarks, or the performance of FastCGI vs mod_php, feel free to leave a comment or question below!

Server Admin , , , , , ,

  1. September 3rd, 2009 at 01:30 | #1

    500 – 2000 requests per second on ab -n 10000 -c 30 for default theme wordpress.

    I had only 1.7 gb of ram and a much lessor processor than your set up.

    I noticed your post and decided to use your ab test settings against a new subdomain blog using default theme on a mu install already under a fair bit of use. There are quite a number of plugins installed in mu-plugin folder.

    I got between 500 – 2000 requests per second on /category/uncategorized/, /2009/09/02/hello-world/, /. Memory and CPU use increase were negligible.

    Using Nginx, php-fpm, memcached and the new nginx fastcgi cache. Memcached is easy to set up. You might like to look into it for your linode with batcache and its other files. The real star is nginx and its new fastcgi cache.

  2. September 3rd, 2009 at 08:35 | #2

    @Hone
    Very impressive.

    From reading your post on the Nginx fastcgi cache I assume the main reason you are getting these results is because you are serving the cached page for all but the first request. As you mentioned, this is very similar to using WP Super Cache and is equivalent to serving static files. I didn’t use WP Super Cache in my benchmarks as I wanted to show the result of fetching raw PHP pages (to show the effect of an opcode cache). On actual sites I use WP SuperCache and it speeds up the site significantly. Opcode caching and page caching are two different things, and in my opinion both should be employed on a PHP site.

    While I still have yet to use Nginx, I have read it can help speed things up as well, especially on my linode where I’m much more resource constrained.

  3. September 3rd, 2009 at 20:31 | #3

    Yes I realize your doing you were primarily testing for the opcode cache and my test is not really equivalent.

    I think Nginx is particularly good if you have low server resources such as memory or cpu.

    Good luck with your server.

  4. September 3rd, 2009 at 20:32 | #4

    Sorry my comment makes no sense. ;-)

  5. December 12th, 2009 at 13:52 | #5

    Great, I run my site on a linode vps too, and we have pretty similar settings!

  6. March 19th, 2010 at 08:53 | #6

    Brandon, this is great stuff. Thanks for providing some hard stats on these issues.

    I think that extra memory usage without APC is real – PHP doesn’t return memory to the operating system until it exits. And PHP needs the memory for compiling purposes immediately before it needs yet more memory to run the actual PHP code, so there isn’t much of an opportunity for pages to swap out and make room for other processes.

    On the other hand, it’s possible your stats aren’t counting the APC cache shared by all of the processes at least once… some OSes won’t show the shared memory at all, others will count it against every process, neither is really entirely fair for benchmarking but we know the size of the APC cache is fixed and we can take that into account when we read stats as long s we know which behavior the OS exhibits in its stats

  7. March 19th, 2010 at 08:56 | #7

    One thing your article doesn’t explicitly measure is the impact of migrating PHP out of Apache on the server’s ability to deliver static content at the same time it is delivering PHP content. I would love to see a benchmark in which static content is also being retrieved at the same time. Apache + FastCGI + APC should beat the daylights out of Apache + mod_php + APC in that scenario, which is of course the real world scenario. Your benchmark seems to beat up only on PHP, so it makes sense that mod_php + Apache and FastCGI + Apache are comparable in that situation.

    (Of course one could also expect good results with an nginx front end serving the static stuff and proxying the PHP stuff to Apache + mod_php + APC. Just another way of reaching the same goal)

  8. March 19th, 2010 at 09:06 | #8

    @Tom Boutell
    I agree benchmarks comparing static content (in addition to dynamic PHP content) would be good to see. I didn’t do a great job of “selling” FastCGI in this post (with regards to raw speed).

    As for the shared APC cache, I think hope I counted that correctly. The ps_mem.py script I used to measure memory does a pretty good job of measuring shared memory usuage in linux.

  9. July 26th, 2010 at 09:52 | #9

    Thanks again, I think i’m justified to give it a try on one of my servers.
    I’ll let you know if i succeed

  10. Hayden
    November 19th, 2011 at 18:52 | #10

    @Tom Boutell
    The last part of your post IS our setup on an Amazon Micro ubuntu ec2 instance using nginx for static files, images, js, css… everything except PHP! PHP serving is passed back to Apache via 127.0.0.1:8008.

    Every setup is different and all php code loads different. For us Mod_php is over 30% faster thank Apache running PHP as FastCGI. Our code is IP.Board by invisionpower. It may be different with blogs that are way less dynamic than php forums.

  11. Loggy
    April 19th, 2012 at 06:12 | #11

    Brandon

    This is very interesting. I am running a skinny VPS with mainly WordPress sites and want to make it as efficient as possible.

    I am using apache2 with mod_php. APC 3.1.3p1 (not the newest) is already enabled and apc.php shows that the it has a shared memory of 30MB – is this the same as your setup? The user cache is empty though – it is not set up for individual users.

    I am considering moving production sites to nginx (leaving dev sites on a different IP but still port 80) but it may be that changing to FastCGI is a better solution at least to start with. nginx needs FastCGI anyway (or Apache to service php).

    The question I have is that I have the server locked down with permissions. Files are in {username}:www-data and ftp is also in the wwww-data group. With permissions either 0640 or 0660 for files adn 2751 or 2771 for directories as appropriate, this stops prying on world-readable files (I have to run a correction script regularly because of sloppy plugin/theme updates of course).

    Would such a permissions system work with Apache2/FastCGI? I don’t want to have contents {username}:{groupname=username} because that would mean problems for ftp I think. And a web server running under www-data surely needs to write to the tree at some time – or is that only ever done by php?

  12. April 19th, 2012 at 09:28 | #12

    @Loggy
    FastCGI runs as whatever user you set it to run as. This can be the webserver user (nobody/www-data), or the user that owns the files. If you run FastCGI as the user, then permissions shouldn’t be a problem as the PHP process will be able to write to the same files as the user. If you run FastCGI as the webserver user, then the same rules as mod-php would apply.

  13. Loggy
    April 19th, 2012 at 10:26 | #13

    @Brandon

    Many thanks for that swift response!

    I will try it first on a test cloud server I think but presumably at the same time the number of threads and processes in Apache could be relaxed as Apache will have a smaller footprint. At the moment it is quite locked down and reload Apache every day so if I look at the load times of sites, quite a lot is waiting time which I interpret as waiting for another slot. This is presumably where nginx would score if the server becomes loaded.

  14. Conor
    May 26th, 2014 at 12:38 | #14

    The trouble is that your test is not strictly comparing the performance of mod_php versus FastCGI. Instead it is measuring the performance of wordpress (a fairly slow framework) when running on mod_php versus FastCGI.

    If the question is whether mod_php or FastCGI runs faster (which is my question) then you really need to test it by running a php script that basically does nothing, and then exits. That way your apache benchmark tests are effectively testing the total startup times of FastCGI and mod_php. If you throw a slow loading website on top of that, then of course there is going to be little apparent performance difference in number of requests completed per second.

  15. July 7th, 2014 at 13:25 | #15

    I see a lot of interesting articles on your page.
    You have to spend a lot of time writing, i know how to save you a
    lot of work, there is a tool that creates unique, SEO friendly articles in couple of minutes,
    just type in google – laranita’s free content source

  16. September 20th, 2015 at 22:25 | #16

    Very soon this website will be famous among all blogging
    and site-building visitors, due to it’s good posts

  17. June 18th, 2018 at 22:33 | #17

    It’s really very complex in this active life to listen news on TV, thus I simply
    use world wide web for that reason, and obtain the hottest news.

  1. December 23rd, 2009 at 08:48 | #1
  2. December 23rd, 2009 at 18:07 | #2
  3. December 23rd, 2009 at 20:21 | #3
  4. December 28th, 2009 at 16:32 | #4
  5. July 13th, 2011 at 04:10 | #5
  6. March 20th, 2012 at 11:15 | #6
  7. August 28th, 2015 at 14:35 | #7
  8. September 29th, 2015 at 16:19 | #8
  9. February 5th, 2016 at 10:14 | #9