I'll preface this post by saying these are not reliable results, the environment is not typical of a production server.. this was only meant as a general evaluation.

2.16 GHz Dual Core Darwin OS (Apple OS 10.4.9)
2 GB Ram
L2 Cache: 4MB per processor
Bus Speed: 667 MHz
120 GB 5400 rpm Serial ATA (SATA) Gen-I (1.5 Gbps)
PHP v5.2.1
Apache 2.2.4 (APC 3.0.12p2 disabled)

Standard default install (all modules), no modifications.

ab -c 10 -t 60 http://localhost/development/pn8
This is ApacheBench, Version 1.3d <$Revision: 1.73 $> apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Completed 30000 requests
Completed 35000 requests
Completed 40000 requests
Completed 45000 requests
Finished 50000 requests
Server Software: Apache/2.2.4
Server Hostname: localhost
Server Port: 80

Document Path: /development/pn8
Document Length: 241 bytes

Concurrency Level: 10
Time taken for tests: 14.658 seconds
Complete requests: 50000
Failed requests: 34500
(Connect: 17250, Length: 17250, Exceptions: 0)
Broken pipe errors: 0
Non-2xx responses: 32757
Total transferred: 16575042 bytes
HTML transferred: 7894437 bytes
Requests per second: 3411.11 [#/sec] (mean)
Time per request: 2.93 [ms] (mean)
Time per request: 0.29 [ms] (mean, across all concurrent requests)
Transfer rate: 1130.78 [Kbytes/sec] received

Connnection Times (ms)
min mean[sd] median max
Connect: 0 0 0.3 0 20
Processing: 0 2 1.6 3 20
Waiting: 0 3 0.9 3 19
Total: 0 2 1.7 3 20

Percentage of the requests served within a certain time (ms)
50% 3
66% 4
75% 4
80% 4
90% 5
95% 5
98% 6
99% 7
100% 20 (last request)


edit: added php/apache versions, disabled APC



edited by: InvalidResponse, Apr 18, 2007 - 03:49 PM

Devin,

We looking at optimising the code at the moment - there have been some improvements since MS3.

Also note that the out of box configuration isn't the most performant. .8 is designed to be cached. I'd suggest turning on pnRender caching or, even better, Theme page caching.

-Mark

--
Visit My homepage and Zikula themes.

thanks mark.

Devin,

I'd add that i've noticed some problems, at least on my local dev setup, with PHP v5.2.1 that have an extremely negative effect on performance. I've not had sufficient time to isolate this yet.

-Mark

--
Visit My homepage and Zikula themes.

Interesting. I ran the same options against a little "home spun" app I use for small sites that uses Smarty and ADOdbLite which should have performed a lot better .. wondering if it may have anything to do with either one or both of those libs. I don't know that it has significant bearing, but I've read that require_once is pretty expensive and both libraries use the function quite a bit. Interested to hear your findings.. thanks again mark.



edited by: InvalidResponse, Apr 18, 2007 - 04:05 PM

Hi IR,

can you please run an identical benchmark against a 0.764 install? This would give us a baseline to compare things to ...

Greetings/Thanks
R

Hi Robert,
fresh install, no modifications/configuration changes:

ab -c 10 -t 60 http://localhost/development/pn764
This is ApacheBench, Version 1.3d apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Completed 30000 requests
Completed 35000 requests
Completed 40000 requests
Completed 45000 requests
Finished 50000 requests
Server Software: Apache/2.2.4
Server Hostname: localhost
Server Port: 80

Document Path: /development/pn764
Document Length: 243 bytes

Concurrency Level: 10
Time taken for tests: 14.265 seconds
Complete requests: 50000
Failed requests: 34482
(Connect: 17241, Length: 17241, Exceptions: 0)
Broken pipe errors: 0
Non-2xx responses: 32762
Total transferred: 16708620 bytes
HTML transferred: 7961166 bytes
Requests per second: 3505.08 [#/sec] (mean)
Time per request: 2.85 [ms] (mean)
Time per request: 0.29 [ms] (mean, across all concurrent requests)
Transfer rate: 1171.30 [Kbytes/sec] received

Connnection Times (ms)
min mean[sd] median max
Connect: 0 0 0.3 0 19
Processing: 0 2 1.5 3 12
Waiting: 0 3 0.8 3 12
Total: 0 2 1.6 3 19

Percentage of the requests served within a certain time (ms)
50% 3
66% 4
75% 4
80% 4
90% 4
95% 5
98% 6
99% 7
100% 19 (last request)

the "issue" appears to be Smarty .. I'd like to see what .750 looks like.. anyone have a copy?



edited by: InvalidResponse, Apr 26, 2007 - 04:15 PM

Thanks to AmmoDump

http://host.dpahl.com/~pndave/downloads/pn_archive/ver0750/PostNuke-0.750.zip
I wonder...

- Igor

thanks Igor..! Smarty was added in .750, but not as integrated, so I think it'll give a better idea of this vs. that ;)

thanks again.

PS: I just realized these are in the NOC.. checked out .726 too (egads! .. like time travel)

v.726 Default install. APC disabled.

ab -c 10 -t 60 http://localhost/development/pn726/index.php
This is ApacheBench, Version 1.3d apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Finished 1017 requests
Server Software: Apache/2.2.4
Server Hostname: localhost
Server Port: 80

Document Path: /development/pn726/index.php
Document Length: 20282 bytes

Concurrency Level: 10
Time taken for tests: 60.046 seconds
Complete requests: 1017
Failed requests: 0
Broken pipe errors: 0
Total transferred: 21079845 bytes
HTML transferred: 20710485 bytes
Requests per second: 16.94 [#/sec] (mean)
Time per request: 590.42 [ms] (mean)
Time per request: 59.04 [ms] (mean, across all concurrent requests)
Transfer rate: 351.06 [Kbytes/sec] received

Connnection Times (ms)
min mean[sd] median max
Connect: 0 0 0.0 0 0
Processing: 100 587 170.8 585 1200
Waiting: 100 587 170.8 585 1200
Total: 100 587 170.8 585 1200

Percentage of the requests served within a certain time (ms)
50% 585
66% 647
75% 690
80% 721
90% 791
95% 855
98% 906
99% 1019
100% 1200 (last request)

WordPress 2.1.2:

dhayes$ ab -c 10 -t 60 http://localhost/test/publishers/wordpress/
This is ApacheBench, Version 1.3d apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Finished 104 requests
Server Software: Apache/2.2.4
Server Hostname: localhost
Server Port: 80

Document Path: /test/publishers/wordpress/
Document Length: 6258 bytes

Concurrency Level: 10
Time taken for tests: 60.006 seconds
Complete requests: 104
Failed requests: 0
Broken pipe errors: 0
Total transferred: 728784 bytes
HTML transferred: 700896 bytes
Requests per second: 1.73 [#/sec] (mean)
Time per request: 5769.81 [ms] (mean)
Time per request: 576.98 [ms] (mean, across all concurrent requests)
Transfer rate: 12.15 [Kbytes/sec] received

Connnection Times (ms)
min mean[sd] median max
Connect: 0 0 0.0 0 0
Processing: 5116 5411 146.9 5423 5641
Waiting: 5116 5411 146.9 5423 5641
Total: 5116 5411 146.9 5423 5641

Percentage of the requests served within a certain time (ms)
50% 5423
66% 5475
75% 5519
80% 5554
90% 5588
95% 5606
98% 5620
99% 5630
100% 5641 (last request)

Joomla 1012:
dhayes$ ab -c 10 -t 60 http://localhost/development/Joomla_1012/
This is ApacheBench, Version 1.3d apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Finished 945 requests
Server Software: Apache/2.2.4
Server Hostname: localhost
Server Port: 80

Document Path: /development/Joomla_1012/
Document Length: 22016 bytes

Concurrency Level: 10
Time taken for tests: 60.020 seconds
Complete requests: 945
Failed requests: 631
(Connect: 0, Length: 631, Exceptions: 0)
Broken pipe errors: 0
Total transferred: 21142490 bytes
HTML transferred: 20747062 bytes
Requests per second: 15.74 [#/sec] (mean)
Time per request: 635.13 [ms] (mean)
Time per request: 63.51 [ms] (mean, across all concurrent requests)
Transfer rate: 352.26 [Kbytes/sec] received

Connnection Times (ms)
min mean[sd] median max
Connect: 0 0 0.0 0 0
Processing: 126 632 88.3 627 1078
Waiting: 126 632 88.3 626 1078
Total: 126 632 88.3 627 1078

Percentage of the requests served within a certain time (ms)
50% 627
66% 645
75% 664
80% 677
90% 721
95% 769
98% 855
99% 913
100% 1078 (last request)

.726 appears to perform pretty well in comparison.

PHP 5.2.2
APC disabled
PN v0.8 pnrender caching enabled, compile check disabled.
Themes caching enabled, compile check disabled:



PS: ran 2 sets. re-started apache between each test, after the 3rd, restarted the machine ran 2 more re-starting apache befor each, the results were pretty "competitive"..

PPS: run against the may 7 export



edited by: InvalidResponse, May 07, 2007 - 03:42 PM

Just wondering - if I create a module that simply outputs "hello world" and exit(); will there be routines run from the core that are not required to acheive this? ie render stuff (Smarty, theme), database stuff (adobe), etc, etc. Is postnuke set up to only run what is required at the time?

I am asking this becuase I want to only use pn8's core permissions and user libraries / functionality without any extra overhead from other non required functionality

-Lobos



edited by: Lobos, May 18, 2007 - 03:35 AM

--
-Lobos
Professional PHP Framework Services: Concept, Development and Deployment

If you are looking to get a solid chart I would also suggest running tests on the following versions.

.80 MS1 & 2
.721
.714
.703

It would be interesting. If the older versions aren't on the NOC i think I still have them archived along with probably .6x series :)

Check out the latest SVN version. Jörn and Mark put some time into optimizing performance. If you use the "blank" module you have Lobos suggested "hello world" module

As far as I understood Jörn, there's no unnecessary SQL call for such a basic call.

--
best regards from Kiel, sailing city

Steffen Voss

Member of the Zikula Steering Committee
Read The Zikulan's Blog "If you want people to RTFM, make a better FM!"

I have just performed a performance review of a stripped down .8 SVN checkout - with no visible blocks and a "hello world" module as you suggest. As a result of this I was able to locate a few places where we could improve performance (which has been done and in SVN). So, as far as I'm able to debug and read the profiler output, there's no unnecessary overhead involved in displaying a simple page.

Mark has in addition to this implemented a singleton pattern for the pnRender/Smarty stuff. This has removed a lot of the performance issues from pnRender.

So, at the time of writing, we are not aware of any easy-to-spot bottlenecks in the core.

I have installed a test-site with the 25. June .8 snapshot together with a customized Pagesetter database to test pnRender and theme caching. pnRender caching works well, and theme caching has improved a lot - but with the following defect: manual entries in Pagesetter filter forms are cached, too. So once you have filtered for something, this filter and its results will always be returned and overwrite any new manual filter entry. Using [nocache] for the whole template did not resolve the issue. The same is true for check boxes in the core setup, e.g. after checking the theme-cache-checkbox you cannot uncheck it anymore. Only manual removing all cache-files from the server enabled me to uncheck it again.

Comparing with a friend the PN0.8 SVN20071007 (almost RC2) with Joomla 1.5 RC3, here are the Benchmarks results.

Important Note:
This is over a desktop PC, nothing compared with a server hardware capabilities. Just Ubuntu 7.04 with xampp 1.6.4 (Apache/2.2.6 PHP/5.2.4)


PostNuke 0.8
pnRender & Theme cache enabled, template check disabled.
Not using sessions for unregistereds.





Joomla 1.5 RC3
Cache enabled (it seems that enabling the cache in Joomla is worst that have it disabled , without cache complete 1807.98 requests per second)




Clean installations,
Andreas08 PN theme vs default Joomla theme,
PN: Who's online, Main Menu, Login block. Two News articles.
Joomla: Who's online, Main Menu, Login Form, Two News articles (mods, components, whatever!) all the rest disabled!


Obviously, i'm not a Joomla fan but my friend yes, and the results with the phisical order of PN, and the cache results, give him some reasons to think that PN is better.
Just to think a little, i'm happy with the pnRender performance, and with the cache design off course, i know that can be improved in a further, because with cache and without PN cache, the difference is minus than 100 request per second...

--
- Mateo T. -
Mis principios... son mis fines

Muy bien mateo... buen post

..this pleases me so :) .. it's probably time I grab a copy, brew some coffee and tool around.

thanks fellers.. truly appreciate the efforts.

Edit:

Seems a little high.. I don't even get those results with a static HTML file



edited by: InvalidResponse, Nov 28, 2007 - 01:37 PM