Aolserver performance benchmark with ab

Dated 27-03-07
Syed Atif Ali

Introduction:
I have always noticed that aolserver memory size keep growing, and does
not stay at a max state . There are some known tcl versions which have memory
leak issue . but even after using the recommended versions of tcl , this problem,
though decreased in severity, but still exists.
After being hinted by Aolserver community gurus that the problem could lie in the
thread allocator that tcl uses, I decided to compile aolserver against Google
perftools which has a reputation of providing a "more efficient” thread allocator .
Instructions on installing aolserver with Google perftools can be found at the end
of this document .There is no network involved in the tests. Yes I know it means
that aolserver performance will be affected by the ab process. but than this will be
a constant for each test so I will ignore this. My main purpose of performing these
tests is to see the difference in Aolserver performance with or without Google
tcmalloc() .

Required :
Test if Google perftools enhances the performance of Aolserver.

Test Environment :
Here is my test environment :
Constants :
Dual-Core AMD Opteron(tm) Processor 2210 HE
3 GB memory
Red Hat Enterprise Linux ES release 4 (Nahant Update 4)
2.6.9-42.ELsmp x86_64 GNU/Linux

Variables:
TCL versions:
tcl 8.4.14: compiled with enable-threads
tcl 8.4.14 compiled enable-threads but without thread allocator to
use Google tcmalloc.
Aolserver 4.0.10 : Linux native thread allocator
Aolserver 4.0.10 : With Google perftools provided tcmalloc()
Aolserver 4.5 : Linux native thread allocator
Aolserver 4.5 : With Google perftools provided tcmalloc()
Aolserver also loads nsoracle 2.7 and nsmysql latest driver modules. But my
tests don’t involve any database connectivity .
Test files: Helloworld.html (
Helloworld.adp
Testhtml.html
Index.adp (Default Aolserver 4.5 file , found in
server/server1/modules/pages)
Other Software Used:
 Gnuplot 4.2: to plot graphs
ab-2.0.40: Apache benchmarks
Google perftools 0.8-1
sar: to collect CPU stats
openoffice writer, calc.

I ran each test multiple times. This machine was running nothing additional and
the CPU load was always before starting any test , between 0.5-0 . I made sure
before starting every test the CPU is in 100% idle state . Aolserver is configured
with maxconnections=100 and maxthreads=10 . ab runs with 10 to 100
simultaneous connections and performs 10,000 requests in each test. .Between
each ab test there is a 5 second pause, which is obvious in graphs
Some terms:
Aol40orig : Aolserver 4.0.10 compiled with tcl which uses native Linux thread
libraries , --enable-64bit , --enable-threads, compiled against tcl-8.4.14 which is
also compiled with enable-threads
Aol40g : Aolserver 4.0.10 and tcl compiled with Google perftools libraries , --
enable-threads –enable-64bit . tcl with –enable-threads but disabled thread
allocation
Aol45orig : Aolserver 4.5 compiled with tcl which uses native Linux thread libraries
, --enable-64bit , --enable-threads, compiled against tcl-8.4.14 which is also
compiled with enable-threads
Aol45g : Aolserver 4.5 and tcl compiled with Google perftools libraries , --enable-
threads –enable-64bit . tcl with –enable-threads but disabled thread allocation

Now straight to the charts

Helloworld test:
Tested with one html and one tcl file both display hello world. This test is
just a comparison of simple html and adp file serving
Tested :
Aol40orig for tcl and html files.

Both these graphs show that aolserver performs at par with both tcl and html
files. cpu consumption and requests per second seem similar in both tcl and html
servings. difference minimal and html serving is a lil better .
Aolserver 4.0.10 with and without google perftools
Tested :
Default page that comes with aolserver 4.5 as index.adp .
Aol40orig and aol40g .

Oh ho , this doesn't look good . I took these test 6 times, starting each test with
cpu being 100% idle. but always got the same results. this shows that aolserver
4.0.10 performs better whatever the simultaneous connections are in its original
state than compiled with google perftools. Even the cpu consumption isn't
different. for some reasons aolserver with tcmalloc() performs bad after 70
simultaneous connections to the web server .This test actually pushed me to test
the same setup on Linux 2.4 kernel. maybe Linux NPTL in 2.6 did really good stuff.
and google perftools might help in Linux 2.4 kernel , i am not sure i will test it
later.
Aolserver 4.5 with and without google perftools
Tested :
Default page that comes with aolserver 4.5 as index.adp .
Aol45orig and aol45g .

Aolserver 4.5 lost to aolserver 4.0.10 in any form , as far as the requests per
seconds are concerned. In this test Aolserver 4.5 with google performed better
than the original alserver 4.5 . In both cases the performance drops after 50
simultaneous connections .
Aolserver 4.0.10 and Aolserver 4.5 both original
Tested :
Default page that comes with aolserver 4.5 as index.adp .
Aol40orig and aol45orig

Aol4.0.10 server way much better than aolserver 4.5 .

Aolserver 4.0.10 and Aolserver 4.5 both with google perftools libraries
Tested :
Default page that comes with aolserver 4.5 as index.adp .
Aol40g and aol45g

Again aolserver 4.0.10 performs better but drops performance after 40

simultaneous connections ,
aol45 with google perftools is better than aolserver 4.0.10 with google perftools.

Result :

On 2.6 kernel platform , there is no benefit of using google perftools for aolserver
enhancement .
If your need is performance improvement than even aolserver 4.5 does not
provide major improvement. but aolserver 4.5 might provide a lot more of
features than aolserver 4.0.10 .