Optimize GlassFish Performance in a Production Environment

Performance White Paper February 2009

Abstract
un GlassFish Application erver v2 is a hi!h performance application server" #his $hite paper $ill provide some !uidance on ho$ to tune the application server and is intended to be used as an introductory step to achievin! better performance for your application" Eleven tunin! parameters $ill be described $ith some !eneral recommendations" #hese recommendations can be used to optimize the performance of the application server" #hese recommendations are follo$ed by data from a benchmar%in! e&ercise that demonstrates the effectiveness of the tunin! chan!es"

Table of Contents
#ip '( )ava *ersion""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + #ip 2( )*, ,ode""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + #ip -( )ava .eap ize""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + #ip +( #une )ava Garba!e /ollection"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" 0 #ip 0( .##P acceptor threads""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" 0 #ip 1( .##P re2uest processin! threads"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""0 #ip 3( 4eep Alive subsystem""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" 1 #ip 5( .##P File /ache"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""1 #ip 9( 6efault7$eb"&ml"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" 3 #ip '0( )68/ #unin!""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" 5 #ip ''( 6isable Access 9o!!in!"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" 9 pecific #unin! for /ool#hreads#, #echnolo!y on ystems $ith :ltra PA;/; #'<#2 Processors ystem pecific #unin!"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" 9

;eferences"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" '' Appendi& =""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" '#ip ' 7 /han!in! )ava version""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" '#ips 27+ 7 Editin!<Addin! )*, options"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" '#ip 0 7 .##P Acceptor #hreads"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""'+ #ip 1 7 .##P ;e2uest Processin! #hreads""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" '1 #ip 3 > 4eep Alive settin!s"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""'3 #ip 5 > .##P File /ache""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""'5 #ip 9 > Edit the default7$eb"&ml"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""2' #ip '0 > )68/ #unin!"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" 2' #ip '' > 6isable Access 9o!!in!""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" 22 #ip for /ool#hreads :ltra PA;/ #'<#2 #unin!"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""2;otatin! access lo!""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""2:se libumem"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" 2:se of F? schedulin! class""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""2@et$or% /onfi!uration""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" 2-

'

un ,icrosystemsA =nc

E&ecutive ummary
#his $hite paper $ill describe and provide !eneral !uidance for initial tunin! values for eleven parameters in order to improve your application performance on the un GlassFish#, application server v2" Additional specific tunin! advice is also provided for the /ool#hreads :ltra PA;/ #'<#2 platform" While optimal values may be different for each deployment of the GlassFish application serverA it is useful to provide these first eleven steps as an initial primer to benchmar%in! your o$n application"

=ntroduction
Obtainin! optimum performance from your application deployed on the GlassFish application server re2uires a thorou!h understandin! of your obBectives" For someA best performance is attained $ith ma&imum throu!hput $hich is defined as the number of transactions completed per second" For othersA best performance may be the ma&imum number of users supported on a system concurrently $ithin a desired response time" :nfortunatelyA there are no universal tunin! parameters you can set that $ill reap the best performance" #here are many factors to consider such as the type of applicationA operatin! systemA )ava versionA system hard$are architectureA deployment architectureA number of usersA e&pected response timeA and net$or% band$idth" While all of these variables may initially appear dauntin!A the aim of this $hitepaper is to reduce the comple&ity and provide some initial steps to consider to achieve your performance !oals on the GlassFish application server"

,ethodolo!y
8efore embar%in! on a tunin! e&erciseA one of the most important steps is to choose your application and testin! environment appropriately" #oo oftenA one rushes to apply tunin!s to improve performance $ithout spendin! enou!h time to analyze the problemA the desired metrics and a thorou!h scrutiny of the collected data" #his methodolo!y includes(
choosin! the appropriate application( =n !eneralA the actual application used in

production is a !ood place to start" =f this application is too comple&A then it may be necessary to $rite a microbenchmar% that is representative of the most common scenario" =t is $orth$hile to spend some time analyzin! $hat the most common usa!e patterns are in your application" #his seems obvious but

un ,icrosystemsA =nc

choosin! the incorrect benchmar% can lead to misleadin! results about soft$are and hard$are re2uirements and ultimately a $aste of time and money" /ontinuous evaluation of $hat you are testin! versus $hat you have in production $ill avoid the red herrin!s" electin! the incorrect application to benchmar% is a%in to choosin! a for% to drin% soup"
plannin! your test environment( =deallyA it is best to have separate system for

drivin! the load and another for your Csystem under testCA or :# on a dedicated separate net$or% so that the net$or% band$idth and traffic are not limitin! factors" =f this is not possibleA you can create separate virtual containers De!" olaris EonesF or different processor sets" For e&ampleA you could have the driver and :# on the same machine but create t$o separate processor sets to monitor the usa!e of the driver and :# resources separately"

6river

GlassFish Application erver

6atabase

Fi!ure '" #ypical benchmar%in! setup $ith driverA :# and 68 on separate machines"
:nderstand the metric bein! measured" For e&ampleA solely loo%in! at

throu!hput could be misleadin! $ithout ta%in! into account the response time" #ypicallyA a threshold value of response time is defined as a passin! criterion" =f the measured response time e&ceed this valueA the transaction is deemed as a failure" A transaction in this conte&t represents a scenario Dor many scenariosF that you are testin!A such as addin! an item to a shoppin! cart or lo!!in! out" ince the avera!e response time of all the transactions can be s%e$ed by a fe$ outlyin! data pointsA a typical statistical measure used is the ma& 90thC percentile" For e&ampleA suppose that the criteria for response time for a particular transaction is 2 seconds" #he overall test is considered a success if the response times are 2 seconds or belo$ for 90G of all recorded transactions" A necessary component is investi!atin! the load testin! tool you are usin! > both the behavior of the load !eneratin! client and the statistics bein! reported" OccasionallyA the client can be found to be the bottlenec% in benchmar%in! and conse2uently cannot place enou!h load on your system under test" AlsoA spend some time not only analyzin! the measurementsA but ho$ the measurements are reported by the tool" #here are many !ood open source and commercial tools availableH one $orthy of mention is FabanA an

un ,icrosystemsA =nc

open7source driver and harness frame$or%"

Plan ho$ the test is carried out( #ypicallyA benchmar%in! can be divided into

three phases" #he firstA the $arm up periodA is used to increase the client load by addin! more concurrent threads" #his $arm up period should be lon! enou!h for the server )*,#, to $arm up and allo$ it to finish the )=# compiler optimizations" #he measurement of the system under test should occur durin! the second periodA the steady state" #his should be lon! enou!h so that the data collected durin! this period is representative sample of the test" Each benchmar% $ill have a de!ree of randomness $ith respect to the metric reported so the steady state duration needs to be lon! enou!h to minimize the randomness" #he final phaseA the ramp do$nA allo$s each client thread to end !racefully" /onsideration should also be made of the Cthin% timeC > the period of time bet$een the end of one re2uest and the be!innin! of the ne$ one"
,onitor basic statistics durin! your benchmar%in! test" Obtainin! a number at

the end $ithout %no$in! ho$ your system behaved durin! the test is not meanin!ful" /ollectin! system statistics li%e /P: utilizationA net$or% and dis% =O statistics durin! your test can serve as an e&cellent source of information" #his introduction of benchmar%in! methodolo!y is by no means completeA but hopefully it can serve as a 2uic% primer on ho$ to !et started"

#op '' #unin! Parameters for GlassFish

#his sectionCs title can lead to incorrect assumptions so it is necessary to add a caveat emptor(

The best tuning tip is the one that improves YOUR application's performance. Your mileage may vary.
=t is impossible to determine a set of tunin!s that $ill improve every application deployed on the GlassFish application server and predict ho$ much your throu!hput $ill increase" 8enchmar%in! consists of a repetitive cycle of testin!A apply tunin!A re7testin! so you $ill need to carry out your e&periments" .o$everA faced $ith all of the %nobs to turnA here is a list of the eleven you can start $ithA in no particular order" ince )ava#, PlatformA tandard Edition 0 D)ava E Platform 0F A there is an er!onomics feature $here the )ava platform $ill match the choice of !arba!e collectorA heap size and runtime compiler based on the class of the machine" #his minimizes the amount of tunin! if there are no e&plicitly set server )*, options" .o$everA since it is impossible to optimally tune for every application $ith er!onomicsA there may be certain parameters than can be t$ea%ed above

un ,icrosystemsA =nc

and beyond the assumptions made by this feature" Please refer to http(<<Bava"sun"com<docs<hotspot<!c0"0<er!o0"html for more details"
@ote( #he e&act steps of confi!urin! each parameter $ill be covered in Appendi& ="

#ip '( )ava *ersion #ypicallyA many performance optimizations are al$ays bein! incorporated in ne$er releases of )ava E" Whenever possibleA it is a !ood idea to up!rade to the latest version of )ava E Platform7to ta%e advanta!e of these optimizations" #ip 2( )*, ,ode 8y defaultA the GlassFish application server is confi!ured to use the client *," For performance testin!A it is recommended to chan!e this to I7serverJ" o $hy is the app server set to I7clientJ by defaultK #ypicallyA the GlassFish application server is installed in developer profile so by usin! the 7client settin! allo$s for ease of development li%e faster startup and deployment times" @ote that if you are usin! the GlassFish application server in cluster or enterprise profileA the 7server should be the default option" #ip -( )ava .eap ize #he size of the heap is determined by the )ava options 7?m& Dma&imumF and 7?ms DminimumF" While a lar!er heap can contain more obBects and reduce the fre2uency of !arba!e collectionA it may result in lon!er !arba!e collection times especially for a full G/ cycle" #he recommendation is to tune the heap based on the size of total available memory in your systemA process data model D-27bit or 1+7bitF and operatin! system" #he heap size should not be bi!!er than the physical memory" 8y defaultA the GlassFish application server is set to 0'2 ,8 and the limit for a -2 bit )*, version7 is appro&imately -"0G8 on olarisA and '"0 G8 on Windo$s $ithout any operatin! system t$ea%s" For 9inu&A it depends on the version you are usin! varyin! from '"+ G8 to a -"0 G8 heap" For 1+ bit process modelsA the ma&imum is theoretically unlimited but your performance may de!radeA dependin! on your applicationA appro&imately 0720G in a 1+ bit mode" Another su!!estion is to set ?m& and ?ms to the same value to avoid unnecessary resizin! of the heap" For more information on )*, heap sizesA please refer to http(<<Bava"sun"com<performance<reference<$hitepapers<tunin!"htmlLsection+"'"2" #ip +( #une )ava Garba!e /ollection #his option should be e&plored if you do some initial G/ analysis De!"

un ,icrosystemsA =nc

7verbose(!cA 7??(MPrintG/#ime tampsA 7??(MPrintG/6etailsF and the )*, machine is spendin! a lot of time doin! !arba!e collection" #here are also visual tools available li%e Bvmstat to connect to a )ava 1 process" 8y virtue of er!onomicsA G/ tunin! is typically not re2uired unless there is an overt G/ problem" 8y defaultA the serial collector is the default !arba!e collector and is typically used for sin!le processor machines and a small heap" .o$everA on server7class machines $ith more than one processorA parallel G/ is the default" Ensure that parallel G/ is bein! used D7??(M:seParallelG/F for multithreaded machines $hich uses multiple threads for minor collections" ,aBor collections are the same as serial collector" Another )*, tunin! does e&ist for parallel old !eneration collector D7??(M:seParallelOldG/F ho$ever this is only applicable in only certain phases of an old !eneration collection" =f short response time is more critical to your applicationA the /, collector is better suited for short G/ pauses at the e&pense of throu!hput" D7??(M:se/onc,ar% $eepG/F" 8y tradin! processor resources $hich $ould other$ise be available to the application for shorter maBor collection pause times the concurrent part of the collection is done by a sin!le !arba!e collection thread" A more thorou!h understand is necessary $hen tin%erin! $ith G/ tunin! options and it is stron!ly recommended to read I#unin! Garba!e /ollection $ith the )ava *irtual ,achineJ for more details and G/ strate!ies" #ip 0( .##P acceptor threads .##P acceptor threads accept ne$ incomin! connections and schedule ne$ re2uests for the e&istin! connections" #he default number of acceptor threads is one" =t is recommended to have ' thread per '7+ coreA althou!h e&perimentation may be necessary to find the optimal number" #ip 1( .##P re2uest processin! threads #his pool of threads retrieve and process incomin! .##P re2uests" #he default number of re2uest processin! threads is 0 but a startin! rule of thumb is to tune the number of .##P re2uest processin! threads to the number of /P:s on the :#" =f you application is =<O boundA you can start $ith double the number of /P:s" =ncrease this number of threads until your throu!hput starts to decline" At the point $hen your throu!hput starts to sufferA the re2uest processin! threads are startin! to contend for /P: resources so some e&perimentation $ill be necessary to find the s$eet spot" #ip 3( 4eep Alive subsystem #his subsystem prevents the server from bein! overloaded $ith connections" A $aitin! %eep alive connection has completed processin! the previous re2uestA and is $aitin! for a ne$ re2uest to arrive on the same connection" #hree factors describe ho$ the 4eep7Alive behavior" #he first factorA the Keep-Alive timeoutA

un ,icrosystemsA =nc

defines ho$ lon! a connection should be maintained since its last activity" #he default value is -0 seconds and it may be increased if the typical time bet$een client interactions is !reater than -0 seconds to avoid the e&pense of creatin! a ne$ connection for a ne$ re2uest" .o$everA you do not $ant to set a very hi!h value since the cost to the server is maintainin! too many open connections" #he second factorA the Keep-Alive max connectionsA define the ma&imum number of re2uests that $ould be handled on a connection before the connection is closed by the server" ettin! this value to 7' $ill disable this feature" A!ainA some e&perimentation may be necessary" #he third parameter is the thread-countA $hich acts much li%e the .##P acceptor thread" A first rule of thumb is to %eep one for every 5 cores in your system" #ip 5( .##P File /ache =f your application contains static filesA it is recommended to enable .##P file cache to optimize performance" #he .##P server cache $ill put fre2uently used static files in memory to avoid readin! the resource from the file system for every re2uest" Files are cate!orized based on user7defined size limits into smallA medium and lar!e !roups" #he contents of small files are read into the )*, heap $hile medium files are memory mapped by GlassFish" 9ar!e files Di"e" any file lar!er than the medium7file7size7limitF are not cached" AdditionallyA the re2uest has to be for a static resource that is serviced by the 6efault ervlet" Files $ith :;= mappin!s to custom servlets $ill not be cached" #he cost of usin! file cachin! is the increased memory footprint of the server instance" pecificallyA small file cachin! $ill increase !arba!e collection since they are cached in the )*, heap $hile memory7mapped medium sized files $ill increase the resident memory of the process" #he follo$in! factors should be ta%en into consideration $hile tunin! the file cache( the number and size of the various filesA the fre2uency of the file re2uestA the heap space confi!ured for the instanceA and the amount of available memory"

et the number of files to be cached based on the number of files that are commonly accessed" For most casesA the default value of '02+ $ill suffice" et the ma&imum a!e of a cache entry based on ho$ often clients access a !iven resource" #he ma&7a!e7in7seconds parameter determines the amount of time a file is retained in the cache" =t is recommended that this value be set lar!e enou!h so as to incur several hits before the file is removed from the cache" /onverselyA the value should not be so hi!h to avoid cachin! of infre2uently used files

un ,icrosystemsA =nc

$hich can impact performance ne!atively or for files that chan!e fre2uently $hich may sho$ incorrect behavior"

et the space available for the small files based on the amount of heap available" =t is recommended that this parameter be set to a value lar!e enou!h to accommodate the commonly accessed files" ettin! this value to be a lar!e proportion of the heap can cause fre2uent !arba!e collection" et the space available for medium files based on the overall available memory" ince medium files are memory mappedA the memory associated $ith this cache is e&ternal to the )*, heap" /achin! these files increase the process memory so it is important to ensure that the overall memory does not !o beyond the available process memory D+G8 for -2 bit )*,F" =f the amount of physical memory in the system is limitedA it is recommended that the space allocated for the medium sized files be reduced so that the overall process memory is lo$er than the available physical memory"

#ip 9( 6efault7$eb"&ml #he default7$eb"&ml file defines features such as filters and security constraints that apply to all $eb applications" #he parameterA development=trueA Dthe default value for developer profileF enables chan!es made to ) P#, 7 code to be instantly visible to the clients" .o$everA there is a cost associated $ith this" #o avoid the cost of chec%in! $hether the ) P code has been modified and hence its recompilationA the first parameterA development=falseA can be used to set development to false since this scenario is unli%ely in a production system" #his chec% affects application scalability $hen multiple users re2uest the same ) P class" #he second parameterA genStrAsCharArray=trueA chan!es the $ay the ) Ps are !enerated by !eneratin! char arrays for every static strin!s in the ) P class li%e for e&ampleA the .#,9 ta!s" 8y defaultA the ) Pcode $riter must call the to/harArrayDF on every trin! on every invocation of the ) Pclass"

#able '" Performance optimization in default7$eb"&ml" <init-param> <param-name>development</param-name> <param-value>false</param-value> </init-param> <init-param> <param-name>genStrAsCharArray</param-name> <param-value>true</param-value> </init-param>

un ,icrosystemsA =nc

#ip '0( )68/ #unin! =f your application uses )ava 6ata8ase /onnectivity DN)68/#,NF soft$are for database accessA it may be beneficial to tune your database connection pool" #une the steady7pool7size and the ma&7pool7sizeA and set them to the same value" #his $ill avoid unnecessary resizin! of the pool durin! the test" A !eneral rule of thumb is to tune the value for ma&7pool7size and steady7pool7size to the same number of .##P re2uest processin! threads" =f your )68/ driver supports this featureA it is advisable to use )68/ drivers that use statement cachin! to re7use prepared statements" /hec% $ith your database vendor on ho$ to do so" For OracleA you can add the follo$in! properties to your )68/ connection pool(
#able 2" Oracle database properties for statement cachin! <property name="ImplicitCachingEnabled" value="true"/> <property name="MaxStatements" value="200"/>

For ,y O9 databases(
#able -" ,y O9 database properties for statement cachin! <property name="cachePrepStmts" value="true"/> <property name="prepStmtCacheSize" value="512"/> <property name="useServerPreparedStmts" value="false" />

For 682 databases(

#able +" 682 database properties for statement cachin! <property name="MaxPooledStatements" value="200"/>

#ip ''( 6isable Access 9o!!in! Disable access logging( #o avoid unnecessary =<O activityA disable the access lo!!in!" pecific #unin! for /ool#hreads#, #echnolo!y on ystems $ith :ltra PA;/; #'<#2 Processors For /ool#hreads technolo!y on systems $ith :ltra PA;/ #'<#2 processorsA there are specific tunin! parameters that can be applied to ta%e advanta!e of performance"

un ,icrosystemsA =nc

Ensure that the )64#, version is at least )ava

E platform '"0"0P01" E platform

7??(9ar!ePa!e ize=n8ytesQ201m Dre2uires )ava

'"0"0P01 and olaris '0 .W2F

7?m&2010m 7?ms2010m Din !eneralA it is optimal to !ive the

application server as much memory as possible $ithin a -27bit address space" When usin! the 9ar!ePa!e ize=n8ytes fla!A the heap size must be a multiple of that pa!e size" =f you must use less memoryA ensure that it is a multiple of 201mF"
??(ParallelG/#hreadsQ+" #his is a !ood startin! point but may be

needed to be tuned accordin! to G/ activity in your application and the number of threads available"

eparate the access lo!sA transaction lo!s and im2 files onto as many dis%s as possible

:se libumem :se F?

chedulin! class

=f you are usin! more than one net$or% interfaceA then ma%e sure that

all the net$or% interrupts are not !oin! to the same core"

Consult the Appendix I for more details.

/ase tudy
#o illustrate the effect of tunin! the GlassFish application server effectivelyA an e7 commerce type $eb application $as deployed on the app server and subBected to a load $ith varyin! tunin!s applied" .o$everA before presentin! the dataA it is a !ood opportunity to restate the caveat emptor(

The best tuning tip is the one that improves YOUR application's performance. Your mileage may vary.
Rou may not observe the same improvement in performance due to the hard$areA O A )64 version and of courseA the benchmar% application" #able = summarizes the tunin! applied to the $eb application" /onfi!uration A is the default tunin! applied to the GlassFish application server $ith the )*, option 7client and 7?m& 0'2m" /onfi!uration 8 %eep the same heap size $ith only the )*, mode set to 7server instead of 7client" /onfi!uration / increases the heap size to '"0 G8 and /onfi!uration 6 applies .##P re2uest processorA .##P 4eep Alive tunin!A .##P Acceptor #hreadA )68/ cachin! and default7$eb"&ml tunin!s"

'0

un ,icrosystemsA =nc

#able 0" ummary of #unin!s Applied for /onfi!uration AA 8A / and 6"

/onfi!uration

6escription 6efault GlassFish tunin!" )*, options( 7clientA ?m& 0'2m )*, options( 7serverA ?m&0'2m )*, options( 7serverA ?m&'000mA '000m )*, options( 7serverA ?m&'000mA '000m

8 /

.##P Acceptor #hreadsA .##P ;e2uest Processin!A .##P 4eepAliveA )68/ cachin!A default7$eb"&ml )*, options( 7serverA ?m&'000mA ?ms'000m .##P ;e2uestProcessin!A .##P 4eepAlive tunin!A )68/ tunin!

#he results obtained $ere from a sin!le instance of the GlassFish application server demonstrate a mar%ed improvement from tunin! many of the parameters outlined in the list described above" Please e&ercise caution in interpretin! these resultsH the improvement sho$n is case specific so your results may not follo$ a similar trend for each tunin! applied" =n some casesA no improvement or a de!radation may even result"

''

un ,icrosystemsA =nc

Effect of #unin! Parameter on #hrou!hput

'200

#hrou!hput transactions<sec

'000 500 100 +00 200 0

A( 7client 8( 7server /( 7server 7?m&'000m 7?ms'000m 6( E( Full DM)67

/onfi!uration

Fi!ure 2(

ummary of Effect of #unin! /onfi!uration on #hrou!hput"

/onclusion
#his $hite paper does not attempt to cover all of the possible tunin! parameters in the GlassFish application serverH instead it provides an initial !uidance on certain parameter for e&perimentin! durin! benchmar%in!" FirstlyA it is imperative to use the correct application for benchmar%in!" #he e&periments should be performed in a controlled environment $ith a clear understandin! of $hat the %ey metrics are crucial for your business scenarioDsF" Each tunin! parameter $ill need to be tested $ith trial and error in an iterative cycle $ith system data collected for each e&periment" Follo$in! !ood methodolo!yA these ten tips represent your first ten steps to$ards obtainin! the best performance on the GlassFish application server"

;eferences
1. GlassFish Performance Tuning Guide 2. GlassFish Application Deployment Guide . De!eloping and Tuning Applications on "ltra#PA$C T1 Chip %ultithreading #ystems &. http'(()a!a.sun.com(performance(reference(*hitepapers(tuning.htm l

'2

;eferences

un ,icrosystemsA =nc

+. http'(()a!a.sun.com(performance(reference(*hitepapers(+.,-perfor mance.html .. GlassFish Administration Guide /. http'((fa0an.sunsource.net( 1. Tuning Gar0age Collection *ith the +., 2a!a3tm4 5irtual %achine 6. 7rgonomics in the +., 2a!a3tm4 5irtual %achine

Ac%no$led!ments
%adhu 8onda9 #cott :a;s9 <inu 2ohn9 #u!een =adipalli and Deep #ingh for re!ie*ing this *hite paper and the 2a!a Performance Team for the contri0ution to the content.

'-

;eferences

un ,icrosystemsA =nc

Appendi& =
@ote > A third option is available for most of these tips althou!h it is not a recommended" Rou can edit the domain"&ml directly via your favorite te&t editor found in SG9A F= .P.O,E<domains<domain'<confi! $here SG9A F= .P.O,E is the directory $here you installed GlassFish" Proceed $ith caution as GlassFish may not start if your domain"&ml is malformed"

#ip ' 7 /han!in! )ava version #o chan!e )ava versionA edit the SG9A F= .P.O,E<confi!<asenv"conf" /han!e the property called IA P)A*AJ to point to the desired )ava version" #ips 27+ 7 Editin!<Addin! )*, options #o apply )*, optionsA you can use the follo$in! options( Option '( via Administration console( '" :se the $eb bro$ser :;9( http(TyourhostnameU(+5+5 Dthe default admin portF 2" 9o!in $ith administrator user and pass$ord Ddefault admin<adminadminF -" /lic% on Application erver node on the left hand sideA )*, settin!s tab on the ri!ht hand sideA then )*, options" +" Edit the desired )*, option Dor add @e$F in the te&tbo&" 0" /lic% ave on the ri!ht hand side

'+

Appendi& =

un ,icrosystemsA =nc

1" ;estart GlassFish"

Option 2( asadmin /9= command Rou can also use the /9= asadmin command to listA delete or create ne$ )*, option( E&amples( asadmin create-jvm-options --user admin "-Xmx1024m:" asadmin delete-jvm-options --user admin "-client:" asadmin create-jvm-options --user admin "-server:" #ip 0 7 .##P Acceptor #hreads Option ' 7 via Administration console( '" 9o!in at the administration :;9( http(<<TyourhostnameU(+5+5 2" E&pand the node on the left hand side by clic%in! on /onfi!uration 77U .##P ervice 77U .##P 9isteners -" /lic% on http7listener7'" +" Edit the IAcceptor #hreadsJ field under the Advanced settin!"

'0

Appendi& =

un ,icrosystemsA =nc

Option 2( via asadmin /9=

use of the asadmin list<!et<set for .##P properties E&ample( asadmin get server.http-service.http-listener.httplistener-1.acceptor-threads asadmin set server.http-service.http-listener.httplistener-1.acceptor-threads = <number of threads> #ip 1 7 .##P ;e2uest Processin! #hreads Option ' 7 via Administration console( '" 9o!in at the administration :;9( http(<<TyourhostnameU(+5+5 2" E&pand the node on the left hand side by clic%in! on /onfi!uration 77U .##P ervice -" /lic% on ;e2uestProcessin! #ab on ri!ht hand side" +" Edit the te&t bo& beside #hread /ount" 0" /lic% ave and restart"

'1

Appendi& =

un ,icrosystemsA =nc

Option 2( via asadmin /9= use of the asadmin list<!et<set for .##P properties E&ample( asadmin get server.http-service.request-processing.threadcount asadmin set server.http-service.request-processing.threadcount = <number of threads>

#ip 3 > 4eep Alive settin!s Option ' 7 via Administration console( '" 9o!in at the administration :;9( http(<<TyourhostnameU(+5+5 2" E&pand the node on the left hand side by clic%in! on /onfi!uration 77U .##P ervice -" /lic% on 4eepAlive #ab on ri!ht hand side" +" Edit #hread /ount te&t bo&" 0" /lic% ave and ;estart"

'3

Appendi& =

un ,icrosystemsA =nc

Option 2( via asadmin /9= use of the asadmin list(get(set for >TTP properties E&ample( asadmin get server.http-service.keep-alive.thread-count asadmin set server.http-service.keep-alive.thread-count = <number of threads> #ip 5 > .##P File /ache #he first optionA enablin! file cache !lobally refers to small file cache D)*, heapF $hile the last optionA IFile /ache EnabledJ is used to enable medium file cache Dm7mapped filesF" Option ' 7 via Administration console( '" 9o!in at the administration :;9( http(<<TyourhostnameU(+5+5 2" E&pand the node on the left hand side by clic%in! on /onfi!uration 77U .##P ervice -" /lic% on .##P File /ache #ab on ri!ht hand side" +" /lic% on the IenabledJ chec% bo&" 0" /lic% ave and restart"

'5

Appendi& =

un ,icrosystemsA =nc

Option 2( via asadmin /9= use of the asadmin list<!et<set for .##P properties E&ample( mall File cachin!( asadmin get server.http-service.http-file-cache.filecaching-enabled asadmin set server.http-service.http-file-cache.filecaching-enabled = true ,edium File cachin!( asadmin get server.http-service.http-file-cache.globallyenabled asadmin set server.http-service.http-file-cache.globallyenabled = true Other properties( server.http-service.http-file-cache.file-transmissionenabled (Windows only) server.http-service.http-file-cache.hash-init-size server.http-service.http-file-cache.max-age-in-seconds server.http-service.http-file-cache.max-files-count server.http-service.http-file-cache.medium-file-size-limitin-bytes

'9

Appendi& =

un ,icrosystemsA =nc

server.http-service.http-file-cache.medium-file-space-inbytes server.http-service.http-file-cache.small-file-size-limitin-bytes server.http-service.http-file-cache.small-file-space-inbytes #ip 9 > Edit the default7$eb"&ml With your favorite te&t editor A edit the default7$eb"&ml found in the directory SG9A F= .P.O,E<domains<domain'<confi!" #ip '0 > )68/ #unin! Option ' 7 via Administration console( '" 9o!in at the administration :;9( http(<<TyourhostnameU(+5+5 2" E&pand the node on the left hand side by clic%in! on ;esource77U )68/ 77U /onnection Pools -" /lic% on Tmy/onnectionPool@ameU +" /lic% on Additional Properties on ri!ht hand side" 0" /lic% on IAdd PropertiesJ button 1" Add the necessary properties"

Option 2( via asadmin /9=

20

Appendi& =

un ,icrosystemsA =nc

use of the asadmin list<!et<set for .##P properties E&ample( asadmin set server.resources.jdbc-connectionpool.<mypoolname>.property.maxStatements=200 #ip '' > 6isable Access 9o!!in! Option ' 7 via Administration console( '" 9o!in at the administration :;9( http(<<TyourhostnameU(+5+5 2" E&pand the node on the left hand side by clic%in! on /onfi!uration 77U .##P ervice" -" /lic% on .##P ervice on ri!ht hand side"

+" :nchec% the bo& for IAccess 9o!!in!J" Option 2( via asadmin /9=

use of the asadmin list<!et<set for .##P properties E&ample( asadmin get server.httpservice.property.accessLoggingEnabled asadmin set server.httpservice.property.accessLoggingEnabled = false #ip for /ool#hreads :ltra PA;/ #'<#2 #unin!

2'

Appendi& =

un ,icrosystemsA =nc

;otatin! access lo! eparate the access lo!sA transaction lo!s and im2 files onto as many dis%s as possible" =f you have t$o dis%sA put the ?A transaction lo!s DinstancePdirectory<lo!s<t&F on one dis% and the im2 files DinstancePdirectory<im2< instances<im2bro%er<fs-00F on another" =f you have more dis%sA put the im2bro%er t&n file on a separate dis%" =f you run multiple instances of the application serverA move the lo!s for each instance onto separate dis%s as much as possible" ,ount the dis% $ith these options( nolo!!in!AdirectioAnoatime" =f the dis%s become a bottlenec%A consider usin! an e&ternal dis% $ith $rite cache" :se libumem #o confi!ure the GlassFish application serverA see belo$( For sh( LD_PRELOAD=/usr/lib/libumem.so asadmin start-domain [arguments] For csh( env LD_PRELOAD=/usr/lib/libumem.so asadmin start-domain [arguments] :se of F? schedulin! class On systems $ith :ltra PA;/ #' processorsA it is recommended to put the application server into the F? schedulin! class" #his standard feature of the olaris O is relevant on this platform because the default time 2uanta for schedulin! doesnCt match $ith the time 2uanta for processin! a $eb re2uest" 8y puttin! the application server in F? classA a simple $eb re2uest is more li%ely to be processed $ithin a sin!le dispatch as it $ill not be preempted by a different threadA causin! an increase in throu!hput" '"Find the process =6 of the GlassFish process( ps -ef | grep appserv 2"Find the process =6 of the ,O 8ro%er daemon( ps -ef | grep mqbrokerd -"For each process =6A e&ecute( /usr/bin/priocntl -s -c FX -m 59 -p 59 -i <process id of appserver> @et$or% /onfi!uration =f you are usin! more than one net$or% interfaceA then you $ill need to ma%e sure that all the net$or% interrupts are not !oin! to the same core" :se the follo$in! script to enable interrupts on one strand and disable interrupts on the remainin! three strands of a core" Rou can verify the processor state chan!e $ith the prsinfo command" E& ( psrinfo output before runnin! the script

22

Appendi& =

un ,icrosystemsA =nc

# psrinfo | more 0 on-line 1 on-line 2 on-line 3 on-line """""""""""""""""

since since since since

01/10/2008 01/13/2008 01/13/2008 01/13/2008

15:06:01 13:50:00 13:50:00 13:50:00

psrinfo output after runnin! the script 0 on-line since 01/10/2008 1 no-intr since 01/13/2008 2 no-intr since 01/13/2008 3 no-intr since 01/13/2008
#able 1" @et$or% interrupt script(

15:06:01 13:55:16 13:55:16 13:55:16

allpsr=`/usr/sbin/psrinfo | grep -v off-line | awk '{ print $1 }'` set $allpsr numpsr=$# while [ $numpsr -gt 0 ]; do shift numpsr=`expr $numpsr - 1` tmp=1 while [ $tmp -ne 4 ]; do /usr/sbin/psradm -i $1 shift numpsr=`expr $numpsr - 1` tmp=`expr $tmp + 1` done done

/ontact ales At( !lassfishVsun"com W 2009 un ,icrosystemsA =nc" All ri!hts reserved" unA un ,icrosystemsA the un lo!o /ool#hreadsA GlassFishA )*,A )avaA ) PA )68/A and )64 are trademar%s or re!istered trademar%s of un ,icrosystemsA =nc" or its subsidiaries in the :nited tates and other countries"