PIA Load Balancing, JVMs and

Garbage Collection
David Kurtz
Go-Faster Consultancy Ltd.

david.kurtz@go-faster.co.uk
www.go-faster.co.uk
 Who am I?
• DBA • Book
– Independent consultant
– www.psftdba.com
• System Performance
tuning
– PeopleSoft ERP
– Oracle RDBMS
• Training Course
• UK Oracle User Group
– Unix SIG
• Oak Table
– www.oaktable.net
PSFT Tech PUG 5.9.05 www.go-faster.co.uk 2
 Resources

• If you can’t hear me say so now.

• Please feel free to ask questions as we go
along.
• The presentation will be available from
– www.go-faster.co.uk

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 3

 A War Story
• Two forms of load
balancing WEB1
– Windows Network PIA1
Load Balancing IIS 50.0.0.7:7001

Load Balancing
Windows Network
50.0.0.7:443
PIA2
– IIS Proxy 50.0.0.7:17001

50.0.0.50:443

WEB2
PIA1
• PSFT Recommendations: IIS 50.0.0.8:7001

– Red Paper HA & Clustering

50.0.0.8:443
PIA2
50.0.0.8:17001

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 4

 Problems

• Strange Pauses
– Mostly to do with garbage collection
• IIS Proxy wouldn’t load balance
– All of load on one PIA or the other
• Pauses cause IIS Proxy failover
• On Failover users loose their sessions

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 5

 IIS Load Balancing

• iisproxy.ini

WebLogicCluster=50.0.0.8:7001,50.0.0.8:17001
DynamicServerList=OFF
…

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 6

 IIS Load Balancing

• Uneven loading of PIAs caused overload of

one PIA.
• Duration of garbage collection with 100%
of load on single PIA much higher

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 7

 How do I know the load is
balanced?

• Web server access log

• PeopleSoft Performance Monitor
• PT>=8.44
– Event 150: JVM Status
– Event 152: Web Site Status
– Event 153: Web Servlet Status

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 8

 How do I know the load is
balanced?
PIA Activity

hrmsweb001 17001:17002 hrmsweb001 7001:7002 hrmsweb002 17001:17002 hrmsweb002 7001:7002

300

250

200
Session in Web-App

150

100

50

0
Wed 29.06.05 12:00 Wed 29.06.05 13:00 Wed 29.06.05 14:00 Wed 29.06.05 15:00

Time

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 9

 JVM Garbage Collection

• Java Command Line Options

– verbosegc
• Writes to weblogic.log
– Xloggc:<filename>

0.000: [GC 46591K->3553K(518464K), 0.1262603 secs]

19.604: [GC 50145K->6662K(518464K), 0.3389070 secs]
70.052: [Full_GC 46565K->10901K(518464K), 0.9507536 secs]
90.967: [Full_GC 55273K->16571K(518464K), 1.1008306 secs]

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 10

 Garbage Collector Log
90.967: [Full_GC 55273K->16571K(518464K), 1.1008306 secs]

– Elapsed time since JVM started

– Collection type
– Size of live objects before and after GC
– Total space available, less permanent objects
– Duration of collection

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 11

 Timestamps on GC log

-XX:+PrintGCTimeStamps

• But I couldn’t get it to work!

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 12

 Other Garbage Collection
Options

• -XX:+UseConcMarkSweepGC
• -XX:+UseParNewGC
• -XX:+CMSParallelRemarkEnabled

• Caveat: These options are not presented as being any kind of panacea, but
something to consider. You need to do your own testing.
• BTW: I got the idea for this from a UKOUG Oracle Application Server SIG!

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 13

 -XX:+UseConcMarkSweepGC
• Concurrent Low Pause Collector

• GC is, as far as possible, concurrent with the

execution of the application.
– Uses a separate thread
• App is paused for shorter periods during GC
• Only effective on multi-processor machines

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 14

 -XX:+UseParNewGC

• Parallel young generation GC with

concurrent GC
– for multiprocessor machines,
– enables multi threaded young generation
collection
– Application is not paused

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 15

 -XX:+CMSParallelRemarkEnabled

• Reduces remark pauses

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 16

 Default Garbage Collector

• Individual GC duration
GC Duration

DURATION

10

1
ÿ 5— È—³ \—G ð—Û k—… 120000
Du ration (s)

0.1

0.01

0.001
Offset Seconds

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 17

 Parallel Garbage Collector

• Individual GC duration
GC Duration

DURATION

10

1
0 í ³… L ä ¶ h d 120000
Duration (s)

0.1

0.01

0.001
Offset Seconds

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 18

 Default Garbage Collector

• Cumulative GC duration / minute

GC Duration

DURATION

10
GC Duration seconds/minute

1
0 øÑÊ —Ñi 7Ñ ÖѨ aÑ] 120000

0.1

0.01
Offset Seconds

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 19

 Parallel Garbage Collection

• Cumulative GC duration / minute

GC Duration

DURATION

10

1
0 í ³… L ä ¶ h d 120000
GC Duration seconds/minute

0.1

0.01

0.001
Offset Seconds

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 20

 Conclusion

• Improved GC performance does not have a

great effect on Component Performance
• It does reduce the ‘strange pause’ giving the
user a better experience
• Parallel GC seems to slowly leak memory.
– Possible bug in a particular JVM
– I have only tested on Windows

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 21

 Questions?

PSFT Tech PUG 5.9.05 www.go-faster.co.uk 22

PIA Load Balancing, JVMs and
Garbage Collection
David Kurtz
Go-Faster Consultancy Ltd.

david.kurtz@go-faster.co.uk
www.go-faster.co.uk