Bea Internals

PeopleTools Application Server &
BEA Tuxedo Internals

david.kurtz@go-faster.co.uk
Go-Faster Consultancy Ltd.
http://www.go-faster.co.uk
Who am I?
 Independent Consultant
 Performance Tuning
 PeopleSoft / Oracle
23 January 2002
2
Rules of Engagement
 Interrupt me:
— if you think I have got something wrong
— if you have a question
— if you can’t hear me
23 January 2002
3
BEA Tuxedo
 Historical Overview
 Tuxedo Internal Architecture
 PeopleSoft 3-tier client implementation
 How does PeopleSoft Internet Architecture (PIA) fit in
 Configuration issues
 Monitoring
23 January 2002
4
Brief Historical Background
 In the beginning
— monolithic
Business Data
Presentation Logic Access
(DBMS)
23 January 2002
5
Client Server
 Two-tier client-server
SQL Data
Presentation Business Raw Data
Logic Access
(DBMS)
LAN
Client Server
23 January 2002
6
3-Tier Client Server
 “1st Generation”
Request Data
GUI Access
+ Response (DBMS)
Application
Logic LAN Business
Logic
Client Server
23 January 2002
7
3-Tier Client Server
 “2nd Generation” - Tuxedo
Presentation Business Logic Data Management
Service 1
Service 2
...
TUXEDO Service n
SQL
Messaging
Presentation
Logic DBMS 1
...
DBMS n
TUXEDO Clients TUXEDO Servers Database Servers

23 January 2002
8
Where did Tuxedo come from?
 Bell Labs - 1993
 Transactions for
 UniX
 Extended for
 Distributed
 Operations
23 January 2002
9
What is Tuxedo?
 It is the knots on the ends of the string!
23 January 2002
10
PeopleSoft 2-Tier Client
Panel
Load
Panel
Presentation Save
Logic Field
Change
etc...
23 January 2002
11
Panel
Load
Tuxedo Server
Tuxedo Client Panel
Presentation Save
Logic Field
Change
etc...
Client Side Server Side
23 January 2002
12
 Client Connects Directly to the Database

— 1 database connection per window
— inactivity timeout
DATABASE
23 January 2002
13
 Client Connects Directly to the Database

— 1 database connection per window
— inactivity timeout
 What happens in 3-Tier?
DATABASE
23 January 2002
14
PS/Tuxedo Internal Structure
 1st Application Server Process Started

— Bulletin Board Liaison (BBL)
— Reads PSTUXCFG configuration file
 Shared Memory Segment

— Bulletin Board (BB)
BB PSTUXCFG DATABASE
BBL 23 January 2002

15
 Workstation Listener (WSL)

— Listens for connection request from clients on a specified port
— Client initially contacts the WSL
WSL
BB
DATABASE
BBL 23 January 2002

16
 Workstation Listener (WSL)

— Client needs to know where to find the WSL
— Machine and port specified in Configuration Manager
WSL
BB
BBL 23 January 2002

17
 Workstation Handler (WSH)

— handles traffic from client after connection
— Client is assigned to a port on the WSH
— WSH port range can be constrained to match a firewall
— Additional WSH can be spawned if necessary
— Compress Messages
WSL WSH
BB
DATABASE
BBL 23 January 2002

18
PSAUTH
AUTHQ
APPQ PSAPPSRV
QCKQ PSQCKSRV
QRYQ PSQRYSRV
APIQ PSAPISRV
SAMQ PSSAMSRV
WSL WSH
BB
DATABASE
BBL 23 January 2002

19
How does it work?
PSAUTH
AUTHQ
APPQ PSAPPSRV
QCKQ PSQCKSRV
QRYQ PSQRYSRV
APIQ PSAPISRV
WSHQ
SAMQ PSSAMSRV
WSL WSH
BB
DATABASE
BBL 23 January 2002

20
How does PIA work?
 Java Client & PIA use Jolt

— Java Class Library
— Jolt Listener maps Java functions to Tuxedo Messages
23 January 2002
21
Jolt Internal Structure
PSAUTH
AUTHQ
APPQ PSAPPSRV
QCKQ PSQCKSRV
QRYQ PSQRYSRV
JSHQ
APIQ PSAPISRV
JSL JSH JREPSRV

SAMQ PSSAMSRV
WSL WSH
JREPOSITORY
BB
DATABASE
BBL 23 January 2002

22
New structure for PIA (Apache Webserver)
Sessionid =
“xy1k…vm8010
http 1.1
address+cookie
(stateless) Cookie
Java
JVM
Servlet Engine
Servlet Container Servlet
thread
Servlet IS stateful
Webserver
Servlet
Thread
Servlet
HTTPD Client Thread
Client Servlet
Thread
mod_jserv Port 8010 Tuxedo

Disk Application
(static JSL JSH Server
JSH
objects) JSH
23 January 2002
23
What is the difference between the web servers?
 Apache
— Separate Servlet Engine
— Separate encryption module (Raven)
 Weblogic
— Contains an integral servlet container
23 January 2002
24
Progressive Connection Concentration
 Typical picture
 35,000 users 35000 users
 1000 concurrent connections

— 1000 users active in the last n minutes
— where n is the Java + JSH timeout 1000 connections
— therefore need 1000 threads 4 JVMs
1000 Java Threads
— but maybe only 4 JVMs
 100 JSH 100 JSHs

— assumes 10 clients per JSH
10-20 PSAPPSRVs
 10-20 PSAPPSRV processes
— each PSAPPSRV connects to database 1 database
 1 database
23 January 2002
25
Configuration
psadmin
psappsrv.val (configuration
dialogue)
psappsrv.cfg ubbgen psappsrv.ubx

(PS Utility)
tmloadcf
psappsrv.ubb (tuxedo
process)
BBL
PSTUXCFG (domain
startup)
23 January 2002
26
Configuration Files
 psappsrv.cfg
— contains configurable parameters
 psappsrv.val
— validation file for configurable values
 psappsrv.ubx
— configuration template for Tuxedo domain
 psappsrv.ubb
— configuration file for Tuxedo domain
 PSTUXCFG
— compiled configuration file
— read by BBL at domain startup
23 January 2002
27
psappsrv.ubx -v- psappsrv.ubb
 psappsrv.ubx & psappsrv.cfg  psappsrv.ubb

— PeopleSoft file — Tuxedo file
— make changes to these files — never make changes to this file
— difficult to read — easier to read
— this file is compiled by tmloadcf to
generate domain configuration file
PSTUXCFG
— variables fully resolved
23 January 2002
28
psappsrv.ubb
 Five (or Six) Sections

— Resources
– resource limits -> BB sizing
— Machine
– Resource limits. Environment
— (Network
– Connection between Machines)
— Groups
– Logical Split (BASE, APPSRV, JREPGRP, JSLGRP)
— Servers
– Number of servers, Command line
— Services
– Timeout, Load, Priority
23 January 2002
29
Application Server Configuration Issue
 Move quick PSAPPSRV services into a second server (PSQCKSRV) (y/n)?

— <=PT7.58 this question is misleading
– services are additionally advertised on PSQCKSRV as well as PSAPPSRV
– so they are advertised on more than one queue
— >=PT7.59 and PT8.1
– services are de-advertised from PSAPPSRV
– quick services not used in PIA, only IC* services
– so not a problem
23 January 2002
30
PeopleTools <=7.58
 Quick Service
APPQ PSAPPSRV
QCKQ PSQCKSRV
WSL WSH
BB
DATABASE
BBL 23 January 2002

31
Inside the Bulletin Board
 Without Load Balancing (default)

— Table of servers process where each
service is advertised
— Read and populated as a stack
— WSL looks for first free server where
PSAPPSRV
request can be enqueued
— This can cause problems
PSQCKSRV
PSQCKSRV
PSQCKSRV
PSAPPSRV
PSAPPSRV
PSAPPSRV
23 January 2002
32
Load Balancing
 Load Balancing is about choosing the queue upon which to place the
message
— by default all requests have the same load
50 50 50
APPQ PSAPPSRV
50
50 50 50 50 50 50
QCKQ PSQCKSRV
23 January 2002
33
Load Balancing Trace
 Add -r to application server command line

— CLOPT="-r -e APPQ.stderr -A -- -C psappsrv.cfg -D GP81O81 -S PSAPPSRV”
— -r enables trace to file ‘stderr’ (not channel 2)

— -e overrides the output filename
— process trace file with Tuxedo utility ‘txrpt’

— obtain average service time
— make load proportional to average service time
23 January 2002
34
Load Balancing Trace
— Output from TXRPT
SVCNAME 18p-19p TOTALS

Num/Avg Num/Avg
--------------- -------- -------
SqlRequest 16/0.10 16/0.10
MgrClear 5/0.04 5/0.04
RamList 1/0.43 1/0.43
SamGetParmsSvc 1/0.50 1/0.50
--------------- ------- -------
TOTALS 23/0.12 23/0.12
23 January 2002
35
Load Balancing
 load = execution time (seconds) x 100
78 386 135
APPQ PSAPPSRV
599
50
6 30 3 63 3 7
QCKQ PSQCKSRV
116
23 January 2002
36
Multiple Queues on Busy System
 You can have many server processes on a single queue

 BEA recommend no more that 10 server processes per queue
 If you have queue contention it is likely to impede the WSH/JSH processes
enqueuing requests. You are unlikely to observe queuing with PQ.
 So, if you want more than 10 PSAPPSRV processes you should use two
APPQ queues
 In, which case you should to enable load balancing and you must calculate
the load.
— Less significant with PIA because services different.
23 January 2002
37
APPQ1 PSAPPSRV
APPQ2 PSAPPSRV
WSL WSH
BB
DATABASE
BBL 23 January 2002

38
 psadmin will not correctly calculate

— MAXACCESSERS
— MAXSERVICES
 tmadmin->bbs
— will report maximum number of services
– count of all services advertised on each server process
 and if you have multiple work station listeners

— MAXWSCLIENTS
23 January 2002
39
Service Priority
 Priority is about queue jumping

— servers de-queue the request
— 9 out of 10 de-queued on the basis of priority
— 1 out 10 is taken from the front of the queue
80 50 50 50 50 50
QCKQ PSQCKSRV
23 January 2002
40
Service Priority
 All PeopleSoft Services have same priority

— specified in SERVICES section in psappsrv.ubb
— therefore there is no queue jumping
— no reason to believe that it would be advantageous
23 January 2002
41
Tuning Tip
 Financials
— On-line journal edit & post - remote call
— Service initiates a Cobol process
— Process can take minutes
— RemoteCall services can block APPQ
— Move RemoteCall service to a separate queue
23 January 2002
42
Multi-Node Domain
 1 domain on many nodes
DBBL
WSL WSH BB APPQ PSAPPSRV
BBL
BRIDGE TLISTEN
DATABASE
BRIDGE TLISTEN
BBL
WSL WSH BB APPQ PSAPPSRV
23 January 2002
43
Multi-Node Domain
 Distinct Bulletin Board Liaison process

— Master Node
 TLISTEN process
— Listens for incoming requests
 BRIDGE
— Sends & Receives messages between nodes
— in a load balancing environment
– configurable bridge load add to remote queues
– limiting number of messages sent across the bridge
23 January 2002
44
Multi-Node Domain
 DO ATTEMPT THIS WITHOUT BEA CONSULTANCY
 At UBS
— PT 7.5x
— susceptible to network errors
— integrated into HP service guard

— ultimately this was removed from production because of transient network errors
between the nodes causing the domain to ‘partition’
23 January 2002
45
So what is going on?
 tmadmin
— Tuxedo command line interface
— also available within psadmin
23 January 2002
46
tmadmin commands
 pq - printqueues
— amount of work queued and waiting
— not always accurate
23 January 2002
47
tmadmin commands
 psr - printservers
— reports activity of individual server processes
23 January 2002
48
tmadmin commands
 psc - printservices
— each service on each server
23 January 2002
49
tmadmin commands
 Lots of other tmadmin commands

 documented in the Tuxedo documentation
 Use them to
— dynamically reconfigure the domain
— dynamically resize the domain
– start/shutdown extra servers
— monitoring scripts
– tmadmin -r for read only mode
23 January 2002
50
Java Administrative Applet
23 January 2002
51
Conclusion
 So now you know

— BEA Tuxedo/Jolt is a messaging protocol
— How the messages are moved around
— How PeopleSoft has used it
— How the PIA has been constructed
— Options for Tuning and Sizing
— Load Balancing
— Multi-Node domains
— Monitoring Scripts
 BEA Documentation
— is on your CD
— http://e-docs.bea.com
23 January 2002
52
More Information about PT8 Application Server
 PeopleSoft 2000 Conference: Americas

— 8006: PeopleTools 8.1 Application Server
— 8010: Inside the Application Server
– Buzz Voelker
– PeopleTools Application Server Team
23 January 2002
53
Final Caveat
 PeopleSoft will not support any of the changes that I have described in this
presentation
 Everything described in this presentation has been done, in production, at

UBS
 If you want to use a multi-node domain you are strongly recommended to

arrange support/consultancy direct from BEA.
23 January 2002
54
Questions?
23 January 2002
55
PeopleTools Application Server &
david.kurtz@go-faster.co.uk
http://www.go-faster.co.uk

Bea Internals

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Bea Internals

Uploaded by

Copyright:

Available Formats

PeopleTools Application Server &

BEA Tuxedo Internals

 “2nd Generation” - Tuxedo

Presentation Business Logic Data Management

TUXEDO Clients TUXEDO Servers Database Servers

 Bell Labs - 1993

 It is the knots on the ends of the string!

Client Side Server Side

 Client Connects Directly to the Database

 Client Connects Directly to the Database

 What happens in 3-Tier?

 1st Application Server Process Started

 Shared Memory Segment

BBL 23 January 2002

 Workstation Listener (WSL)

BBL 23 January 2002

 Workstation Listener (WSL)

BBL 23 January 2002

 Workstation Handler (WSH)

BBL 23 January 2002

BBL 23 January 2002

BBL 23 January 2002

 Java Client & PIA use Jolt

JSL JSH JREPSRV

BBL 23 January 2002

mod_jserv Port 8010 Tuxedo

 1000 concurrent connections

 100 JSH 100 JSHs

psappsrv.cfg ubbgen psappsrv.ubx

 psappsrv.ubx & psappsrv.cfg  psappsrv.ubb

 Five (or Six) Sections

 Move quick PSAPPSRV services into a second server (PSQCKSRV) (y/n)?

BBL 23 January 2002

 Without Load Balancing (default)

 Add -r to application server command line

— -r enables trace to file ‘stderr’ (not channel 2)

— process trace file with Tuxedo utility ‘txrpt’

— make load proportional to average service time

— Output from TXRPT

SVCNAME 18p-19p TOTALS

 load = execution time (seconds) x 100

 You can have many server processes on a single queue

BBL 23 January 2002

 psadmin will not correctly calculate

 and if you have multiple work station listeners

 Priority is about queue jumping

 All PeopleSoft Services have same priority

— no reason to believe that it would be advantageous

 1 domain on many nodes

WSL WSH BB APPQ PSAPPSRV

WSL WSH BB APPQ PSAPPSRV

 Distinct Bulletin Board Liaison process

 DO ATTEMPT THIS WITHOUT BEA CONSULTANCY

— integrated into HP service guard

 Lots of other tmadmin commands

 So now you know

 PeopleSoft 2000 Conference: Americas

 Everything described in this presentation has been done, in production, at

 If you want to use a multi-node domain you are strongly recommended to

You might also like