Active Session History: Feel The Power

ASH – Active Session History
Feel the Power
Kyle Hailey
http://perfvision.com
#.2
ASH
Whole New Paradigm in
Technology
The Power of ASH lies in
Simplifying Performance Tuning
 Totally new and exciting methodology

 Cheaper, Quicker, Richer and better tasting too
Copyright 2006 Kyle Hailey

#.3
Why should you care?
Because ASH can Change your life …
 10g immediately Accessible

 Geeks: Via scripts in SQL
 Managers : Graphical OEM
 Stuck on Oracle 7,8,9 ?

… my apologies because 10g rocks, but …
no worries, the data is accessible for you too via scripts
http://perfvision.com/sash.php

#.4
Here’s a piece of Personal

history

#.5
This is the story of how

ASH
changed my Life
Once upon a time I was stuck
in dull tech support …
Finally I was promoted to head
DBA, thinking my life was
getting better…
#.6
But … It was then that I learned I

needed to know what the
databases were doing ALL the
time
Because lives and more importantly my job
depended on it

#.7
What could I do?!
Clearly a job for Stats Pack

Because stats pack showed me every hour
 All the Stats!

 All the top Bottlenecks
 Almost all the top SQL

#.8
So why wasn’t I feeling better?

Hmmm … but … wait
 What if my database hits a bottleneck 15

minutes after my last stats pack?!
 What about that rouge coder who writes bad
SQL … can I find his module gone mad?
 Ok I found the top SQL but … now what?
 What do I do with the top WAIT?

#.9
If lives (and my job) depended on

my databases
 Was once an hour really enough ?

 Could Stats Pack find that rogue coder?
 Could Stats Pack tell me why an SQL
bottlenecked?
 Could Stats Pack tell me the root of a wait
bottleneck?

#.10
Don’t tell anyone but …
I couldn’t
Often, even
I didn’tdecipher
understandStats Pack half
statspack’s 1000+the time
lines
#.11
Oh no … it’s 3am my manager
calls
“why is the database hanging?!”
 Who did it?
 Where did the SQL get blocked?
 What if the bottlenecked started 5 minutes
after my last Stats Pack?

#.12
In that moment of crises

Stats Pack couldn’t tell
me
…
but ASH can

#.13
The crises of my day

have left the “ASHes” of
today.
I didn’t have ASH but

You do
#.14
It’s a Revolution and it’s an

Evolution
 Oracle 6 … ie the dark ages … there was
once the Cache Buffer Hit Ratio
 Oracle 7 … turned the lights on … Wait
Events … hallelujah I can see the light
 Oracle 10g … ASH has landed
(like the wheels on the suitcases)

#.15
ASH uses Sampling
New Paradigm
 Clear Vision of the future
 Use new sampling technology
 Lets go of the need to know 100%
 Statistical approximation
 Cheaper
 Richer Multidimensional Data
 Identifies
 SQL
 Sessions
 Objects
 Machine
 User

#.16
To identify Players before ASH
 Sessions
# sessions x (# wait events + statistics)
 Example (150 x (800+200) = 150,000 )
 SQL
 v$sql
 Could be 10000s
 Takes out latches that compete with other sql executions
 Objects
 V$segstat 9i+
 Could be 1000s of objects
Expensive !
#.17
ASH – Intelligence for the new

Millennium
 Intelligently Collects Data
 It self adjusts for your needs
 More activity, more data collected
 Less activity, less data collected
 Those old methods collected everything

 Obfuscated the problem, too many statistics too late
 Costly
 Too Granular – once an hour ?! Give me a break

ASH Samples Session State
Every second
10:00:00 10:00:01 10:00:02 10:00:03 10:00:04 10:00:05
? ? ? ? ? TIME
Sessions change a lot quicker but can get the main

picture via sampling by sampling faster
#.19
Sampling is like taking

Pictures

#.20
If happens a lot or for long …

we’ll catch it, guaranteed

#.21
Session States
IO CPU Wait Idle
Latency
Work
Contention

#.22
IDLE
 Example : SQL*Net Message from Client
 10g select name
from v$event_name
where wait_class='Idle';
58 Rows
 9i or below – just have to keep track

 Statspack STATS$IDLE_EVENT

#.23
CPU
 ASH: SESSION_STATE = “ON CPU”
 ASH: wait_time > 0
10g: v$session
9i: v$session_wait
w.wait_time != 0 /* on CPU */ and

s.status='ACTIVE' /* ACTIVE */
Identifying CPU via v$session_wait is

a huge break through
#.24
IO
 ASH: SESSION_STATE=‘WAITING’
&
 10g: WAIT_CLASS=‘User I/O’

 9i:
 Look at 10g for ideas
 db file sequential read
 db file scattered read
 db file parallel read
 direct path read

#.25
WAITING
 ASH: SESSION_STATE=‘WAITING’
 ASH: WAIT_TIME <= 0
 WAIT_CLASS
1. Administrative
2. Application
3. Cluster
4. Commit
5. Concurrency
6. Configuration
7. Network
8. Other
9. Scheduler
10. System I/O
 800+ WAIT

Samples for all users
Session 1
Session 2
Session 3
Session 4
10:15:00 10:15:01 10:15:02 10:15:03 10:15:04 10:15:05 10:15:06 10:15:07

TIME
ASH Fields
SQL> v$active_session_history
#.27
Name Null? Type

SAMPLE_ID NUMBER
SAMPLE_TIME TIMESTAMP(3)
SESSION_ID NUMBER
SESSION_SERIAL# NUMBER
USER_ID NUMBER
SQL_ID VARCHAR2(13)
SQL_CHILD_NUMBER NUMBER
SQL_PLAN_HASH_VALUE NUMBER
SQL_OPCODE NUMBER
SERVICE_HASH NUMBER
SESSION_TYPE VARCHAR2(10)
SESSION_STATE VARCHAR2(7)
QC_SESSION_ID NUMBER
QC_INSTANCE_ID NUMBER
EVENT VARCHAR2(64)
EVENT_ID NUMBER
EVENT# NUMBER
SEQ# NUMBER
P1 NUMBER
P2 NUMBER
P3 NUMBER
WAIT_TIME NUMBER
TIME_WAITED NUMBER
CURRENT_OBJ# NUMBER
CURRENT_FILE# NUMBER
CURRENT_BLOCK# NUMBER
PROGRAM VARCHAR2(48)
MODULE VARCHAR2(48)
ACTION VARCHAR2(32)
CLIENT_ID Copyright 2006 Kyle Hailey VARCHAR2(64
v$active_session_history #.28
SAMPLE_ID
SAMPLE_TIME
NUMBER
TIMESTAMP(3) When
SESSION_ID NUMBER
SESSION_SERIAL# NUMBER
USER_ID NUMBER Session
SERVICE_HASH NUMBER
SESSION_TYPE VARCHAR2(10)
PROGRAM VARCHAR2(64)
MODULE VARCHAR2(48)
ACTION VARCHAR2(32)
CLIENT_ID VARCHAR2(64)
SESSION_STATE
WAIT_TIME
VARCHAR2(7)
NUMBER
State
EVENT VARCHAR2(64)
EVENT_ID
EVENT#
NUMBER
NUMBER Wait
SEQ# NUMBER
P1 NUMBER
P2 NUMBER
P3 NUMBER
WAIT_TIME NUMBER
TIME_WAITED NUMBER
CURRENT_OBJ# NUMBER
CURRENT_FILE# NUMBER
CURRENT_BLOCK# NUMBER0
SQL_ID VARCHAR2(13)
SQL_CHILD_NUMBER
SQL_PLAN_HASH_VALUE
NUMBER
NUMBER
SQL
SQL_OPCODE NUMBER
QC_SESSION_ID NUMBER
QC_INSTANCE_ID NUMBER
TIME_WAITED Copyright 2006 Kyle Hailey
NUMBER
#.29
Primary Fields of ASH
SAMPLE_TIME When
SESSION_STATE State
SESSION_ID Session
SQL_ID SQL
EVENT Wait
TIME_WAITED Duration
#.30
Amazing things YOU can do with

ASH
Consumers Resources
 Top Session • CPU
 Top User
• Waits
 Top SQL
 Top Object • Event (800*)
 Top Module.Action X • I/O
 Top Program • File
 Top Service • Block
 Top Client • Time
 Top Wait
No machine 
(32 columns in ASH)
#.31
Groupings – Top Consumer
SESSION_ID
SESSION_SERIAL# (signal SID reuse)
SESSION_TYPE (FOREGROUND,BACKGROUND)
USER_ID (SYS, SYSTEM, SCOTT etc)
SERVICE_HASH (OE,GL,HR)
MODULE.ACTION(PLSQL tagging)
CLIENT_ID (identifying users in a session pool)
PROGRAM (SQL, JDBC, Forms etc)
SQL_ID
QC_SESSION_ID - Query Coordinator
QC_INSTANCE_ID – RAC
EVENT + P1, P2, P3
CURRENT_OBJ#
CURRENT_FILE#
CURRENT_BLOCK#
#.32
Top CPU Session

Who is the rogue session ?
Select
Select
session_id,
session_id,
count(*)
count(*)
from
from
v$active_session_history
where
where
session_state=
session_state= ‘ON
‘ONCPU‘
CPU‘ and
and
SAMPLE_TIME
SAMPLE_TIME>>sysdate
sysdate––(5/(24*60))
(5/(24*60))
group
groupby
by
session_id
session_id
order
orderby
by
count(*)
count(*) desc;
desc;
#.33
Results Top CPU Session
SESSION_ID
SESSION_ID COUNT(*)
COUNT(*)
SESSION_ID COUNT(*)
---------- ----------
----------
---------- ----------
----------
265 256
265
265 264 256
256 15
257 12
264
264 271 15
1512
257
257 276 12
121
271
271 12
12
276
276 11

#.34
CPU with Bars
SESSION_ID
SESSION_ID COUNT(*)
COUNT(*) %% Bar
Bar
---------- ---------- ---------- ------------
---------- ---------- ---------- ------------
257
257 75
75 25
25 |***
|*** ||
263
263 62
62 21
21 |**
|** ||
256
256 32
32 11
11 |*
|* ||
264
264 99 33 || ||
277
277 33 1 |
1 | ||
258
258 11 00 || ||
280
280 11 00 || ||
Bar shows 10% increments

#.35
Top Waiting Session
 Top Waiting Session in last 5 minutes
Select
Select
session_id,
session_id,
count(*)
count(*)
from
from
where
where
session_state=‘WAITING’
session_state=‘WAITING’ and
and
SAMPLE_TIME
SAMPLE_TIME>> SYSDATE
SYSDATE--(5/(24*60))
(5/(24*60))
group
groupbyby
session_id
session_id
order
orderbyby
count(*)
count(*)desc;
desc;
#.36
Top Waiting Session Results

SESSION_ID
SESSION_ID COUNT(*)
COUNT(*)
----------
---------- ----------
----------
272
272 224
224
254
254 88
249
249 55
276
276 55
277
277 44
270
270 11

#.37
Top SQL from ASH
select
select
ash.SQL_ID
ash.SQL_ID, ,
sum(decode(ash.session_state,'ON
sum(decode(ash.session_state,'ONCPU',1,0))
CPU',1,0)) "CPU",
"CPU",
sum(decode(ash.session_state,'WAITING',1,0))
sum(decode(ash.session_state,'WAITING',1,0)) --
sum(decode(ash.session_state,'WAITING',
sum(decode(ash.session_state,'WAITING',decode(en.wait_class,
decode(en.wait_class,'User
'UserI/O',1,0),0))
I/O',1,0),0)) "WAIT"
"WAIT", ,
sum(decode(ash.session_state,'WAITING',decode(en.wait_class,
'UserI/O',1,0),0))
I/O',1,0),0)) "IO"
"IO", ,
sum(decode(ash.session_state,'ONCPU',1,1))
CPU',1,1)) "TOTAL"
"TOTAL"
from
fromv$active_session_history
v$active_session_historyash,
ash,
v$event_name
v$event_nameen en
where
whereSQL_ID
SQL_IDisisnot
notNULL
NULL and
anden.event#=ash.event#
en.event#=ash.event#
group
groupby
bysql_id
sql_id
order
orderby
bysum(decode(session_state,'ON
sum(decode(session_state,'ONCPU',1,1))
CPU',1,1)) desc
desc
#.38
Top SQL from ASH Results

SQL_ID
SQL_ID CPU
CPU WAITING
WAITING IO
IO TOTAL
TOTAL
-------------
------------- ----------
---------- ----------
---------- ----------
---------- ----------
----------
4c1xvq9ufwcjc
4c1xvq9ufwcjc 23386
23386 00 00 23386
23386
6wjw6rz5uvbp3
6wjw6rz5uvbp3 99
99 00 23
23 122
122
968dm8hr9qd03
968dm8hr9qd03 97
97 00 22
22 119
119
938jp5gasmrah
938jp5gasmrah 90
90 00 25
25 115
115
cv8xnv81kf582
cv8xnv81kf582 42
42 00 99 51
51
6p9bzu19v965k
6p9bzu19v965k 21
21 00 00 21
21
5zu8pxnun66bu
5zu8pxnun66bu 15
15 00 00 15
15
db2jr13nup72v
db2jr13nup72v 99 00 00 99
7ks5gnj38hghv
7ks5gnj38hghv 88 00 00 88

#.39
Top Session
select
select select
ash.session_id,
ash.session_id,
ash.session_id,
ash.session_serial#,
ash.user_id,
ash.user_id,
ash.user_id,
ash.program,
ash.program,
ash.program,
sum(decode(ash.session_state,'ON CPU',1,0))
CPU',1,0)) "CPU",
CPU',1,0)) "CPU",
"CPU",
sum(decode(ash.session_state,'WAITING',1,0)) ---
decode(en.wait_class,'User I/O',1,
decode(en.wait_class,'User 00),0),0))
I/O',1,
I/O',1, 0)) "WAITING"
),0)) "WAITING" ,, ,
"WAITING"
decode(en.wait_class,'User I/O',1,
decode(en.wait_class,'User 00),0),0))
I/O',1,
I/O',1, 0)) "IO"
),0)) "IO",,,
"IO"
sum(decode(session_state,'ON
sum(decode(session_state,'ON CPU',1,1))
CPU',1,1)) "TOTAL"
CPU',1,1)) "TOTAL"
"TOTAL"
from from v$active_session_history
fromv$active_session_history
v$active_session_history ash,
ash,
ash,
v$event_name
v$event_name
v$event_name enen en
where where
whereen.event# en.event#
en.event# = ash.event#
==ash.event#
ash.event#
group
groupby group
by by session_id,user_id,session_serial#,program
session_id,user_id,session_serial#,program
session_id,user_id,session_serial#,program
order
orderby order
by by sum(decode(session_state,'ON
sum(decode(session_state,'ON CPU',1,1))
CPU',1,1))
CPU',1,1))
#.40
Top Session Results

SESSION_ID SERIAL# USER_ID PROGRAM CPU WAITING IO
SESSION_ID SERIAL# USER_ID PROGRAM CPU WAITING IO
----------
---------- -------
------- ----------
---------- -------------------------
------------------------- -------
------- ----------
---------- ----------
----------
247 61970 1 sqlplus 11698 0 0
247 61970 1 sqlplus 11698 0 0
277
277 11 00 oracle@labsfrh903
oracle@labsfrh903 (LGWR)
(LGWR) 14
14 21
21 00
276 1 0 oracle@labsfrh903 (CKPT) 19 10 0
276 1 0 oracle@labsfrh903 (CKPT) 19 10 0
278 1 0 oracle@labsfrh903 (DBW0) 29 0 0
278 1 0 oracle@labsfrh903 (DBW0) 29 0 0
280
280 11 00 oracle@labsfrh903
oracle@labsfrh903 (PMON)
(PMON) 19
19 00 00
254 22617 5 Executor.exe 13 0 3
254 22617 5 Executor.exe 13 0 3
255
255 12877
12877 55 Executor.exe
Executor.exe 11
11 00 55
257 33729 5 Executor.exe 15 0 1
257 33729 5 Executor.exe 15 0 1
255 13417 5 Executor.exe 14 0 2
255 13417 5 Executor.exe 14 0 2

#.41
Top Session w/ Username
select
select
decode(nvl(to_char(s.sid),-1),-1,'DISCONNECTED','CONNECTED')
decode(nvl(to_char(s.sid),-1),-1,'DISCONNECTED','CONNECTED')
"STATUS",
"STATUS",
topsession.session_id
topsession.session_id "SESSION_ID",
"SESSION_ID",
u.name "NAME",
u.name "NAME",
topsession.program
topsession.program "PROGRAM",
"PROGRAM",
max(topsession.CPU)
max(topsession.CPU) "CPU",
"CPU",
max(topsession.WAITING)
max(topsession.WAITING) "WAITING", "WAITING",
max(topsession.IO)
max(topsession.IO) "IO",
"IO",
max(topsession.TOTAL)
max(topsession.TOTAL) "TOTAL"
"TOTAL"
from
from( ( {previous
{previousquery}
query} ) ) topsession,
topsession,
v$session
v$sessions,s,
user$
user$uu
where
where
u.user#
u.user#=topsession.user_id
=topsession.user_idand and
/*/*outer
outerjoin
jointotov$session
v$sessionbecause
becausethe
thesession
sessionmight
mightbebedisconnected
disconnected*/*/
topsession.session_id
topsession.session_id ==s.sid
s.sid (+)
(+)and
and
topsession.session_serial#
topsession.session_serial#==s.serial#s.serial# (+)
(+)
group
groupbyby topsession.session_id,
topsession.session_id,topsession.session_serial#,
topsession.session_serial#,topsession.user_id,
topsession.user_id,
topsession.program, s.username,s.sid,s.paddr,u.name
topsession.program, s.username,s.sid,s.paddr,u.name
order
order by max(topsession.TOTAL)desc
by max(topsession.TOTAL) desc

#.42
Top Session
Finding a Rogue User
STATUS SESSION_ID NAME PROGRAM CPU WAITING IO
STATUS SESSION_ID NAME PROGRAM CPU WAITING IO
--------------- ---------- ---------- ------------------------- ----- ---------- ----
--------------- ---------- ---------- ------------------------- ----- ---------- ----
CONNECTED 247 CPU_Monger ChMgr304.exe 11704 0 0
CONNECTED 247 CPU_Monger ChMgr304.exe 11704 0 0
CONNECTED 277 SYS oracle@labsfrh903 (LGWR) 14 19 0
CONNECTED 277 SYS oracle@labsfrh903 (LGWR) 14 19 0
CONNECTED 278 SYS oracle@labsfrh903 (DBW0) 29 0 0
CONNECTED 278 SYS oracle@labsfrh903 (DBW0) 29 0 0
CONNECTED 276 SYS oracle@labsfrh903 (CKPT) 18 9 0
CONNECTED 276 SYS oracle@labsfrh903 (CKPT) 18 9 0
CONNECTED 280 SYS oracle@labsfrh903 (PMON) 20 0 0
CONNECTED 280 SYS oracle@labsfrh903 (PMON) 20 0 0
DISCONNECTED 255 SYSTEM Executor.exe 11 4 5

#.43
SASH – DB Detailed ASL
@act
@act
Analysis
Analysis Begin
Begin Time
Time :
: 2007-07-24
2007-07-24 11:04:48
11:04:48
Analysis
Analysis End
End Time
Time :
: 2007-07-24
2007-07-24 11:19:45
11:19:45
Start
Start time,
time, mins
mins ago:
ago: 15
15
Request
Request Duration
Duration :
: 15
15
Collections :
Collections : 528528
Data
Data Values
Values :
: 3327
3327
Elapsed
Elapsed Time:
Time: 15
15 mins
mins
WAIT_EVENT
WAIT_EVENT CNT CNT %% Active
Active Ave_Act_Sess
Ave_Act_Sess
--------------------------------------
-------------------------------------- ----------
---------- ------------
------------
latch free
latch free 10
10 .3 .02
.3 .02
log buffer space 13 .39
log buffer space 13 .39 .02 .02
buffer
buffer busy
busy waits
waits 14
14 .42
.42 .03
.03
db file scattered read 15 .45
db file scattered read 15 .45 .03 .03
library
library cache
cache pin
pin
78
78 2.34
2.34 .15
.15
log
log file
file sync
sync 213
213 6.4
6.4 .4
.4
ON
ON CPU
CPU 726
726 21.82
21.82 1.38
1.38
enqueue
enqueue 855
855 25.7
25.7 1.62
1.62
db
db file
file sequential
sequential read
read 1399
1399 42.05
42.05 2.65
2.65

------------
------------
sum
sum 6.3 6.3
#.44
SASH – ASL over time
@aveact
@aveact
TM
TM NPTS
NPTS AVEACT
AVEACT GRAPHGRAPH CPU
CPU WAITS
WAITS
---------------- ------ ------- ---------------------- ----
---------------- ------ ------- ---------------------- ---- ----- -----
06-AUG
06-AUG 13:00:00
13:00:00 270
270 .33
.33 +- +- 22 29
29 59
59
06-AUG 14:00:00
06-AUG 14:00:00 1040
1040 2.24 ++--------2---
2.24 ++--------2--- 341 1984
341 1984
06-AUG 15:00:00
06-AUG 15:00:00 623
623 6.67 ++++------2----------
6.67 ++++------2---------- 438
438 3718
3718
06-AUG 16:00:00
06-AUG 16:00:00 1088
1088 2.59 ++--------2----
2.59 ++--------2---- 335 2486
335 2486
06-AUG 17:00:00
06-AUG 17:00:00 1104
1104 1.26 ++-----
1.26 ++----- 22 349
349 1043
1043
06-AUG 18:00:00
06-AUG 18:00:00 1093
1093 1.38 +++----
1.38 +++---- 22 663
663 842842
06-AUG 19:00:00
06-AUG 19:00:00 1012
1012 1.74 ++-------
1.74 ++------- 2 2 373 1388
373 1388
06-AUG 20:00:00
06-AUG 20:00:00 1131
1131 .99 +----
.99 +---- 22 304
304 820820
06-AUG 21:00:00
06-AUG 21:00:00 1111
1111 1.22 ++-----
1.22 ++----- 22 344 1012
344 1012
06-AUG 22:00:00
06-AUG 22:00:00 1010
1010 1.66 ++------
1.66 ++------ 2 2 414
414 1259
1259
06-AUG 23:00:00
06-AUG 23:00:00 1120
1120 1.08 +----
1.08 +---- 22 298
298 913913
07-AUG 00:00:00
07-AUG 00:00:00 1024
1024 .83 +---
.83 +--- 22 273
273 576
576
07-AUG 01:00:00
07-AUG 01:00:00 1006
1006 1.74 ++-------
1.74 ++------- 2 2 319 1428
319 1428
07-AUG 02:00:00
07-AUG 02:00:00 1090
1090 2.47 ++--------2----
2.47 ++--------2---- 347
347 2345
2345
07-AUG 03:00:00
07-AUG 03:00:00 687
687 6.59 +++-------2----------
6.59 +++-------2---------- 382 4142
382 4142
07-AUG 04:00:00
07-AUG 04:00:00 1004
1004 1.95 ++++++---
1.95 ++++++--- 2 2 1299
1299 659659
07-AUG 05:00:00
07-AUG 05:00:00 1104
1104 3.08 +++++-----2------
3.08 +++++-----2------ 1170 2226
1170 2226
07-AUG 06:00:00
07-AUG 06:00:00 1122
1122 1.91 +++++++--
1.91 +++++++-- 2 2 1582
1582 558558
07-AUG 07:00:00
07-AUG 07:00:00 1115
1115 1.06 +++---
1.06 +++--- 22 559
559 618
618
07-AUG 08:00:00
07-AUG 08:00:00 1140
1140 .81 ++--
.81 ++-- 22 403
403 519
519
07-AUG 09:00:00
07-AUG 09:00:00 1128
1128 .88
Copyright ++---
2006
.88 ++--- Kyle Hailey 22 386
386 601
601
#.45
SASH – ASL over time with waits
@aveactn
@aveactn
TO_CHAR(STA AAS AAS1 FIRST AAS2 SECOND GRAPH
TO_CHAR(STA AAS AAS1 FIRST AAS2 SECOND GRAPH
-----------
----------- ------- ----- --------------- ----- --------------- ------------------------
------- ----- --------------- ----- --------------- ------------------------
06 13:00:00 .30 .11 db file sequent .10 CPU - 2
06 13:00:00 .30 .11 db file sequent .10 CPU - 2
06 14:00:00 2.24 .87 direct path wri .49 log file sync ++--------2---
06 14:00:00 2.24 .87 direct path wri .49 log file sync ++--------2---
06
06 15:00:00
15:00:00 6.67
6.67 2.07
2.07 direct
direct path
path wri
wri 1.93
1.93 log
log file
file sync
sync ++++------2----------
++++------2----------
06 16:00:00
06 16:00:00 2.59 1.01 direct path
2.59 1.01 direct path wri wri .57 log file sync
.57 log file sync ++--------2----
++--------2----
06 17:00:00 1.26 .63 log file sync .32 CPU ++----- 2
06 17:00:00 1.26 .63 log file sync .32 CPU ++----- 2
06 18:00:00 1.38 .61 CPU .45 log file sync +++---- 2
06 18:00:00 1.38 .61 CPU .45 log file sync +++---- 2
06 19:00:00 1.86 .97 log file sync .39 CPU ++------- 2
06 19:00:00 1.86 .97 log file sync .39 CPU ++------- 2
06 20:00:00 .99 .38 log file sync .27 CPU +---- 2
06 20:00:00 .99 .38 log file sync .27 CPU +---- 2
06
06 21:00:00
21:00:00 1.22
1.22 .57.57 log
log file
file sync
sync .31
.31 CPU
CPU ++-----
++----- 22
06
06 22:00:00
22:00:00 1.66
1.66 .81.81 log
log file
file sync
sync .41
.41 CPU
CPU ++------
++------ 22
06 23:00:00 1.08 .49 log file sync .27 CPU +---- 2
06 23:00:00 1.08 .49 log file sync .27 CPU +---- 2
07 00:00:00 .83 .37 log file sync .27 CPU +--- 2
07 00:00:00 .83 .37 log file sync .27 CPU +--- 2
07 01:00:00 1.74 .96 log file sync .31 CPU ++------- 2
07 01:00:00 1.74 .96 log file sync .31 CPU ++------- 2
07 02:00:00 2.48 1.21 direct path wri .52 log file sync ++--------2----
07 02:00:00 2.48 1.21 direct path wri .52 log file sync ++--------2----
07 03:00:00
07 03:00:00 6.59 2.63 direct path wri 1.58 log file
6.59 2.63 direct path wri 1.58 log file sync sync +++-------2----------
+++-------2----------
07 04:00:00 1.95 1.29 CPU .27 log file sync ++++++--- 2
07 04:00:00 1.95 1.29 CPU .27 log file sync ++++++--- 2
07 05:00:00 3.08 1.05 CPU .83 enqueue +++++-----2------
07 05:00:00 3.08 1.05 CPU .83 enqueue +++++-----2------

#.46
Many Ways to Attack Problems
Confusing
 How to Attack the problem?

 Top SQL?
 Top wait for that SQL?
 Top Waiting Session ?
 Top Waits for that Session
 Top Waits for Database?
 Top Session waiting for that wait
 Top SQL for that wait
Solution - Graphics

#.47
Mining Data is Non Trivial

 Many Dimensions to consider
 Constantly Varying time frames
 Luckily 10g automates all of this functionality.
 collectsall of the data
 analyses all the wait events
 reports on bottlenecks
 supplies solutions
 Graphics add tremendous ease and power

Graphical ASH
Session 1
Session 2
Session 3
Session 4
TIME
#.49
Graph of User States

#.50
One Second Graph

#.51
15 Second Averages

#.52
Maximum CPU Line

#.53
Idle Users

#.54
OEM Perf Page

#.55
OEM Perf Page

#.56
ASH
vs
Statistics
 Statistics
 are more expensive
 have lag time
 lack clear identification of culprits

#.57
Statistic Lag Time

Samples
Slight Lags
Counters

#.58
CPU Lag Problem

 ASH is the only way to see CPU usage
realtime
 V$sysstat reports CPU but
 isonly updated at the end of the call.
 Long calls look deceiving like no CPU is being
used

#.59
CPU in ASH vs Stats

#.60
ASH Sampling Cheap

Three Methods
1. With PL/SQL it’s less that 1 % CPU with 10
average active sessions
2. DMA - direct memory attach to SGA Using C
code 200x Cheaper
3. ASH in 10g should be even cheaper than
DMA because it’s done in the kernel

#.61
ASH – In Memory
 Collects active session data only
 History v$session_wait + v$session + extras
 Circular Buffer - 1M to 128M (~2% of SGA)
 Flushed every hour to disk or when buffer 2/3
full (it protects itself so you can relax)

#.62
ASH Sizing …
 Avg row around 150bytes

 3600 secs in an hour
 ~ ½ Meg per Active Session per hour
 That’s generally over an hour of ASH

#.63
How ASH works
V$session_wait
+ v$session +
extras
MMON/
MMNL
v$active_session_history Only writes out 1 in 10 rows

via direct path inserts
wrh$_active_session_history

#.64
ASH buffer
Insert one direction
Select reads
backwards - Touch up wait times
-No latching
-No read consistency
-Index on time Insert point

#.65
Family of ASH Tables
v$session_wait
V$session_wait_history
DBA_HIST_ACTIVE_SESS_HISTORY
wrh$active_session_history
ASH Tables #.66
current 10 samples hour 7 days (disk)

1 in 10
v$session_wait v$session_wait_history v$active_session_history DBA_HIST_ACTIVE_SESS_HISTORY
wrh$active_session_history
SID SID
SAMPLE_ID
SEQ# SEQ# SNAP_D
SAMPLE_TIME
EVENT EVENT# DBID
EVENT SESSION_ID INSTANCE_NUMBER
P1TEXT
P1 P1TEXT SESSION_SERIAL# SAMPLE_ID
P1RAW P1 USER_ID
SAMPLE_TIME
P2TEXT P2TEXT SQL_ID
P2 SESSION_ID
P2 SQL_CHILD_NUMBER
P2RAW SESSION_SERIAL#
P3TEXT
P3TEXT SQL_PLAN_HASH_VALUE USER_ID
P3
P3 SQL_OPCODE
WAIT_TIME SQL_ID
P3RAW SERVICE_HASH
WAIT_CLASS_ID SQL_CHILD_NUMBER
SESSION_TYPE SQL_PLAN_HASH_VALUE
WAIT_CLASS#
WAIT_CLASS SESSION_STATE SQL_OPCODE
WAIT_TIME QC_SESSION_ID
SERVICE_HASH
SECONDS_IN_WAIT QC_INSTANCE_ID
SESSION_TYPE
STATE
SEQ# SESSION_STATE
EVENT# QC_SESSION_ID
P1
QC_INSTANCE_ID
P2
SEQ#
v$event_name P3 EVENT_ID
WAIT_TIME P1
EVENT_ID
TIME_WAITED P2
EVENT# CURRENT_OBJ#
P3
EVENT_ID CURRENT_FILE#
WAIT_TIME
NAME CURRENT_BLOCK# TIME_WAITED
PARAMETER1 PROGRAM CURRENT_OBJ#
MODULE
PARAMETER2 CURRENT_FILE#
ACTION
PARAMETER3 CURRENT_BLOCK#
CLIENT_ID PROGRAM
WAIT_CLASS_ID
MODULE
WAIT_CLASS#
ACTION
WAIT_CLASS
CLIENT_ID
#.67
Wait Time vs Time Waited
 SESSION_STATE
 Waiting,
on CPU
 Based on WAIT_TIME
 WAIT_TIME (v$session, v$session_wait, v$ash)

 0 => waiting
 >0 => CPU (value is time of last wait)
 TIME_WAITED
 Actual time waited for event
 0 until wait finishes
 Fix up values (no one else can do this)

#.68
Oradebug
 Dump to trace file
SQL>
SQL> oradebug
oradebug dumpashdump
ash 5 5
SQL> Alter session set events ‘immediate
SQL> Alter session set events ‘immediate
tracenameashdump
tracename ashdump level 5’;
level 5’;
level 5 = # of minutes
loader file rdbms/demo/ashldr.ctl

#.69
INIT.ORA
statistics_level = Typical (default)
PARAMETER SESSION_VALUE INSTANCE_VAL

----------------------- ---------- ------------
_ash_sampling_interval 1000 1000
milliseconds
_ash_size 1048618 1048618
ASH buffer size
_ash_enable TRUE TRUE
Turn on/off ASH sampling, flushing and the V$ views on ASH
_ash_disk_write_enable TRUE TRUE
Flush to disk
_ash_disk_filter_ratio 10 10
write 1 in 10 points
_ash_sample_all FALSE FALSE
Sample including idle waits
#.70
10.2 ASH Extras
 BLOCKING_SESSION
 BLOCKING_SESSION_STATUS
 BLOCKING_SESSION_SERIAL#
P1TEXT
P2TEXT
P3TEXT
WAIT_CLASS
WAIT_CLASS_ID
XID
12/07/21 70
#.71
10.2 Extras
 10gR2 add fields to ASH
 Blocking Session Id, serial# and state ***
 XID, transaction ID
 RAC event Fixup
 Plan Hash Fix up
 10gR1 - during parsing no plan, good way to find parsing
problems
 10gR2 – get plan hash fixup – good but lose some ability to find
parsing problems
 Wait Class, needed this for grouping
 Force_matching_signature – sql profile id

#.72
10.2.0.3 ASH Extras

FORCE_MATCHING_SIGNATURE
PLSQL_ENTRY_OBJECT_ID
PLSQL_ENTRY_SUBPROGRAM_ID
PLSQL_OBJECT_ID
PLSQL_SUBPROGRAM_ID
ALL_PROCEDURES
where object_id = plsql_object_id
and subprogram_id = plsql_subprogram_id
12/07/21 72
#.73
PLSQL Tracking
calling_code SQL_ID COUNT(*)

-----------------------------------------------
Package/Procedure/SQL
ORDERENTRY.NEWORDER
tracking ----------
10.2.0.3
0uuqgjq7k12nf 258
ORDERENTRY.NEWCUSTOMER 0bzhqhhj9mpaa 262
ORDERENTRY.BROWSEANDUPDATEORDERS 41zu158rqf4kf 301
ORDERENTRY.NEWORDER 0yas01u2p9ch4 569
PARSE_SAME 3vjxpmhhzngu4 874
ORDERENTRY.BROWSEANDUPDATEORDERS 05s4vdwsf5802 1669
select
object_name package||.|| procedure_name, sql_id, count(*)
from v$active_session_history ash,
all_procedures procs
where
ash.PLSQL_ENTRY_OBJECT_ID = procs.object_id
and ash.PLSQL_ENTRY_SUBPROGRAM_ID = procs.SUBPROGRAM_ID
12/07/21 73
11g ASH extras #.74
SQL Elapsed CPU Analysis – non Timed

SQL_EXEC_ID IN_CONNECTION_MGMT
SQL_EXEC_START IN_PARSE
IN_HARD_PARSE
SQL Row Source IN_SQL_EXECUTION
SQL_PLAN_LINE_ID IN_PLSQL_EXECUTION
SQL_PLAN_OPERATION IN_PLSQL_RPC
SQL_PLAN_OPTIONS IN_PLSQL_COMPILATION
IN_JAVA_EXECUTION
IN_BIND
RAC – remote transfers
IN_CURSOR_CLOSE
REMOTE_INSTANCE#
ETC
Recursive SQL CURRENT_ROW#
EVENT#
TOP_LEVEL_SQL_ID QC_SESSION_SERIAL#
TOP_LEVEL_SQL_OPCODE CONSUMER_GROUP_ID
FLAGS

#.75
11g ASH extras

Run-time SQL row source information
 Identifies current row source within plan
SQL execution ID
 Is this same execution as last sample?
 can see how long sql has been running
Operation bit vector

 Capture non-timed operations
 Examples: fetch, binding, close
Remote instance id for Cache transfers

 Which instance sourced requested block?

#.76
How Many Active Sessions?

 How much data does ASH Collect ?
 1 CPU means max 1 Avg Active Session unless there
is a bottleneck
Big site examples:
 Oracle 4 way RAC internal apps
 10,000 connected, 200 active
 One Site
 3000 connected, 30 Active
 Site
 12,000 connected, 100 active

#.77
Simulating ASH
 ASH is new in 10g
 ASH data exist since V7
 Need Diagnostic Pack License 
 Collect it yourself 
 SASH – Simulated ASH
http://perfvision.com/sash.php
 Graphical Monitor
http://perfvision.com/ashmon.php
Consumes < 1% CPU for 10 active sessions (a lot)

#.78
Sampling
 Sampling is the future
 Took clean vision to create
 Bit of a leap of faith
 Less Accurate but more powerful than past
 Inaccuracies unimportant for the most part
 Feasible since version 7
 You can implement it yourself
 Graphics are the only way to full harness the
power

#.79
Summary
 To solve hard problems you need Sampling
 If you have 10g and a license you can use
 V$active_session_history
 OEM 10g (can access some 9 sampling)
 ?/rdbms/admim/ashrpt.sql
 If you are on 7,8,9 or 10g without the license

 Need custom sampling
 SASH – Simulated ASH http://perfvision.com/ash.php
 ASHMON – graphical http://perfvision.com/ashmon.php

#.80
ASH Summary
Faster, Cheaper, instant (no lag) …
but most importantly
Rich & Multidimensional

 Overview of system load
 Drilldowns into problems
 Sql
 Session
 Wait event
 Details information on problem resolution
 OEM simplifies mining of ASH data

Active Session History: Feel The Power

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Active Session History: Feel The Power

Uploaded by

Copyright:

Available Formats

ASH – Active Session History

Feel the Power

 Totally new and exciting methodology

Copyright 2006 Kyle Hailey

 10g immediately Accessible

 Stuck on Oracle 7,8,9 ?

Copyright 2006 Kyle Hailey

Here’s a piece of Personal

Copyright 2006 Kyle Hailey

This is the story of how

But … It was then that I learned I

Copyright 2006 Kyle Hailey

Clearly a job for Stats Pack

 All the Stats!

Copyright 2006 Kyle Hailey

So why wasn’t I feeling better?

 What if my database hits a bottleneck 15

Copyright 2006 Kyle Hailey

If lives (and my job) depended on

 Was once an hour really enough ?

Copyright 2006 Kyle Hailey

Don’t tell anyone but …

Copyright 2006 Kyle Hailey

In that moment of crises

Copyright 2006 Kyle Hailey

The crises of my day

I didn’t have ASH but

It’s a Revolution and it’s an

(like the wheels on the suitcases)

Copyright 2006 Kyle Hailey

Copyright 2006 Kyle Hailey

ASH – Intelligence for the new

 Those old methods collected everything

Copyright 2006 Kyle Hailey

Sessions change a lot quicker but can get the main

Sampling is like taking

Copyright 2006 Kyle Hailey

If happens a lot or for long …

Copyright 2006 Kyle Hailey

IO CPU Wait Idle

Copyright 2006 Kyle Hailey

 9i or below – just have to keep track

Copyright 2006 Kyle Hailey

w.wait_time != 0 /* on CPU */ and

Identifying CPU via v$session_wait is

 10g: WAIT_CLASS=‘User I/O’

Copyright 2006 Kyle Hailey

Copyright 2006 Kyle Hailey

10:15:00 10:15:01 10:15:02 10:15:03 10:15:04 10:15:05 10:15:06 10:15:07

Name Null? Type

Primary Fields of ASH

Amazing things YOU can do with

Top CPU Session

Results Top CPU Session

Copyright 2006 Kyle Hailey

CPU with Bars

Bar shows 10% increments

Copyright 2006 Kyle Hailey

Top Waiting Session Results

Copyright 2006 Kyle Hailey

Top SQL from ASH Results

Copyright 2006 Kyle Hailey

Top Session Results