Who Hit The Turbo Button?

Presenter: Stephen Watson

ITS System Manager
Stephen F. Austin State University

February 13, 2013

Stephen F. Austin State University

Stephen Watson
Systems Manager ITS
Previous jobs at SFASU
(Programmer Analyst in Student/HR/Budget)
(Systems Programmer I and II)
18 Years at SFASU

Gabriel Nguyen
Sr. Sales Consultant
Oracle Corporation
 Agenda

How SFA went from sluggish to turbo performance...

 Introduction
 Challenges
 Architecture Selection Process
 Exadata Implementation Process
 Performance Results
 Where We Are Today
 Q and A
 Stephen F. Austin State University

Academic and Research University

Undergraduate, Graduate and Doctoral

Enrollment of 13,000

Both Traditional and Online Courses

 SFA Support Staff

 3 DBA’s

 7 Systems

 11 Programming
 Current Environment

 Ellucian Banner 8
 Student

 Human Resources

 Finance

 Financial Aid

 WebFOCUS Reporting

 UC4 AppManager
 Oracle 11g R2 DB and 11gR2 WLS/FMW
 Why Did We Need A New DB Platform?

 Registration Performance Issues

 Future DB Platform Support from Oracle

and Ellucian
 Problem 1: Registration Performance

Student Experience Impact

 2008 Banner 8 Go-Live
 Registration load issues during orientations
and pre-registration periods
 Limited to 250-300 simultaneous users
 Had to break registration in very small
groups (<500)
 Problem 1: Registration Performance

Peripheral Impact
 University reputation (bad press, social media…)

 Student satisfaction declined

 Negatively impacted enrollment

 Negative comments from parents about

experience
Making the News
 Problem 2: Product End of Life

Technology Impact
 HP de-support Open VMS
 Ellucian de-support of Open VMS
 Oracle de-support of Itanium platform
 HPUX/11G would be de-supported soon
soon as well
 Environment Before Exadata

 HP Itanium rx6600 Running Open VMS 8.3 and

Oracle 10G R2 (3 identical servers)
 Banner Production on one server
 Banner Test instances (4) on second server
 Banner Prod DR on third server, ready to come
up when needed
 No high-availability, data protection, or
disaster recovery capabilities
 Hardware Refresh Time

 We could run HPUX/11G on

the servers but this would be
de-supported soon as well

 After getting another year

from the hardware we would
have to buy new hardware
 Why we selected Exadata

 Investigation and research

 Oracle EX-CITE conducted on campus
 Key advantages
 Fast transaction processing
 Extreme performance and redundant appliance type
architecture
 One throat to choke (so to speak)
 One process to patch everything
(OS, switches, storage and database homes)
 Lots of storage capacity and processing power
for future growth
 What is Exadata?

 Oracle’s Engineered Database Machine

 Pre-configured, pre-integrated and highly efficient combination of
hardware and software to provide a complete solution to the customer

 Optimized for OLTP and DW database workloads

 The Exadata Quarter “Server” or Rack contains
Multiple database nodes (2)
Multiple Cell or storage servers (3)
Shared storage (12 3 TB disk per cell )
All connected with a fast Infinband network (40 GB/sec)
Designed to work as a unit to increase transaction performance
Lots of RAID redundant storage. (108 TB raw )
ASR Server monitors hardware failures continually
 Solution: SFA Exadata Platform
Grid Control
Console
OEM Grid Control
Prod DR/Dev/Test/QA
X2-2 Quarter Rack X2-2 Quarter Rack

• 10g to 11g migration • Stand-by PROD

databases
• RAC Active Data Guard • 5 databases, 10
• 3 databases, 6
(physical standby) Instances
Instances
• 3.0 TB size
• 700 GB size

Data Center A – Site 1 Data Center B – Site 2

X2-2 Quarter Rack GigE Backup

• 2 Sun Fire x4170 DB Servers Network
• 3 Exadata Storage Servers (SAS)
• Raw Disk – 108TB (SAS Drives)
• Raw Flash – 1.2TB
• User Data – 40TB ( No Compression) ZFS Storage Appliance
•Backups of database
ZFS
•Export dumps
Backup Area – 20 TB
 Bottom Line

 SFA needed a solution that “just works”

 SFA enrollment growing – need a solution that will
keep up with demands of administration and students
 Opportunity for IT consolidation to lower costs
 Avoid further loss of confidence from faculty/staff
in our IT management and staff
 Bottom Line

 Loss of confidence from the students in SFA being

able to deliver on their mission promise: “...comprehensive
institution dedicated to excellence in teaching, research,
scholarship, creative work, and service”
 To effectively recruit quality students in the future and
confidence from parents in SFA being the right university
for their family members
 Project Timeline

A very rapid deployment

• DB Export Started on 4/5/2012 at 5:00 pm

• DB import into Exadata Completed 4/7/2012
• Testing and Data Validation 4/7 thru 4/8/2012
• Go live Sunday 4/8/2012 at 7:00 pm
 Summary of Implementation

 Delivery to go live one of the shortest I have ever

been involved with
 Lots of long hours including evenings and
weekends (several 80-90 hour weeks)
 Oracle could not believe we implemented in this
short time frame
 Usual implementation of Exadata
(6 months to 1 year)
 Ellucian and Oracle consulting were crucial
 Where we are today

 2 successful large registrations, fall and spring

 5 successful orientations (registration)
 Returned to larger groups for registration
 Classifications (about 3000 per group)

 All Processes running faster

 Better faculty/staff experience in Banner
 Better student experience with
registration
 Where we are today (continued)

 Administration very pleased with the success of the

registration process
 Strategically positioned for future IT consolidation of
databases on campus
 No complaints from students
 Benefits

 Time to Value
 Return on Investment
 Performance

Before: After: Performance

Times Times Improvement
Query run time 1 to 3 minutes 10 to 30 seconds 6 fold increase
Reporting
AR Processes 5 Min to 3 Hours 5 Sec to 10 min 60 fold increase
Other Processes
Backup Times Full Hot Backups Full Hot Backups Time cut in half
4 Hours 2 hours
Registration Less than 200 Up to 1000 5 Fold increase
concurrent concurrent
 Highlights

 One Vendor for OS and DB issues

 Oracle patches everything (OS, DB, Switches and

Storage)

 More disk space than we know what to do with

 FLASH CACHE storage on disk helps performance

 Highlights

 Lots of room for future growth

 No system tweaking - delivered for performance

 Faster backups

 Faster cloning
 Questions?

For additional information contact:

Stephen Watson
IT System Manager
Information Technology Services
swatson@sfasu.edu