Release Control with PowerCenter at UBS / Switzerland

Friedrich Lehn Configuration & Release Manager October 2000

Agenda

Project Overview PowerCenter Configuration Configuration Management Deployment Process Release Process Summary

Project Overview
UBS

Global, integrated investment services firm offering - the full range of wealth management, asset management, investment banking and securities services - to the full spectrum of clients Worlds largest private bank Top ten financial services firm by market capitalization - $62 billion dollars Total client assets of US$1 trillion

Project Overview
System Environments

Development Development

Test Test

Production Production

IBM AIX SP2 cluster DB2 UDB EEE PowerCenter 1.6

System of Records

Project Overview
Load Process

File Preprocessing

Conditioning

AutoLoader

PowerCenter

AutoLoader

SQL Apply

Common Data Warehouse

Project Overview
Delivery Stream

Main organizational element Three letter acronym as base for naming standards Each delivery stream gets the same infrastructure: UNIX directories, meta data area, PowerCenter folders Each delivery stream has business responsible, data modeler, database administrator, delivery stream manager Team size typically between 1 and 5 Delivery streams release independently from each other

Project Overview
Application Structure
RCL MDR

System of Records

DSF
DSF

FDS

Common Data Warehouse

Delivery streams: RCL MDR DSF FDS <SDA> <MAR> release control tools meta data repository DWP sourcing framework feed specific data processing subject data area data mart
<SDA>

...

<SDA>

DWP Sourcing Framework

<MAR>

...

<MAR>

Project Overview
Directory Structure

Two areas: /dwp_root /dwp_data

UNIX based files, version controlled dynamic data, archival on demand

Additional directory level for the environment in order to support more than one environment on one system /dwp_root is organized by delivery streams

Project Overview
Directory Structure (continued)

/dwp_data is organized by logical processing steps and by delivery streams Symbolic links from PowerCenter program area to /dwp_data Tool support for generation of directories (delta processing, integration with release procedures)

PowerCenter Configuration
Design Principles

Start with one single (local) repository Support different, clearly separated environments with different responsibilities All environments have identical structure (products, databases, PowerCenter server configurations) All program changes (UNIX, PowerCenter) are done on the development system All changes on test and production systems go through the release process and are clearly tracked

PowerCenter Configuration
Repository Structure
D
Server

A
Server

P
Server

D S V A P

development staging versioning integration / user acceptance test production

PowerCenter Configuration
Folder Structure
user area (avoid shortcuts) team area

release area

Release area is identical in all environments.

PowerCenter Configuration
Roles and Responsibilities
Role
developer delivery stream manager release manager

Responsibility
development, unit and integration testing manager, release planning deployment, tracking, configuration control PowerCenter administration session management for test and production systems

session manager

Configuration Management
ClearCase

External tool for software configuration management (Rational Corporation) Powerful version control, work space and build management Same directory structure as in development environment Fully automated access layer

Configuration Management
Why external version control?

Integrate all release specific data: sourcing framework, DDL, documentation, scripts & SQL Avoid problems with current PowerCenter version mechanism (shortcuts, sessions) Enable automation of release process Different responsibilities for different environments Enforce change tracking

Configuration Management
Release Naming

<delivery stream>_<major>.<minor>.<patch> e. g. ACC_1.0.1 <major> <minor> <patch> major release number (high level wave planning) minor release number (delivery stream development plan) patch level (bug fixes, emergency releases)

Configuration Management
Versioning Example
reset_acc.sql /main/1 /main/2
ACC_1.0.1 ACC_1.0.0

/main/3 Labeling Schema: ACC_x.y.z release label RLS_A current version in user acceptance test RLS_P current version in production /main/4 /main/5

ACC_1.0.2 RLS_P

ACC_2.0.0 RLS_A

Deployment Process
PowerCenter Deployment Facilities

Repository Copy: - via GUI / database backup & restore - trackable (result is a file) - problem: overwrites server configuration Folder Copy: - only via GUI - and therefore not trackable Mapping Copy: - only via GUI - used only within development repository

Deployment Process
Approach

Repository Copy is the only trackable method Delivery Streams require folder level deployment Use combination of both methods:

Introduce staging repositories in all environments Staging repository is used to separate folder from repository, using the Folder Copy method Staging repository is then deployed using the Repository Copy method

Deployment Process
Freeze, Deliver and Receive Processes
D
Freeze

A
Receive

ClearCase
Deliver

manual folder copy database unload / load automated process delivery package

Deployment Process
Repository Unload
sub unload { system("db2 connect to $repository user $user using $password"); db2select(\@tables, SELECT tabname FROM syscat.tables WHERE tabschema = 'PCSTAGE' and (tabname LIKE 'OPB_%' or tabname LIKE 'REP_%') and type = 'T' ORDER BY tabname"); foreach $table (@tables) { system("db2 export to $table of del modified by coldel0x02 chardel0x03 select \\* from $user.$table); }; system("tar cvf $tarfile *"); }

Deployment Process
Repository Load
sub load { system("db2 connect to $repository user $user using $password"); system("tar xvf $tarfile"); foreach $file (("OPB_REPOSIT", "OPB_REPOSIT_INFO")) { system("sed -e 's/CDW_StagingD/$name/g' $file > tmp.out"); system("sed -e 's/DDBPCR/$dbs/g' tmp.out > $file"); } foreach $table (<*>) { system("db2 delete from $user.$table); system("db2 import from $table of del modified by coldel0x02 chardel0x03 delprioritychar insert into $user.$table); }; }

Release Process
Change Control Board

Responsible for high level planning and impact analysis Defines release scope and release numbers on base of delivery streams Assigns responsibilities (delivery stream manager, data modeler, database administrator, business responsible) Result is documented in wave plan document Wave plan is base for all subsequent design, implementation and quality assurance tasks

Release Process
Release Procedure
1. Preparation of UNIX and PowerCenter release areas 2. Issue release request 3. Create new release (Freeze) - Copy to staging repository, unload - Create / update ClearCase file versions - Create release label and attach to new release 4. Create release package (Deliver) - delta / full processing - lock release in ClearCase 5. Install release (Receive) - Load, copy from staging repository - Post install script 6. Create / update database objects in target environment 7. Restart PowerCenter server

Responsible
delivery stream manager delivery stream manager release manager

release manager

IT integration / IT operation

database administrator IT integration / IT operation

Release Process
Release Database

Provide easy-to-use information on all releases Today: ClearCase based In future: Integration with meta data repository
DSF_1.1.4 DSF_1.1.3 DSF_1.1.2 DSF_1.1.1 DSF_1.1.0 DSF_1.0.2 DSF_1.0.1 DSF_1.0.0 A 27-Jul-00.17:32:27 20-Jul-00.13:16:38 P 19-Jul-00.14:05:30 18-Jul-00.17:51:19 17-Jul-00.17:11:47 11-Jun-00.14:07:01 09-Jun-00.13:20:03 24-May-00.17:48:42 new config files fix for single node load problem correct JDM, error handling removed environment dependencies see release notes changed Gen_Environment see release notes initial version

Example:

Summary
Experiences

PowerCenter was successfully introduced at UBS and proved to work well within the sourcing framework Configuration control with ClearCase is efficient and straight forward Release process meets the requirements and is efficient (so far ~200 successful releases, effort usually below hour) Generally little administration overhead although some improvements in PowerCenter would make administrators and release managers life easier

Summary
Wish List

Copy of sessions Export / import folders via (UNIX) command line for further automation of release process Avoid restart after deployment Logical backups / restores via command line with result on server side (more flexible than database backup, avoid ODBC overhead) User administration via command line

Contact Information

Friedrich Lehn UBS AG, Switzerland phone: +49 171 360 60 84 email: friedrich.lehn@t-online.de