Data Management & Warehousing

From Volume to Value: What Next Generation Telco Data Warehouses Must Do to Provide Value to the Business David M. Walker davidw@datamgmt.com

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 1 of 30 30 March 2006

What do we have ?
•  A European Mobile Telco:
–  Data warehouse has over 150 Billion CDRs –  Over 2000 registered users

•  But:
–  It takes 20 minutes to get answers to even the most basic question which should only take seconds –  Less than 100 people use it every day and they all hate the reporting tools –  Operations and support costs are soaring –  Can’t get changes to the system through fast enough
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 2 of 30 30 March 2006

What are the engagement issues ?
•  It doesn’t have anything for my role
–  Don’t let the data warehouse sit outside a business process
•  Identify where a report changes/helps

–  People don’t need to know that they are using the data warehouse
•  Ensure that it is integrated in their daily activities

–  Pro-actively educate people about what is available
•  Most people see data warehouses as something remote

•  I’ve already got a report that does this
–  Normally the response from the spreadsheet jockey
•  But how accurate is it and how long does it take to produce ?

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 3 of 30 30 March 2006

What are the users issues ?
•  I don’t know what information is available!
–  Users are unwilling to search too hard for what is available –  Users are unable to comment quickly and easily on what is available –  Users ‘just’ want ‘the right report’ fed to them

•  The report I got was wrong!
–  Published data profiles tell users where the data issues are –  Helps users understand the requirement for data cleansing back into operational systems (GIGO) –  Nobody available to quickly modify a report to what the user actually wants

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 4 of 30 30 March 2006

What are the users issues (cont.) ?
•  I asked them for this new report and they told me it would be two months!
–  Who helps users to understand what is already available ? –  Who is available develop a report quickly?

•  I went on a training course for the reporting tool but that was six months ago and I can’t remember how to use it now!
–  If users are going to use a tool they should be frequent users with someone to support them –  If not, then provide resource to do the work for them

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 5 of 30 30 March 2006

Five things that can help
•  Exploitation/QuickService Teams •  Data Profiling & Data Cleansing •  Process Integration •  Business Information Portals •  RSS - Really Simple Syndication
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 6 of 30 30 March 2006

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 7 of 30 30 March 2006

Exploitation/QuickService Teams
•  Have a (small) team that is ‘here to help’
–  Available via:
•  •  •  •  Telephone Chat Room Web Conference Issue Tracking System

–  Technology reduces costs of running a team and makes the data warehouse feel more accessible

•  Will respond quickly to urgent requests
–  Even if the answer is it will take sometime to fulfil the total requirement here’s what we can do now
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 8 of 30 30 March 2006

Exploitation/QuickService Teams (2)
•  Look for heavy users and heavy queries and find ways to help them
–  –  –  –  Cut out unused parts of the data warehouse Optimise response of major users Revise archiving strategy based on required data Ultimately reduces operational and support costs

•  Visit frequent callers to the helpdesk
–  Re-enforce training –  Gather new requirements –  Pre-empt the need to call the helpdesk

•  But basically provide proactive support
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 9 of 30 30 March 2006

Direct Access to the Quick Service Team What’s related develops understanding

Pre-empt FAQ’s

Quick Service Team can monitor and respond
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 10 of 30 30 March 2006

Data Profiling
•  Look at your source systems and understand what the data quality issues are
–  –  –  –  Which required fields are not populated ? Which fields always have a default value ? Do all customers have sufficient contact details ? Etc.

•  Detect and capture issues in the Data Warehouse
–  Often related to issues of integration across systems

•  Set up targets to improve the data quality
–  Especially in the source system –  Publish the metrics and identify responsible owners
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 11 of 30 30 March 2006

Allow users to see related data quality

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 12 of 30 30 March 2006

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 13 of 30 30 March 2006

Data Cleansing
•  Fix data in the source systems
–  A data quality issue fixed in the source will have benefits for other areas and often highlight business process issues

•  Embed a call to the cleaning tool in all ETL
–  Rule based cleansers simple and easy to implement –  Add the call even if there is no current requirement –  Use a metadata driven tool so new rules can be added –  Track the success rate of the results –  BUT: Maintain copies of the original data
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 14 of 30 30 March 2006

Example Data Cleansing Issues
•  Standardisation of text
–  Prevents correct aggregation –  Multiple spellings
•  e.g. Zürich, Zuerich, Zurich => Zurich, Rd => Road

–  Spaces
•  e.g. David_ _Walker is not the same as David_Walker

–  Standardization of case
•  E.g. David Walker => DAVID WALKER, Zurich => ZURICH

•  Range validation if dates
–  13-Mar-0006 becomes 13-Mar-2006
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 15 of 30 30 March 2006

More Data Cleansing Issues
•  Mapping of codes & translation
–  01 means ‘Fixed Line’ in one system and ‘Roaming’ in another –  A code meant one thing for a period of time and then it’s use was changed to mean another thing after a certain date

•  Overcome System Defaults
–  80% or all customers are MALE
•  Actually the default is MALE and most operators just tab over the field

–  Date of Birth is nearly always empty
•  Optional field in source system – change to mandatory

–  Date of Birth is 01-01-1900
•  Mandatory field with no range checking and no option for declined
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 16 of 30 30 March 2006

Process Integration
•  Put the data warehouse into the process rather than sitting to one side
–  Use it to allow customers to compare price plans online
•  But restrict price plans available based on profile

–  Trigger on-line offers and customised content when customers log into the website –  Add web ‘popup’ pages to existing internal applications
•  Call centre gets an ‘image’ of the person they are talking to

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 17 of 30 30 March 2006

Process Integration
•  Put the data warehouse into the process rather than sitting to one side
–  Use it to allow customers to compare price plans onlinePlan: Price Prepay 5
Last Call (Detail): 0118 and customised content when –  Trigger on-line offers9776 911 Last Contact: 20-Mar-2006 customers log into the website Next Best Offer: Text pack 50 Open Cases (View): –  Add web ‘popup’ 3  pages to existing internal Dropped Calls:  Teenager Network Quality: applications 

• Last Bill (View): price £0.50 available based on profile But restrict plans

Stop Churn:

No

 ‘image’ •  Call centre gets an Texter 100of the person they are talking to

Handset Type:

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 18 of 30 30 March 2006

Process Integration
•  Put the data warehouse into the process rather than sitting to one side
–  Use it to allow customers to compare price plans Price Domestic 100 online Plan: –  – 
Stop Churn: Yes •  Last Bill (View): But restrict price plans available based on profile £37.50 Last Call (Detail): 07990 594 372 Trigger on-line offers and customised content when Last Contact: 10-Jan-2003 Next Best Offer: customers log into Handset +£50 the website Open Cases (View): 0 Add web Calls: Dropped ‘popup’ pages to existing internal  Single Working Female Network Quality:  applications Handset Type:  Cheapo X79a

•  Call centre gets an ‘image’ of the person they are talking to

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 19 of 30 30 March 2006

Process Integration
•  Put the data warehouse into the process rather than sitting to one side
–  Use it to allow customers to compare price plans online Price Plan: Business 350 –  – 
Last Bill (View): £180 Trigger on-line offers 028 911 Last Call (Detail): 07050 and customised content when Last Contact: 15-Sep-2005 customers log into the +£0 website Next Best Offer: Handset Open Cases (View): Add web ‘popup’None pages to existing internal Dropped Calls:  Business User applications Network Quality:  Handset Type: •  Call centre gets  Executive 3030 person they are talking to an ‘image’ of the Stop Churn: •  But restrict priceYes plans available based on profile

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 20 of 30 30 March 2006

Business Information Portals
•  Single touch point
–  The delivery mechanism for all business information services.

•  Collaboration
–  Allows users to communicate
•  Synchronously (through chat & messaging) •  Asynchronously (through threaded discussion & email digests)

•  Integration
–  The connection of functions and data from multiple systems into new components
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 21 of 30 30 March 2006

Business Information Portals (2)
•  Content and document management
–  Services that support the full life cycle of document creation and provide mechanisms for authoring, approval, version control, scheduled publishing, indexing and searching. –  Consider a Wiki: a user editable webpage

•  Personalization
–  Allows users to subscribe (or be subscribed) to specific types of content and services. –  Users can also customize the look and feel of their environment.
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 22 of 30 30 March 2006

RSS - Really Simple Syndication
•  If I could offer you only one tip for the future, RSS would be it.
–  Already an inbuilt technology in most web browsers and mail clients –  Very cheap to modify existing reports to work with it –  Allows publish/subscribe to ‘news feeds’
•  These feeds would be reports by subject area

–  An established technology already widely in use
•  e.g. BBC, most newspapers, etc. & Podcasts

–  Can easily be integrated with textual content
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 23 of 30 30 March 2006

RSS - Really Simple Syndication
•  If I could offer you only one tip for the future, RSS would be it.
<?xml version="1.0" ?> - <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns=http://purl.org/rss/1.0/> - <channel rdf:about="http://www.datamgmt.com/files/phpwsrssfeeds/backend3.php"> <title>Data Warehousing Knowledge Base</title> <link>http://www.datamgmt.com/</link> <description>The Data Management & Warehousing Knowledge Base provides information and techniques about the design, build and implementation of data warehousing solutions that we as a company use and hope that you will also find useful.</description> <dc:date>2006-03-29T14:16:50+00:00</dc:date> <image rdf:resource="http://www.datamgmt.com/images/phpwsrssfeeds/thumbs/logo_tn.gif" /> - <items> - <rdf:Seq> <rdf:li resource="http://www.datamgmt.com/index.php?module=article&view=news" /> </rdf:Seq> </items> </channel> - <image rdf:about="http://www.datamgmt.com/images/phpwsrssfeeds/thumbs/logo_tn.gif"> <title>Data Warehousing Knowledge Base</title> <link>http://www.datamgmt.com/</link> <url /> </image> - <item rdf:about="http://www.datamgmt.com/index.php?module=article&view=76"> <title>Data Management & Warehousing White Papers</title> <link>http://www.datamgmt.com/index.php?module=article&view=76</link> <description>Data Management & Warehousing is publishing a series of white papers on topics relating to data warehousing. This article lists each paper and provides a synopsis<br />Updated:</description> </item> …

–  Already an inbuilt technology in most web browsers and mail clients –  Very cheap to modify existing reports to work with it –  Allows publish/subscribe to ‘news feeds’
•  These feeds would be reports by subject area •  e.g. BBC, most newspapers, etc.

–  An established technology already widely in use –  Can easily be integrated with textual content
Oracle Business Intelligence Thames Valley Park

© 2006 Data Management & Warehousing Speaker: David M. Walker

Page 24 of 30 30 March 2006

RSS - Really Simple Syndication
•  If I could offer you only one tip for the future, RSS would be it.
–  Already an inbuilt technology in most web browsers and mail clients –  Very cheap to modify existing reports to work with it –  Allows publish/subscribe to ‘news feeds’
•  These feeds would be reports by subject area

–  An established technology already widely in use
•  e.g. BBC, most newspapers, etc.

–  Can easily be integrated with textual content
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 25 of 30 30 March 2006

RSS - Really Simple Syndication
•  If I could offer you only one tip for the future, RSS would be it.
–  Already an inbuilt technology in most web browsers and mail clients –  Very cheap to modify existing reports to work with it –  Allows publish/subscribe to ‘news feeds’
•  These feeds would be reports by subject area

–  An established technology already widely in use
•  e.g. BBC, most newspapers, etc.

–  Can easily be integrated with textual content
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 26 of 30 30 March 2006

What to use RSS for
•  Publishing batch reports
–  By subject area –  By user community

•  •  •  • 

Publishing requirements Publishing analysis Publishing data quality issues and reports Publishing merged feeds
–  Reports & Data Quality issues together

•  Podcasting
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 27 of 30 30 March 2006

More than just reporting
•  Some of the biggest benefits come from the process of building the data warehouse and integrating it into the business
–  Builds a better understanding of what data is available –  What the data means to the organisation –  How it can be structured to make more sense across the whole organisation. –  Where information sits in the business process

© 2006 Data Management & Warehousing Speaker: David M. Walker

Oracle Business Intelligence Thames Valley Park

Page 28 of 30 30 March 2006

What do we want to be ?
•  A European Mobile Telco:
–  Data warehouse has over 150 Billion CDRs –  Over 2000 registered users

•  But: And:
–  Users know minutes to get data from to even the most –  It takes 20where to get basic answers quickly and effective. –  basic question which shouldtake totakeunderstand the They know how long a report will only run, seconds data quality and can subscribe to have it delivered to them –  Less than 100 people use it every day and they all –  100’s of people visiting the business information portal each day hatevery few directly using reporting tools, and 1000’s using the with the reporting tools data without and support –  Operationseven realising costs are soaring –  Operations, support costs and change through fast enough –  Can’t get changes to the system are targeted against the highest value returns
© 2006 Data Management & Warehousing Speaker: David M. Walker Oracle Business Intelligence Thames Valley Park Page 29 of 30 30 March 2006

Data Management & Warehousing
Thank you ! •  For more information:
–  Visit our website at http://www.datamgmt.com –  Call us on 07050 028 911 –  E-mail davidw@datamgmt.com

Winning Teams - Great Team Players Data Management & Warehousing are proud player sponsors for the 2005/06 season of Joe Worsley, utility back row with the English Rugby Premiership Champions London Wasps. Joe has helped London Wasps win the Zurich Premiership in 2002-03, 2003-04 and 2004-05 ©as wellManagement Heineken Cup in 2003-04. Joe was also a member of the England World Cup squad of 30 2006 Data as the & Warehousing Oracle Business Intelligence Page 30 Speaker: David M. Walker Thames Valley Park 30 March 2006 and was awarded an MBE by the Queen.