You are on page 1of 30

Data Management & Warehousing

From Volume to Value:
What Next Generation Telco Data Warehouses
Must Do to Provide Value to the Business

David M. Walker
davidw@datamgmt.com

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 1 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
What do we have ?

•  A European Mobile Telco:
–  Data warehouse has over 150 Billion CDRs
–  Over 2000 registered users
•  But:
–  It takes 20 minutes to get answers to even the most
basic question which should only take seconds
–  Less than 100 people use it every day and they all
hate the reporting tools
–  Operations and support costs are soaring
–  Can’t get changes to the system through fast enough
© 2006 Data Management & Warehousing Oracle Business Intelligence Page 2 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
What are the engagement issues ?

•  It doesn’t have anything for my role
–  Don’t let the data warehouse sit outside a business process
•  Identify where a report changes/helps
–  People don’t need to know that they are using the data
warehouse
•  Ensure that it is integrated in their daily activities
–  Pro-actively educate people about what is available
•  Most people see data warehouses as something remote
•  I’ve already got a report that does this
–  Normally the response from the spreadsheet jockey
•  But how accurate is it and how long does it take to produce ?

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 3 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
What are the users issues ?

•  I don’t know what information is available!
–  Users are unwilling to search too hard for what is available
–  Users are unable to comment quickly and easily on what is
available
–  Users ‘just’ want ‘the right report’ fed to them
•  The report I got was wrong!
–  Published data profiles tell users where the data issues are
–  Helps users understand the requirement for data cleansing
back into operational systems (GIGO)
–  Nobody available to quickly modify a report to what the user
actually wants

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 4 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
What are the users issues (cont.) ?

•  I asked them for this new report and they told me it
would be two months!
–  Who helps users to understand what is already available ?
–  Who is available develop a report quickly?
•  I went on a training course for the reporting tool but
that was six months ago and I can’t remember how
to use it now!
–  If users are going to use a tool they should be frequent
users with someone to support them
–  If not, then provide resource to do the work for them

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 5 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Five things that can help

•  Exploitation/QuickService Teams

•  Data Profiling & Data Cleansing

•  Process Integration

•  Business Information Portals

•  RSS - Really Simple Syndication
© 2006 Data Management & Warehousing Oracle Business Intelligence Page 6 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
© 2006 Data Management & Warehousing Oracle Business Intelligence Page 7 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Exploitation/QuickService Teams

•  Have a (small) team that is ‘here to help’
–  Available via:
•  Telephone
•  Chat Room
•  Web Conference
•  Issue Tracking System
–  Technology reduces costs of running a team and
makes the data warehouse feel more accessible
•  Will respond quickly to urgent requests
–  Even if the answer is it will take sometime to fulfil the
total requirement here’s what we can do now

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 8 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Exploitation/QuickService Teams (2)

•  Look for heavy users and heavy queries and find ways to
help them
–  Cut out unused parts of the data warehouse
–  Optimise response of major users
–  Revise archiving strategy based on required data
–  Ultimately reduces operational and support costs
•  Visit frequent callers to the helpdesk
–  Re-enforce training
–  Gather new requirements
–  Pre-empt the need to call the helpdesk
•  But basically provide proactive support

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 9 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Direct Access
to the Quick
Service Team

What’s related
develops
understanding

Pre-empt
FAQ’s

Quick Service
Team can monitor
and respond
© 2006 Data Management & Warehousing Oracle Business Intelligence Page 10 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Data Profiling

•  Look at your source systems and understand what the
data quality issues are
–  Which required fields are not populated ?
–  Which fields always have a default value ?
–  Do all customers have sufficient contact details ?
–  Etc.
•  Detect and capture issues in the Data Warehouse
–  Often related to issues of integration across systems
•  Set up targets to improve the data quality
–  Especially in the source system
–  Publish the metrics and identify responsible owners

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 11 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Allow users to
see related
data quality

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 12 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
© 2006 Data Management & Warehousing Oracle Business Intelligence Page 13 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Data Cleansing

•  Fix data in the source systems
–  A data quality issue fixed in the source will have
benefits for other areas and often highlight business
process issues
•  Embed a call to the cleaning tool in all ETL
–  Rule based cleansers simple and easy to implement
–  Add the call even if there is no current requirement
–  Use a metadata driven tool so new rules can be
added
–  Track the success rate of the results
–  BUT: Maintain copies of the original data
© 2006 Data Management & Warehousing Oracle Business Intelligence Page 14 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Example Data Cleansing Issues

•  Standardisation of text
–  Prevents correct aggregation
–  Multiple spellings
•  e.g. Zürich, Zuerich, Zurich => Zurich, Rd => Road
–  Spaces
•  e.g. David_ _Walker is not the same as David_Walker
–  Standardization of case
•  E.g. David Walker => DAVID WALKER, Zurich => ZURICH
•  Range validation if dates
–  13-Mar-0006 becomes 13-Mar-2006
© 2006 Data Management & Warehousing Oracle Business Intelligence Page 15 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
More Data Cleansing Issues

•  Mapping of codes & translation
–  01 means ‘Fixed Line’ in one system and ‘Roaming’ in another
–  A code meant one thing for a period of time and then it’s use
was changed to mean another thing after a certain date
•  Overcome System Defaults
–  80% or all customers are MALE
•  Actually the default is MALE and most operators just tab over the
field
–  Date of Birth is nearly always empty
•  Optional field in source system – change to mandatory
–  Date of Birth is 01-01-1900
•  Mandatory field with no range checking and no option for declined

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 16 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Process Integration

•  Put the data warehouse into the process rather
than sitting to one side
–  Use it to allow customers to compare price plans
online
•  But restrict price plans available based on profile
–  Trigger on-line offers and customised content when
customers log into the website
–  Add web ‘popup’ pages to existing internal
applications
•  Call centre gets an ‘image’ of the person they are talking to

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 17 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Process Integration

•  Put the data warehouse into the process rather
than sitting to one side
–  Use it to allow customers to compare price plans
online
Price Plan: Prepay 5
Stop Churn: No
• Last
ButBill
restrict
(View):
price £0.50
plans available based on profile
–  Trigger
Last Callon-line
(Detail): offers and911customised content when
0118 9776
Last Contact: 20-Mar-2006
customers log
Next Best Offer: into
Textthe
packwebsite
50
Open Cases (View): 3
–  Add webCalls:
Dropped ‘popup’ pages
 to existing internal
Teenager
applications
Network Quality: 
Handset Type:  Texter 100of the person they are talking to
•  Call centre gets an ‘image’

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 18 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Process Integration

•  Put the data warehouse into the process rather
than sitting to one side
–  Use it to allow customers to compare price plans
online
Price Plan: Domestic 100
Stop Churn: Yes
•  Last
But Bill
restrict price plans
(View): available based on profile
£37.50
Last Call (Detail): 07990 594 372
–  Trigger on-line
Last Contact:
offers and customised content when
10-Jan-2003
customers log into Handset
Next Best Offer: the website
+£50
Open Cases (View): 0
–  AddDropped
web Calls:
‘popup’ pages  to existing internal
Single Working Female
applications
Network Quality: 
Handset Type:  Cheapo X79a
•  Call centre gets an ‘image’ of the person they are talking to

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 19 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Process Integration

•  Put the data warehouse into the process rather
than sitting to one side
–  Use it to allow customers to compare price plans
online
Price Plan: Business 350
• Stop
ButChurn:
restrict priceYes
plans available based on profile
Last Bill (View): £180
–  Trigger
Last Call on-line
(Detail): offers and
07050 028 911customised content when
Last Contact: 15-Sep-2005
customers
Next Best Offer:
log into the website
Handset +£0
–  Add
OpenwebCases‘popup’ pages to existing internal
(View): None
Dropped Calls:  Business User
applications
Network Quality: 
• Handset Type: gets 
Call centre anExecutive
‘image’3030
of the person they are talking to

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 20 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Business Information Portals

•  Single touch point
–  The delivery mechanism for all business information
services.
•  Collaboration
–  Allows users to communicate
•  Synchronously (through chat & messaging)
•  Asynchronously (through threaded discussion & email
digests)
•  Integration
–  The connection of functions and data from multiple
systems into new components

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 21 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Business Information Portals (2)

•  Content and document management
–  Services that support the full life cycle of document
creation and provide mechanisms for authoring,
approval, version control, scheduled publishing,
indexing and searching.
–  Consider a Wiki: a user editable webpage
•  Personalization
–  Allows users to subscribe (or be subscribed) to
specific types of content and services.
–  Users can also customize the look and feel of their
environment.

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 22 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
RSS - Really Simple Syndication

•  If I could offer you only one tip for the future,
RSS would be it.
–  Already an inbuilt technology in most web browsers
and mail clients
–  Very cheap to modify existing reports to work with it
–  Allows publish/subscribe to ‘news feeds’
•  These feeds would be reports by subject area
–  An established technology already widely in use
•  e.g. BBC, most newspapers, etc. & Podcasts
–  Can easily be integrated with textual content

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 23 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
RSS - Really Simple Syndication

•  If I could offer you only one tip for the future,
<?xml version="1.0" ?>
- <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns=http://purl.org/rss/1.0/>

RSS would be it.
- <channel rdf:about="http://www.datamgmt.com/files/phpwsrssfeeds/backend3.php">
<title>Data Warehousing Knowledge Base</title>
<link>http://www.datamgmt.com/</link>
<description>The Data Management & Warehousing Knowledge Base provides information and techniques about the design, build
–  Already an inbuilt technology in most web browsers
and implementation of data warehousing solutions that we as a company use and hope that you will also find useful.</description>
<dc:date>2006-03-29T14:16:50+00:00</dc:date>
and mail clients
<image rdf:resource="http://www.datamgmt.com/images/phpwsrssfeeds/thumbs/logo_tn.gif" />
- <items>
- <rdf:Seq>
–  Very cheap to modify existing reports to work with it
<rdf:li resource="http://www.datamgmt.com/index.php?module=article&view=news" />
</rdf:Seq>
</items>
–  Allows publish/subscribe to ‘news feeds’
</channel>
- <image rdf:about="http://www.datamgmt.com/images/phpwsrssfeeds/thumbs/logo_tn.gif">
<title>Data Warehousing Knowledge Base</title>
•  These feeds would be reports by subject area
<link>http://www.datamgmt.com/</link>
<url />
–  An established technology already widely in use
</image>
- <item rdf:about="http://www.datamgmt.com/index.php?module=article&view=76">
<title>Data Management & Warehousing White Papers</title>
•  e.g. BBC, most newspapers, etc.
<link>http://www.datamgmt.com/index.php?module=article&view=76</link>
<description>Data Management & Warehousing is publishing a series of white papers on topics relating to data warehousing.

–  Can easily be integrated with textual content
This article lists each paper and provides a synopsis<br />Updated:</description>
</item>

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 24 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
RSS - Really Simple Syndication

•  If I could offer you only one tip for the future,
RSS would be it.
–  Already an inbuilt technology in most web browsers
and mail clients
–  Very cheap to modify existing reports to work with it
–  Allows publish/subscribe to ‘news feeds’
•  These feeds would be reports by subject area
–  An established technology already widely in use
•  e.g. BBC, most newspapers, etc.
–  Can easily be integrated with textual content

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 25 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
RSS - Really Simple Syndication

•  If I could offer you only one tip for the future,
RSS would be it.
–  Already an inbuilt technology in most web browsers
and mail clients
–  Very cheap to modify existing reports to work with it
–  Allows publish/subscribe to ‘news feeds’
•  These feeds would be reports by subject area
–  An established technology already widely in use
•  e.g. BBC, most newspapers, etc.
–  Can easily be integrated with textual content

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 26 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
What to use RSS for

•  Publishing batch reports
–  By subject area
–  By user community
•  Publishing requirements
•  Publishing analysis
•  Publishing data quality issues and reports
•  Publishing merged feeds
–  Reports & Data Quality issues together
•  Podcasting
© 2006 Data Management & Warehousing Oracle Business Intelligence Page 27 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
More than just reporting

•  Some of the biggest benefits come from the
process of building the data warehouse and
integrating it into the business
–  Builds a better understanding of what data is available
–  What the data means to the organisation
–  How it can be structured to make more sense across
the whole organisation.
–  Where information sits in the business process

© 2006 Data Management & Warehousing Oracle Business Intelligence Page 28 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
What do we want to be ?

•  A European Mobile Telco:
–  Data warehouse has over 150 Billion CDRs
–  Over 2000 registered users
•  But:
And:
– 
–  It takes
Users 20where
know minutes
to gettobasic
get answers to even
data from quickly andthe most
effective.
–  basic question
They know which
how long should
a report onlytotake
will take seconds the
run, understand
data quality and can subscribe to have it delivered to them
–  Less than 100 people use it every day and they all
–  100’s of people visiting the business information portal each day
hate thefew
with very reporting toolsreporting tools, and 1000’s using the
directly using
–  Operations andrealising
data without even support costs are soaring
–  Can’t
–  Operations, support costs
get changes andsystem
to the change are targeted
through against
fast enoughthe
highest value returns
© 2006 Data Management & Warehousing Oracle Business Intelligence Page 29 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
Data Management & Warehousing

Thank you !

•  For more information:
–  Visit our website at http://www.datamgmt.com
–  Call us on 07050 028 911
–  E-mail davidw@datamgmt.com

Winning Teams - Great Team Players

Data Management & Warehousing are proud player sponsors for the 2005/06 season of
Joe Worsley, utility back row with the English Rugby Premiership Champions London Wasps.

Joe has helped London Wasps win the Zurich Premiership in 2002-03, 2003-04 and 2004-05
©as
2006well as the Heineken
Data Management & Warehousing Cup in 2003-04. Joe
Oraclewas also
Business a member of the England World Cup squad
Intelligence Page 30 of 30
Speaker: David M. Walker Thames Valley Park 30 March 2006
and was awarded an MBE by the Queen.