Data Warehouse Testing

By :
Kartikey Brahmkshatriya
(M.C.A)
Index
...............................................................................................................................................2
Introduction.............................................................................................................................3
About Data Warehouse...........................................................................................................3
Data Warehouse definition.................................................................................................3
Testing Process for Data warehouse:......................................................................................3
Requirements Testing :......................................................................................................3
Unit Testing : ....................................................................................................................
Integration Testing : ..........................................................................................................
!cenarios to be co"ered in Integration Testing...............................................................#
$a%idating the Re&ort data..............................................................................................'
User Acce&tance Testing....................................................................................................'
(onc%usion..............................................................................................................................'
Introduction
This document details the testing process involved in data warehouse testing and test coverage
areas. It explains the importance of data warehouse application testing and the various steps of
the testing process.
Aout Data Warehouse
Data warehouse is the main repository of the organization's historical data. It contains the data
for management's decision support system. The important factor leading to the use of a data
warehouse is that a data analyst can perform complex queries and analysis (data mining) on the
information within data warehouse without slowing down the operational systems.
Data Warehouse definition
Subject-oriented : u!"ect #riented $Data warehouses are designed to help you analyse
data. %or example& to learn more a!out your company's sales data& you can !uild a
warehouse that concentrates on sales. 'sing this warehouse& you can answer questions
li(e )*ho was our !est customer for this item last year+) This a!ility to define a data
warehouse !y su!"ect matter& sales in this case& ma(es the data warehouse su!"ect
oriented. The data is organized so that all the data elements relating to the same real$
world event or o!"ect are lin(ed together.
Integrated : Integration is closely related to su!"ect orientation. Data warehouses must
put data from disparate sources into a consistent format. The data!ase contains data
from most or all of an organization's operational applications and is made consistent.
Time-variant : The changes to the data in the data!ase are trac(ed and recorded to
produce reports on data changed over time. In order to discover trends in !usiness&
analysts need large amounts of data. , data warehouse's focus on change over time is
what is meant !y the term time variant.
Non-volatile : Data in the data!ase is never over$written or deleted& once committed&
the data is static& read$only& !ut retained for future reporting. #nce entered into the
warehouse& data should not change. This is logical !ecause the purpose of a data
warehouse is to ena!le you to analyse what has occurred.
Testing !rocess "or Data #arehouse:
Testing for a Data warehouse consists of requirements testing& unit testing& integration testing
and acceptance testing.
Requirements Testing :
The main aim for doing -equirements testing is to chec( stated requirements for
completeness. -equirements can !e tested on following factors.
.. ,re the requirements /omplete+
0. ,re the requirements ingular+
1. ,re the requirements ,m!iguous+
2. ,re the requirements Developa!le+
3. ,re the requirements Testa!le+
In a Data warehouse& the requirements are mostly around reporting. 4ence it !ecomes more
important to verify whether these reporting requirements can !e catered using the data
availa!le.
uccessful requirements are those structured closely to !usiness rules and address
functionality and performance. These !usiness rules and requirements provide a solid
foundation to the data architects. 'sing the defined requirements and !usiness rules& high
level design of the data model is created. #nce requirements and !usiness rules are
availa!le& rough scripts can !e drafted to validate the data model constraints against the
defined !usiness rules.
Unit Testing :
'nit testing for data warehouses is *4IT56#7. It should chec( the 5T8
procedures9mappings9"o!s and the reports developed. This is usually done !y the
developers.
'nit testing will involve following
.. *hether 5T8s are accessing and pic(ing up right data from right source.
0. ,ll the data transformations are correct according to the !usiness rules and data
warehouse is correctly populated with the transformed data.
1. Testing the re"ected records that don:t fulfil transformation rules.
Integration Testing :
,fter unit testing is complete& it should form the !asis of starting integration testing.
Integration testing should test out initial and incremental loading of the data warehouse.
Integration testing will involve following
.. equence of 5T8s "o!s in !atch.
0. Initial loading of records on data warehouse.
1. Incremental loading of records at a later date to verify the newly inserted or updated
data.
2. Testing the re"ected records that don:t fulfil transformation rules.
3. 5rror log generation.
The overall Integration testing life cycle executed is planned in four phases; -equirements
'nderstanding& Test <lanning and Design& Test /ase <reparation and Test 5xecution.
$cenarios to e co%ered in Integration Testing
Integration Testing would cover 5nd$to$5nd Testing for D*4. The coverage of the tests
would include the !elow;
1. Count Validation
$ -ecord /ount =erification D*4 !ac(end9-eporting queries against source and
target as a initial chec(.
2. Source Isolation
$ =alidation after isolating the driving sources.
3. Dimensional nal!sis
$ Data integrity !etween the various source ta!les and relationships.
". Statistical nal!sis
$ =alidation for various calculations.
#. Data $ualit! Validation
$ /hec( for missing data& negatives and consistency. %ield$!y$%ield data verification
can !e done to chec( the consistency of source and target data.
%. &ranularit!
$ =alidate at the lowest granular level possi!le (8owest in the hierarchy 5.g.
/ountry$/ity$treet > start with test cases on street).
'. (t)er validations
$ ?raphs& lice9dice& meaningfulness& accuracy
.
@, Team -eviews 6-D for
completeness.
@, Team !uilds Test <lan
Develop Test /ases and
@8 @ueries
Test 5xecution
<rocess for Data warehouse Testing
Business &e'uirement
Document(&e'uirement
Traceai)ity Matrix
Business &e'uirement
Document(&e'uirement
Traceai)ity Matrix
&e'uirements Testing
&e'uirements Testing
*igh +e%e) Design
document
*igh +e%e) Design
document
&e%ie# o" *+D
&e%ie# o" *+D
Test Case !re,aration
Test Case !re,aration
-unctiona) Testing
-unctiona) Testing
&egression Testing
&egression Testing
!er"ormance Testing
!er"ormance Testing
.ser Acce,tance Testing (.AT)
.ser Acce,tance Testing (.AT)
.nit Testing
.nit Testing
/a)idating the &e,ort data
#nce the 5T8s are tested for count and data verification& the data !eing showed onto the
reports hold utmost importance. @, team should verify the data reported with the
source data for consistency and accuracy.
.. =erify -eport data with source
$ ,lthough the data present in a data warehouse will !e stored at an aggregate level
compare to source systems. 4ere the @, team should verify the granular data
stored in data warehouse against the source data availa!le.
0. %ield level data verification
$ @, team must understand the lin(ages for the fields displayed in the report and
should trace !ac( and compare that with the source systems.
1. /reating @8s
$ /reate @8 queries to fetch and verify the data from ource and Target.
ometimes it:s not possi!le to do the complex transformations done in 5T8. In such
a case the data can !e transferred to some file and calculations can !e performed.
User Acceptance Testing
4ere the system is tested with full functionality and is expected to function as in production. ,t
the end of ',T& the system should !e accepta!le to the client for use in terms of 5T8 process
integrity and !usiness functionality and reporting.
Conc)usion
5volving needs of the !usiness and changes in the source systems will drive continuous change
in the data warehouse schema and the data !eing loaded. 4ence& it is necessary that
development and testing processes are clearly defined& followed !y impact$analysis and strong
alignment !etween development& operations and the !usiness.