Welcome to Scribd. Sign in or start your free trial to enjoy unlimited e-books, audiobooks & documents.Find out more
Standard view
Full view
of .
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
Sample Solution

Sample Solution

|Views: 241|Likes:
Published by api-3733985

More info:

Published by: api-3733985 on Nov 22, 2009
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
download as DOC, PDF, TXT or read online from Scribd
See more
See less





Proposed Solution
Current System

TCS understands that the current system of Lousiana Department of Revenue (LDR) has different applications like tax administration,budget development, and performance measurement which reside in different relational database like DB2, SQL Server, MS Access and non relational data sources (sequential, VSAM files) All the Operational data is lying in different isolated organisational silos. The information for various parts of the business are held either within the division or/and the Data Warehouse.

The LDR captures administrative and general information. This data could be used for various purposes but currently being under-utilised in absence of a centralised architecture and integrated information approach.

There is an absence of information governance with regards to access to data within the data warehouse. The Data Quality within some source systems is not up to the mark.

The LDR aims to build a highly accessible, intuitive, single point of access to the information
contained within the LDR\u2019s various applications. In essence the different application data
needs to be integrated into this common Enterprise wide structure to model and
support the business management process.

The data pertaining to all organizational entities and stakeholders will be held at the lowest level of granularity possible to maximize the number and type of management information requirements that can be satisfied. The DWH will initially source from within its own systems but in the future will look to enhance this data with third party information providers.

The future requirements of the client cannot be met by the present system.
Solution Architecture

To build a single and coherent common information platform for LDR, TCS proposes a solution based on SQL Server 2005 at the back end and utilising the, COGNOS Reporting Services and Integration Services bundles of SQL Server 2005 to aid in the enterprise wide reporting as a medium to make this information available to the end users,

The figure below depicts the overall architecture for the solution:

Microsoft SQL Server 2005 Integration Services (SSIS) will be used for extraction, transformation, and loading (ETL) of data from various data sources to the staging database and from there to enterprise data warehouse. Cognos 8 Framework Manager/Report Studio will be used to build/generate different kinds of prepared reports from Data Mart/Enterprise Data warehouse.

Task flow and data flow engine

SSIS consists of both an operations-oriented task-flow engine as well as a scalable and fast data-flow engine. The data flow exists in the context of an overall task flow. It is the task-flow engine that provides the runtime resource and operational support for the data-flow engine. This combination of task flow and data flow enables SSIS to be effective in l ETL or data warehouse (DW).

Pipeline architecture

At the core of SSIS is the data transformation pipeline. This pipeline has a buffer- oriented architecture that is extremely fast at manipulating rowsets of data once they have been loaded into memory. The approach is to perform all data transformation steps of the ETL process in a single operation without staging data,

although specific hardware requirements need to be addressed. Nevertheless, for maximum performance, the architecture avoids staging. Even copying the data in memory is avoided as far as possible. With SSIS, all types of data (structured, unstructured, XML, etc.) are converted to a tabular (columns and rows) structure before being loaded into its buffers. Any data operation that can be applied to tabular data can be applied to the data at any step in the data-flow pipeline. This means that a single data-flow pipeline can integrate diverse sources of data and perform arbitrarily complex operations on these data without having to stage the data.

It should also be noted though, that if staging is required for business or operational
reasons, SSIS has good support for these implementations as well.
SSIS for Datawarehouse loading

SSIS can consume data from (and land data into) a variety of sources including OLE DB, managed (ADO.NET), ODBC, flat file, Excel, and XML using a specialized set of components called adapters. SSIS can even consume data from custom data adapters (developed in-house or by third parties). This allows the wrapping of legacy data loading logic into a data source that can be seamlessly consumed in the SSIS data flow. SSIS includes a set of powerful data transformation components that allow data manipulations that are essential for building data warehouses. These transformation components include:

Aggregate: Performs multiple aggregates in a single pass.
Sort: Sorts data in the flow.
Lookup: Performs flexible cached lookup operations to reference datasets.
Pivot and UnPivot: Two separate transformations
Merge, Merge Join, and UnionAll: Can perform join and union operations.

Derived Column: Performs column-level manipulations such as string, numeric,
date/time, etc. operations, and code page translations.
Data Conversion: Converts data between various types (numeric, string, etc.).
Audit: Adds columns with lineage metadata and other operational audit data.

In addition to providing a professional development environment, SSIS exposes all its functionality via a set of rich APIs. These APIs are both managed (.NET Framework) and native (Win32) and allow developers to extend the functionality of SSIS by developing custom components in any language supported by the .NET

Activity (12)

You've already reviewed this. Edit your review.
1 thousand reads
1 hundred reads
mona257 liked this
akashakas liked this
nidhinrajcs liked this
allan237 liked this
Mahesh Kumar liked this
NelaNastasia liked this

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->