Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
5Activity
0 of .
Results for:
No results containing your search query
P. 1
Eight Challenges in Data Integration

Eight Challenges in Data Integration

Ratings: (0)|Views: 530|Likes:
Published by Charteris Plc

More info:

Published by: Charteris Plc on Oct 19, 2008
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

04/17/2011

pdf

text

original

 
 
©
2002 Charteris plc
EIGHT CHALLENGES IN DATA INTEGRATION Author: Robert P Worden
20 November 2002
 
 20 November 2002Draft 01Eight Challenges in Data Integration Page 2 of 12
CONTENTS
1.
 
LIMITATIONS OF DATA TRANSLATION TOOLS 3
 
2.
 
PROBLEMS OF MANAGEMENT AND COST 6
 
2.1
 
 The N-Squared Problem 6
 
2.2
 
 The N-fold Maintenance Problem 6
 
2.3
 
 The Double Knowledge Problem 6
 
3.
 
PROBLEMS OF INTRINSIC TRANSLATION CAPABILITY 8
 
3.1
 
 The Nesting Problem 8
 
3.2
 
 The De-Normalisation Problem 9
 
3.3
 
 The Uncommitted Language Problem 10
 
3.4
 
 The Data Grouping Problem 10
 
3.5
 
Bringing It Together 11
 
4.
 
 A WORKING SOLUTION TO THE EIGHT KEY PROBLEMS 12
 
 
 20 November 2002Draft 01Eight Challenges in Data Integration Page 3 of 12
1.
 
LIMITATIONS OF DATA TRANSLATION TOOLS
 The problem of application integration has been around for a long time. It is a very expensiveproblem, and most would say it has not been satisfactorily solved. Application integration is a large andever-increasing fraction of IT budgets – and is at the root of many project failures.For more than ten years, there have been specialist toolsets available for Enterprise ApplicationIntegration (EAI). These tools offer a range of facilities - for business process orchestration,transaction management, security, package ‘adaptors’ and so on. They all have facilities to translate databetween different applications and databases. As XML has become the preferred choice of inter-application ‘glue’ – both within and acrossorganisations, with or without the label of ‘web services’ – EAI tools have been extended to handleXML. There has also emerged a set of specialist XML translation tools.Most of these translation tools work in the same way. In a design phase, you define equivalencesbetween data items in two different data sources. For this, you use a process of field-to-field mapping. Then at run time, the mappings are used to make the data translations automatically. There are many of these field-to-field data mapping tools on the market. For instance:
 
 WebSphere Data Interchange from IBM
 
BizTalk Mapper from Microsoft
 
 Tibco Message Broker
 
Mercator Integration Broker
 
E-Biz Integrator from Sybase
 
GoXML from XML Global
 
Embarcadero D/T Designer
 
Data Mirror Transformation ServerField-to-field mapping works as follows: The design tool automatically captures the structure of somedata source (for a database, its relational schema; for an XML source, the nesting of elements from itsXML schema or DTD). This structure is displayed as a tree diagram. The nodes and leaves of the treeare ‘fields’ (e.g. columns in a database, or XML attributes) which hold the smallest items of data. You display the tree structures of two different data sources side by side. You drag-and-drop to tell thetool: ‘this field in source A is equivalent to that field in source B’ – drawing a line across, to denote amapping between the two fields. Maybe you put a box on the line to define some data translation (e.g.between different representations of dates) – using a palette of pre-defined translation functions, oradding custom functions.Once you have defined the mappings, run-time data translation is done automatically by the tool, or by code generated from the tool. Therefore you save the cost of hand-coding data translations.Data mapping would appear to be a big cost-saver. Based on that promise, over the years many peoplehave bought these mapping-style translation tools. However, these tools have not yet had a big impacton the practice of data integration; their use has remained localised. We know of no major IT user which regularly uses a single data integration tool for all its application integration needs. Thedominant method of data integration is still hand-coding. Why is this? This note describes how field-to-field mapping fails to tackle eight of the most importantchallenges of data integration. If these problems are not tackled properly, they come back to bite you,and force you back to hand-coding. That is possibly why the data mapping products have not achieved widespread use – in spite of having been around for many years. Three key challenges of data integration are problems of management and cost:

Activity (5)

You've already reviewed this. Edit your review.
1 hundred reads
1 thousand reads
TEJASKO liked this
suvransu4u liked this
lovelysunnyfru liked this

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->