Welcome to Scribd. Sign in or start your free trial to enjoy unlimited e-books, audiobooks & documents.Find out more
Download
Standard view
Full view
of .
Look up keyword
Like this
1Activity
0 of .
Results for:
No results containing your search query
P. 1
Real-Time Data Integration the Lifeline for Enterprise Insight and Intelligence

Real-Time Data Integration the Lifeline for Enterprise Insight and Intelligence

Ratings: (0)|Views: 39|Likes:
Published by Deep

More info:

Published by: Deep on Sep 19, 2009
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

05/11/2014

pdf

text

original

 
n
r^
d
r^
aos
Chief Technology OfficerKnightsbridge Solutions
with a Data Integration Framework
I
\\J
Owwl
vI/L
that chaos prevals
in the average IT environment. Ambiguity, redundancy, andincompatibility add crippling complexity to our IT worlds,throwing practitioners off course from their long-term strategicobjectives — if these existed in the first place. If we broadenour definition of data integration to that of a data integrationframework.'" we can solve these perennial IT challenges.A aata integration framework greatly simplifies architecturesand creates a structLire within which data-centric applicationscan be btiiit the "right" way today
A Supplement to
DM Review Magazine
 
The Current Reality: Chaos in Your Universe
IT organizations exist in state of constantchange and ever-increasing complexity'; newpeople, legacy applications, time-to-marketpressures, financial constraints — a steadybarrage of urgent demands,
AH
of these createa jumble of great disorder and conftision,resulting in valiant efforts in reaction to thelatest fire drill. When technologists findthemselves unable to reconcile short-termversus long-term demands, the picture ofcomprehensibility-eroding complexity ispainted. Must we resign ourselves to thiscontinuous chaotic state? No.
The Ideal State: Conceptual Purity
The inverse ot chaos
is
comprehensibility.Fortimattly, the two extremes canand do coexist. Comprehensiblfcomplexity exists:
an
alternate uiiK'erse go\'emedby
conceptual
purity,
a Utopian state wherebusiness problems are fixed the "right" way.Consider four universal IT dimensions:organization, data, processes and platforms.For all comp!exit}'-inducmg dimensionswithin these,
a
conceptually
pure
version exists.For an IT organization, this translates into
o>-gamzational effkimcy
and
tneta-injbrmatioii.
Reining in Chaos: An Essential Ingredient
Meta-information lies at the core of:achieving organizational efficiency. Anaccurate and complete meta-mkirmationrepository' never grows stale or inconsistentwith physical implementation (data, process,platform), nor does it become inconsistentwith the flinction;il business view of theapplications
these
s^-stem
implemenrations sen'c.It contains historical informationabout pre\ioiis system versions— for example, what thesystem looked like lastchaotic puzzle into a more desired state otcomprehensibility, organizations are morelikely to stay on course with their overallnavigational path. Therefore, it
ii
possible tomeet imminent business requirements andultimately build IT systems the "right" way.
The
Physical Layer: The "What Is"
One can emision layers, where the bottomlayer constitutes physical applications. ITdirects the vast majorit\- of
its
attention andresources here. At this la\'er, objects can ver\'clearly be classified as data, process orplatform; they are our working systems. Forexample, relational databases, dimensionaldatabases and flat-file staging areas occupythe data dimension. Programs, jobs, scriptsand services occupy the process layer.
Data about Data
+
Data about Process
+
Data about Platform
=
Meta-lnformation
enterprises look toward the long term andmake the appropriate investments in it.Theyhave "one version of the truth" and enablepractitioners and business decision-makersto see the whole picture. Betbre designingnew systems, information technologistscomprehend
bow
current systems are builtand
w6nt
answers they were designed todeliver. It is understood that this level ofanalysis is the foundation for developinga clear, long-term IT strategy to ser\'e as aguidepost for future activities, ultimatelyhelping an organization move from a reac-tive nature to a proactive one.Complexity, therefore, is the obviousenemy of comprehensihility. One need notlook too hard in the typical IT organizationfor evidence that complexity' routinelyincreases over time. Yet an antithesis ofto operate, how often it failed andhow tast it performed. Above all else,the meta-infonmation repository completelydescribes the current implementation andfacilitates maintenance and new develop-ment efforts.
Bridging the Gap Between Two Worlds
We know thatcomplexit}'is not a desiredenvironment, but IT practitioners can resteasier knowing that neither is conceptualpurit)'. Why? Because a piirel)' king-term,strategic approach would entail a significantinvestment of resources, thereby ignoringthe immediate needs of the business. Thiswould place the business at a disadvantageagainst its competitors.Achieving a balance between the rwoextremes is the goal. By moving pieces of theStorage, servers, networks, and vendorsoftware occupy the platform layer.One consequence of so much attention atthe physical layer is increasing complexity'because nobody can see the completepicture. The complete picture means not)ust a logical or conceptual view of a singleapplication, but also a conceptual view of allsystems that participate in the data, processor platform dimension.
The Conceptual Layer: Where We'd Like To Be
At the opposite end of the stack from thephysical layer is the conceptually pure layer.There is one universal "database in theskv'" for all enterprise data. There is oneuniversal business rule repository'. Platforms
continued
on
page
6
A Supplement to
DM Review Magazine
 
continued rom
page
5
Data Integration Framework
Conceptual LayerMeta-information
Process Data 9 Ratform
are represented
by
abstract, interconnectedresource
pools.
The conceptual
view is
wherewe all want
to
be.
We may never actually getthere,
but it is
what
we
would like
our IT
world
to
look like.
The Integration Layer: Bridging
the
Two
The conceptual layer must then
be
"connected"
to the
physical
layer.
This
is the
role
of the
integration
layer.
The integrationlayer says, "Here's what
we
have today
and
how
it
maps into
a
clean, simple view
ot
our enterprise." The standards
and
businessrules contained
in
this simple view
arc
verysimilar to
the
conceptual view.The trick
is to be
able
to
migrate systemsfrom
the
physical layer into
the
integrationlayer over time. For example,
the
integrationlayer
has a
unified view of "customer."
For a
data mart
to
move
up
into
the
integrationlayer,
it
must first coiilorm
to
this unified
Over time, moredimensions willconform and
the
organizationmoves towardits desired state.
: Nonconforming dimensionsstay
in the
physical layerConforming dimensions residein
the
integration layer; Dimensions
of new or
existingprocesses, data
or
platformsthat conform reside
in the
inte-gration layer, while those thatdon't stay
in the
physical layer
view.
If it
cannot,
it
stays
in the
physic.illayer,
and
elements
of it
that
map to the
unified view are connected
to
the integrationlayer. Noncontorming elements
do not
appear
in the
integration layer. Over time,more dimensions
map to the
integration
layer,
bringing
us
closer
to
the conceptual view.Without this integration layer,
one
lacksthe compass by which
to
navigate changes
in
the physical layer.
For
example,
a
businessuser requests access
to
certain types of infor-mation.
IT
detennines that this intormationcan only
he
delivered with
a new
"tempo-rary" data mart.
It's
temporary because
we
intend
to
extend
the
data warehouse
to
include
the
subject area, and we'll propagatethat subject ;irea into
a
mart that alreadyexists
at
some future date.
The
temporarybypass of the warehouse
and
direct access
to
source systems
to
collect
the
data needed
for
tbis temporary mart creates complexity.Unfortunately,
not
unlike
a
"temporary"
tax,
the "temporary'" data mart turns
out to he
not very temporary'.
Not
only does
it
existeven though
the
warehouse
has
beenextended, while
the
warehouse
was
being
extended,
tlie
definition
oi tliis
m;irt gniw
as
well.The business users gain benefit fromthe "temporary mart"
and are not
willingto tlind
its
replacement. They would ratherspend that money
on
another project theyfeel
is
more important. Many enterprise
IT
systems
are
collections
of
such "temporary"systems.In this example,
the
warehouse
may fit
our definition
ot
conceptually pure data.The dara warehouse moves into
the
dataintegration
layer.
The new "temporary" martstays
in
the physical
layer.
Business rules thattransform
and
cleanse data bound
for the
warehouse also move into
the
integrationlayer.
The
platform upon which
the
warehouse runs
may
also move into
the
integration layer.
The Integration Layer
as
Data Integration Framework
Tiie integration layer
is a
framework.More specifically,
it is a
data integrationframework containing information about data,process
and
platform. Meta-informationis
the
glue that binds data, process
and
platform together.Generally, data integration
is
regardedas
a
connector
of
data consumers
and
producers
in the
physical layer. Data movesamong transactional systems
and
then
to
operational data stores
or
data warehouses.From warehouses, data moves
to
marts
-
continued on page
S
A Supplement
to
DM
Review Magazine

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->