Concepts, Technologies

,
and Best Practices
Beth Gold-Bernstein & Gary So
Integration and
SOA
next
© 2006
2
About webMethods
Get There Faster.
webMethods provides total business integration to the world’s largest corporations and
government agencies. webMethods fagship product suite, webMethods Fabric, leverages
leading SOA and BPM technologies and methodologies to deliver rapid ROI to our 1,300
customers around the globe. With webMethods, customers can take a process-centric
approach to their business problems, allowing them to re-use their existing IT assets,
dramatically improve business process productivity and ROI, and rapidly create competitive
advantage by making their business processes work harder for their company. webMethods
(NASDAQ: WEBM) is headquartered in Fairfax, VA, USA and has offces throughout the United
States, Europe, Asia Pacifc and Japan.
For more information on our company and our products, we invite you to visit our website,
www.webMethods.com
View our SOA webinar series – www.webMethods.com/Events/Web/GetRealwithSOA
Learn more about us – www.webMethods.com/About
Hear our vision – www.webMethods.com/Vision
Meet our customers – www.webMethods.com/Customers
View our products – www.webMethods.com/Products
If you would like additional copies of the book, please email soabook@webMethods.com
About ebizQ
ebizQ – The Insiders Guide to SOA and Integration
ebizQ is the leading community of business and IT executives, managers, architects and
developers focused on business integration and SOA. ebizQ’s analyst-centric multimedia
content includes interactive training, webinars, and analysis of industry trends and best
practices. You can access daily aggregation of in-house and industry-wide blogs, breaking
news, feature articles and white papers.
Learn more by visiting www.ebizQ.net and gain access to the following great content!
Webinars: www.ebizQ.net/webinars/
White Papers: www.ebizQ.net/white_papers/
Online Curriculum: www.ebizQ.net/trainingcenter
Blogs: www.ebizQ.net/blogs
CIO audio: www.ebizQ.net/executive_corner/cioaudio/
Buyers Guide: www.ebizQ.net/vendors
Bring the rest of your team up to speed. Attend our online companion course, a Manager’s
Guide to Integration and SOA: www.ebizQ.net/managersguide or call
1-866-55-EBIZQ (1-866-55-32497)
3
Tables of Contents
Chapter 1 The Business Imperative for Integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Executive Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Challenges to Business Agility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
The Mandate for Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Integration and SOA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
The Evolution of Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Integration: A Business Imperative . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
How to Use This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Chapter 2 Building the Business Case for Integration . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Executive Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Tangible and Intangible Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Areas for ROI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Defining Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Chapter 3 Common Integration Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Executive Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Data Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Enterprise Application Integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Mainframe Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
B2B Integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Single View of Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Multichannel Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Composite Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Business Process Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Business Performance Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Best Practices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4
Chapter 4 Guide to Integration Technologies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Executive Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Application Integration Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Information Integration Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Interface Integration Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Composite Application Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Process Integration Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Business Optimization Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Compliance and Industry Solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Management and Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Architecture and Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Chapter 5 Integration and SOA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Executive Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Why SOA? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
SOA Explained . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
The Role of Web Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Enterprise-Class SOA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Best Practices in SOA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Chapter 6 Best Practices for Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Executive Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Common Pitfalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Taking a Strategic Approach to Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Organizing for Success . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Responsibilities of the ICC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Creating an ICC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Critical Success Factors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
5
Executive Overview
S
ometime around 500 B.C., the Greek philosopher Heraclitus said, “Nothing endures
but change.” While change may have been a constant since time immemorial, the
rate of change is accelerating far faster than ever before, and this is having a profound
effect on business. Business cycles are shrinking rapidly. The way business was con-
ducted even a decade ago is no longer acceptable if an organization wishes to remain
competitive. Organizations have had to change how they interact with customers, how
they manufacture goods, and how they are organized and managed.
Rapid changes are only possible when the organization itself is agile. The notion of
an agile business has long captured the imagination of business executives. The agile
business is able to embrace changes in market conditions, organizational structure,
and the regulatory environment without missing a beat. An agile business empow-
ers the management team to focus its collective acumen on delivering substantially
increased value to its stakeholders.
The desire for greater business agility is making integration increasingly important.
Agility is the combination of speed and adaptability: speed to bring new solutions to
market more quickly, and adaptability to new business requirements and competitive
pressures. This can only be achieved when business processes are able to be changed
nimbly. And that in turn is possible only when the underlying IT systems are integrated
flexibly to accommodate the speed of change required.
Integration also enables organizations to leverage existing IT investments to stream-
line their processes for greater efficiency and productivity. IT infrastructures need to
utilize existing applications and systems to the extent possible. This requires an inte-
grated infrastructure along with end-to-end visibility of the business processes across
disparate systems. It also requires an approach that supports the implementation of
business solutions from existing components. This is the idea behind Service Oriented
Architecture, or SOA. (See Chapter 5, Integration and SOA.)
The Business Imperative for Integration
1
6
Chapter 1
The Business Imperative for Integration
Challenges to Business Agility
Currently, business agility is constrained by a number of obstacles. These include inflex-
ible applications that cannot be easily changed or enhanced; non-integrated stovepipe
applications; inefficient business processes; a lack of visibility into business processes
and operations; challenges brought about by mergers, acquisitions, and regulations;
and a lack of alignment between an organization’s strategic objectives and daily op-
erations. Integration technology can play a role in removing these obstacles. In fact, in
today’s business climate, organizational growth may be inhibited without integration.
Integration technology is a key enabling factor in helping business and IT executives
transform their organizations; get to market more quickly; respond faster to business
opportunities, competitive pressures, and regulatory requirements; and differentiate
how their organizations do business. Business responsiveness has become a function
of an organization’s ability to rapidly marshal the underlying IT systems in alignment
with business needs. This means leveraging existing assets while creating new busi-
ness functionality.
The Mandate for Integration
One only needs to look at the history of computing and the evolution of business soft-
ware to see why integration is a priority for most chief information officers (CIOs) today.
In the beginning, back-office operations were run by mainframe systems. These sys-
tems were built to optimize expensive computing resources, but not business agility.
They were—and remain—difficult to change and brittle to boot: Changing one thing
could easily break something else. The legacy systems still in place today generally run
an organization’s core back-end operations in a robust and reliable way. However, they
are usually batch systems and are unable to respond flexibly to business needs for real-
time information or new functionality.
That unfulfilled business need gave rise to the first wave of distributed computing with
the emergence of minicomputers. Department managers could purchase them with
their budgets and select them based on their particular needs. Often, the availability of
7
Chapter 1
The Business Imperative for Integration
a packaged application that met 50% to 80% of the department’s specific needs led to
the introduction of these new platforms.
Unix, PCs, and client server software further reduced the cost of department
systems, and they began to proliferate throughout organizations. Low-cost desktop
productivity and development tools—such as Microsoft Excel, Access, and Visual Ba-
sic—supported ad hoc solutions, which also proliferated and often became more stra-
tegic than they were originally designed to be. Individual business units established
their own computing facilities and application development capabilities, in effect set-
ting up shadow IT organizations. This gave them the desired level of agility—at least at
first—to respond more rapidly and independently to new business needs. However, it
also led to islands of automation: hundreds of applications spread across the organiza-
tion, many of them on desktops. These applications used organizational information
and were in turn used to report on business operations. But they were not under orga-
nizational IT management and were certainly not integrated consistently.
The emergence of the Internet and the associated rush to e-business further punctu-
ated the need for organizations—and, by association, IT—to become more responsive
to market dynamics. But with different departments launching their own initiatives
and often duplicating one another’s work, the result was reduced visibility and control,
along with reduced economies of scale.
The widespread adoption of distributed systems often meant that large organizations
had multiple platforms running hundreds of applications that managed similar infor-
mation through different portions of various business processes. Unfortunately, these
applications were not designed to integrate with one another, so the organizations
had to find ways to keep the information in synch across the systems. Rekeying the
data—a means of last resort, but all too often the approach taken—was slow, resource-
intensive, and prone to errors that could be costly to trace and resolve. So an easier way
to integrate the disparate stand-alone systems was needed.
Another problem was that each packaged application was designed to focus on spe-
cific department processes. Business agility requires the optimization of business pro-
cesses end-to-end. This includes improving the process for initiators such as customers,
8
Chapter 1
The Business Imperative for Integration
partners, and suppliers as well as people in different business roles, and developing
systems that support different parts of the business.
While some emerging technologies look for a business problem to address, integration
was a business problem long before it was a technology. Packaged software systems
were far from being turnkey solutions. The integration costs of implementing the typi-
cal enterprise resource planning (ERP) system could be three to five times the cost of
the software. The problem was that the integration involved point-to-point hand-cod-
ing. This required an understanding of the application program interfaces (APIs) of all
the systems to be integrated and a high level of expertise. It also took a great deal of
time and was inflexible to change. The number of interfaces rose exponentially with
the number of systems being integrated. Such integration “spaghetti” was difficult to
manage and change, and upgrading to a new version of any of the applications meant
going through the process all over again.
This brings us to the present. Although the current state of affairs is largely the result of
short-term business decisions, it is ultimately viewed as an IT problem. That’s because
business change is intimately tied to the underlying IT systems’ flexibility—or lack
thereof. As markets move quickly and new opportunities and competitors emerge, or-
ganizations are increasingly challenged to close the gap between their business needs
and the lack of flexibility in their IT infrastructures. Perhaps the biggest obstacle to
closing this gap is that change has to be implemented while the core IT operations that
support the business continue to function smoothly and seamlessly. The challenge is
akin to upgrading the wings of a plane while it’s in flight. This is where Service Oriented
Architecture enters the picture.
9
Chapter 1
The Business Imperative for Integration
Case Study 1-1
Company: RBank
Company Description
RBank is a pseudonym for the retail banking subsidiary of one of the nation’s largest financial
services companies. It provides investment management, retail and commercial banking, re-
tirement, consumer finance, and investment banking products and services to individuals and
organizations throughout the United States and, for certain businesses, internationally.
Business Challenges
RBank had more than 60 different customer response systems and applications, and all
customer service requests had to be routed to specific lines of business—a process involving
several steps—before the customer was directed to the appropriate representative. RBank
also faced challenges complying with an increasing number of federal laws. The Patriot Act,
for example, expanded anti-money-laundering requirements through provisions to deter illicit
financial activities that support the financing of terrorism. RBank needed to ensure that it was
submitting the appropriate customer information to its agency in a timely manner to meet the
compliance requirements.
How Integration Addressed the Challenges
RBank’s enterprise architecture division adopted an SOA and the webMethods Fabric inte-
gration platform to build applications faster and cheaper. RBank representatives now have
comprehensive information on all customer interactions with the bank on one screen, making
the customer experience much more positive. Using the integration solution, RBank employees
can make changes to customer information, such as address updates, in one place and have
them propagated and synchronized across all customer systems. RBank also used the new
integration solution to automate previously manual processes, including customer loan ap-
plications. Applications are now routed through the integration platform to RBank’s financial
systems for processing, and decisions are returned to the customer in real-time.
The SOA approach and integration platform solution has also played a critical role in compli-
ance efforts. RBank can now connect directly to its agency to transmit all the required cus-
tomer information and manage the verification process in real-time.
Bottom-Line Business Benefits
The 360-degree customer view has enabled RBank to customize services to customers as
well as to cross-sell and up-sell offerings, improving customer satisfaction and growing sales
opportunities. By automating processes and eliminating error-prone manual steps, RBank has
realized substantial cost savings.
10
Chapter 1
The Business Imperative for Integration
Integration and SOA
At the core, SOA is about creating systems out of standard building blocks. The concept
is not new. Many of the principles underlying SOA, such as isolating functionality to
promote reusability, are long-standing best practices. In the past, however, the adop-
tion of SOA was hindered by a lack of widely accepted standards for putting together
these building blocks. Now, the pressing demands of business—which necessitate in-
creased system flexibility and adaptability—have inspired nearly all organizations to
align on a single set of standards for SOA: Web services.
While SOA involves much more than Web services, standardization is largely respon-
sible for removing the barriers to SOA and fostering a future where systems can be
more easily assembled and incrementally modified. Today, in the face of growing com-
petitive pressure and the accelerating pace of business, organizations realize that they
run a risk if they do not move toward SOA.
So how are SOA and integration related? In fact, they are highly complementary.
SOA is inherently about a distributed architecture, with systems that span computing
platforms, data sources, and technologies. A distributed architecture requires integra-
tion. By standardizing how systems interoperate, Web services simplify the task of in-
tegration. Web services alone, however, do not suffice. Organizations need an evolu-
tionary approach to SOA that incorporates legacy (non-Web-services-based) systems.
Integration software provides the bridge between the legacy systems and SOA, allow-
ing organizations to leverage existing software assets while managing their transition
to SOA.
Integration solutions also contribute mature technologies—such as messaging, rout-
ing, data translation and transformation, and event management—along with organi-
zational disciplines that are necessary for full-fledged, enterprise SOA. (See Chapter 4,
Guide to Integration Technologies.) Moreover, integration capabilities such as business
process management (BPM) and business activity monitoring (BAM) allow organiza-
tions to realize a higher level of business productivity from SOA by enabling the opti-
mization of business processes and the alignment of strategic objectives with opera-
tional actions. In short, integration should play a central role in any organization’s SOA
strategy.
11
Chapter 1
The Business Imperative for Integration
The Evolution of Integration
An organization’s IT history remains its present. Organizations seldom have the luxury
of starting with a clean slate. With the vast investments that organizations have made
in systems and infrastructures, CIOs have an overriding financial imperative to leverage
existing resources to support new business requirements. In addition, it is very risky to
rip out and replace complex systems that serve mission-critical business operations.
So new or modified business services need to be delivered rapidly without affecting
current operations. They also need to maximize the use of in-place assets, with all their
associated constraints.
Integration software has emerged as an effective and viable approach to bridging
organizational silos. But so far organizations have had only modest success in truly
leveraging their existing in-place application assets—whether legacy or packaged ap-
plications—to support new business initiatives. To successfully deliver business agility,
organizations must evolve their definition of integration and its application to their
business objectives. No longer only about integrating systems, the role of integration
now also encompasses quickly assembling new solutions for the business, and moni-
toring and optimizing an organization’s business processes and operations.
Integrate. To manage the full scope of the business and to increase efficiencies,
organizations must integrate previously autonomous business processes and
their underlying applications. Integration software on its own is insufficient.
At best, it will only allow organizations to streamline their existing processes.
Business agility requires the ability to extend existing IT assets as reusable ser-
vices. This is the concept behind SOA. Not only does it enable organizations
to leverage the investments they have already made, but it promotes reuse,
with all the associated benefits of reduced cost and improved quality. The new
world order, therefore, demands an SOA-based approach to integration.
AssembIe. Putting in place an SOA-based integration foundation enables the
rapid assembly of new solutions from the catalog of available services. This
ability to decompose monolithic applications into reusable business services,
and then to recompose them as needed, contributes directly to business agil-
ity by enabling organizations to rapidly change and assemble new business
processes to support changing business priorities. When combined with BPM
4
4
12
Chapter 1
The Business Imperative for Integration
to enable processes to be composed from existing building blocks, SOA allows
operational changes to be delivered faster, cheaper, with a higher degree of
quality and with a higher return on assets.
Monitor. SOA also makes it uniquely possible for organizations to monitor ev-
ery aspect of their key business processes and gain insight into how they can
optimize these business processes. By using the services within an SOA-based
system as measurement points, organizations can gain unprecedented visibil-
ity into the metrics—such as order processing times, average order amount,
and so on—that reflect the operational and business performance of a pro-
cess.
Dptimize. Then, by comparing these real-time performance metrics against
previously defined key performance indicators, organizations can create a
feedback loop that facilitates iterative improvement of their processes—for
example, by helping identify and reduce the bottlenecks in an order-to-cash
workflow. Again, SOA plays a role by simplifying and speeding an organiza-
tion’s ability to make the desired business process changes, or implement
the necessary system solutions, to drive these improvements. This ability to
iteratively improve the way the business operates ultimately results in greater
business process productivity for the organization.
Integration: A Business Imperative
Business agility is becoming a strategic goal to organizations in every industry, and
in both the private and public sectors. The bottom line is that IT must be able to re-
spond more quickly to changing business needs, opportunities, and threats. It is a mat-
ter of survival. As discussed in this chapter, agility is enabled through an integration
infrastructure implemented with SOA principles. The glue that binds together all these
business goals and implementation strategies is integration. Organizations need to
create core competencies around integration and SOA, much the way they did with
databases and data centers in the 1980s and ’90s. Integration is nothing less than a
core requirement for enabling business agility.
4
4
13
Chapter 1
The Business Imperative for Integration
Case Study 1-2
Company: Electro
Company Description
Electro is a pseudonym for a leading global distributor of electronic components and comput-
er products. With more than 100,000 customers and 300 suppliers, it markets, distributes, and
adds value to a wide variety of electronic components, enterprise computing products, and
embedded systems. The company is positioned between multiple tiers of customers—value-
added resellers (VARs), contract manufacturers, and end users—and a large supplier base.
Business Challenges
One of Electro’s major suppliers mandated that its software customers buy through the
distribution channel. This forced multiple VARs to quickly choose a distribution partner for
order fulfillment because they could no longer buy directly from the manufacturer. Most of the
VARs used EDI for business, and they needed to select and migrate their current EDI ordering
processes to a new partner within 60 days. Electro saw the opportunity to provide drop-in
integration capability and capture the substantial revenue stream. To capitalize on the oppor-
tunity, however, it needed to quickly put in place a solution that would connect the two ends
of the supply chain by allowing each partner to leverage its existing EDI solution.
How Integration Addressed the Challenges
Using an out-of-the-box EDI solution from webMethods, Electro accelerated the creation of an
EDI framework so potential VAR customers could connect for fulfillment. The company lever-
aged its existing webMethods integration infrastructure, which contained many key integra-
tion points and core services necessary to construct a solution rapidly. The EDI modules were
installed into the infrastructure and utilized for document translation and data transformation.
The resulting framework was reusable and enabled Electro to provide drop-in integration
capability to the targeted VARs.
Bottom-Line Business Benefits
Electro created more than $200 million in new, incremental revenue with a minimal invest-
ment ($50,000). With the new EDI integration solution, it quickly added a number of key
VARs to its customer list. In addition, the project yielded productivity gains of 30% to 45%.
This included an estimated 60% of reuse from existing integration assets, which sped time-
to-market. The solution enabled Electro to not only target new customers and their potential
revenues but also create entirely new business models that otherwise could not have been
addressed.
14
Chapter 1
The Business Imperative for Integration
How to Use This Book
This book is designed as a primer for those who need to understand integration and
how to apply it in their organizations to achieve business goals. The book is organized
as follows:
Chapter 2, Building the Business Case for Integration, more deeply defines the
business benefits and return on investment that can be achieved through in-
tegration. This chapter helps you build a business case for integration proj-
ects.
Chapter 3, Common Integration Scenarios, helps you identify the business ini-
tiatives that can be driven by integration technology.
Chapter 4, Guide to Integration Technologies, further explains the technolo-
gies available and how they relate to different business solutions.
Chapter 5, Integration and SOA, defines the relationships between SOA and
integration, and shows how to optimize your IT infrastructure to enable busi-
ness agility.
Chapter 6, Best Practices for Integration, offers guidelines to ensure that enter-
prise integration becomes a core competency in your organization.
Throughout the book, we have provided case studies that illustrate the benefits of in-
tegration in specific situations. Although we have changed the companies’ names to
maintain anonymity, they are actual organizations that have used integration products
to achieve their goals.
In closing, remember that integration is not an end point. It is not a single technology
that comes in a shrink-wrapped box, nor is it even a single project. As you read this
book, it will become increasingly clear that integration is a journey. Success requires
technology, process, and people. We sincerely hope that this book will be a valuable
guide on your integration journey.
4
4
4
4
4
15
Executive Overview
M
any past investments in infrastructure technology have failed to deliver real
business benefits and return on investment (ROI), making some organizations
leery of investing in integration software. However, integration is a key component in
the creation of a more agile enterprise. It can help businesses streamline operations,
provide better customer service through access to more complete information, and
enable faster response to competitive threats. Progressive organizations are leverag-
ing integration strategically to fundamentally differentiate their products and services
and to gain competitive advantages that drive revenue growth. In IT, integration can
reduce the cost and time required to implement new solutions while improving ef-
ficiency. More intangible benefits include the ability to enhance the organization’s
brand value by enabling modern, consumer-oriented solutions and a higher degree
of service quality.
Business cases for integration should reflect a combination of tangible returns in hard
dollars and more intangible returns that, although difficult to enumerate, offer sustain-
able business value.
Organizations also need to understand that integration was a requirement long before
packaged integration middleware solutions became available. Before the availability
of off-the-shelf integration technology, implementing a software package with inter-
faces to other applications, or combining systems because of a merger or acquisition,
typically meant hard-wiring the systems together in a point-to-point fashion using
hand-coded interfaces. With this type of integration, the number of interfaces involved
rises exponentially for each additional system being integrated. Furthermore, the pro-
prietary nature of each interface makes it impossible to leverage existing work when
adding new systems. And, because they are hand-coded, the interfaces tend to be
brittle and resistant to change. As a result, upgrading a software package can require
integration costs several times the purchase price of the package. The net result is that
Building the Business Case for Integration
2
16
Chapter 2
Building the Business Case for Integration
a significant percentage of IT organizations’ budgets are being consumed by the high
ongoing maintenance and support costs for their home-grown integration solutions.
When making a buy vs. build decision for integration software, organizations need to
factor in the total cost of ownership. In nearly all cases, off-the-shelf integration soft-
ware solutions deliver a better ROI than internally developed alternatives. Off-the-shelf
integration software is also typically more functional and robust, since the vendor can
invest more R&D dollars into development and provide support for the software.
This chapter helps you to identify the benefits of integration and to build a business
case and calculate the ROI you can expect from your investment in a packaged integra-
tion solution.
Tangible and Intangible Benefits
Because of the complexity of integration and the mixture of tangible and intangible
benefits, it is rarely possible to build an integration value case that is purely financially
based. Instead, the overall business case for investing in an integration infrastructure is
likely to have three parts:
Dollars representing the direct cost benefits of integration
Dollars representing the indirect benefits of integration
Soft or intangible factors that must be considered as additional weight in any
judgment
Direct cost benefits from an integration solution represent actual dollars saved, such
as reduced development costs for new projects, reduced maintenance costs, reduced
errors and the cost of fixing errors, and reduced resource needs.
Assigning estimated values to indirect benefits is more challenging but equally es-
sential. While indirect benefits may be difficult to quantify in the short term, they often
provide the largest long-term benefits. The key here is to ensure that the affected busi-
ness units agree with the assessed benefit value. For example, if improved integration
4
4
4
17
Chapter 2
Building the Business Case for Integration
delivers a more efficient just-in-time process for a manufacturing company, one ben-
efit will be reduced inventory. On this basis, the inventory manager can provide a value
estimate. It should be possible to validate this figure as the project is deployed.
The third part of the justification—soft or intangible benefits—is the most difficult to
address, but it is nevertheless an important part of the consideration. Suppose the in-
tegration project delivers a more responsive customer self-service system. One benefit
should be improved customer satisfaction and, hence, customer loyalty. Though this
may be a key organizational goal, assigning a dollar value to the benefit is difficult. Or
suppose the integration project is part of fulfilling legislation, such as Sarbanes-Oxley
requirements. Again, it is hard to place a dollar value on this, though noncompliance
would have significant implications for the business.
Soft or intangible factors make evaluating integration projects an art as well as a sci-
ence, in that a value judgment is required in addition to hard numbers. Nevertheless,
decision makers can still obtain a meaningful broad-brush feel as to whether the in-
vestment makes sense. And later refinement will make the case more accurate, even if
some benefits remain intangible and subjective.
Figure 2-1. Examples of benefits to consider
in an integration investment decision
18
Chapter 2
Building the Business Case for Integration
Areas for ROI
Integration is a key infrastructure technology for delivering a new breed of more agile
enterprise applications, but it does present challenges for justifying investments. First,
business investments are usually made on a project-by-project basis, while infrastruc-
ture investments span multiple projects. It is often easier to justify investments across
projects if an organization takes an enterprise approach to integration and the recom-
mendations come from an Enterprise Architecture Group or Integration Competency
Center (ICC) within the Architecture Group. To justify integration infrastructure invest-
ments, look for both IT and business benefits, as well as benefits to the organization as
a whole.
IT BenefITs
IT can benefit from integration through decreased costs and increased agility. Cost re-
duction can be measured in real dollars. Increased agility is more difficult to quantify,
but it can provide potentially much larger long-term benefits and ROI.
Direct Cost Benefits
The direct cost savings for IT through integration include reductions in three areas:
Development and implementation costs
Skills requirements
Operating costs
Integration software provides a single interface point to individual applications. Instead
of programming to a different application program interface (API) for each application,
integration platforms and Web services provide a standard interface to the integra-
tion software that acts as a broker. This drastically reduces the number of interfaces
required—and, thus, the number of interfaces that need to be developed, tested, and
maintained. The savings can be calculated by first determining the cost of develop-
ing interfaces for each application that needs to be integrated, then adding the cost
of maintaining those interfaces, and finally comparing the total cost with the cost of
purchasing the vendor-developed and maintained adapters.
4
4
4
19
Chapter 2
Building the Business Case for Integration
Case Study 2-1
Company: ScandAir
Company Description
ScandAir is a pseudonym for an Iceland-based airline that carries 1.5 million passengers each year
on its fleet of Boeing 757 and 767 jets. ScandAir has sales offices and flight destinations across
Europe and North America, and serves tourist and business traffic to Iceland as well as interna-
tional traffic via its hub in Reykjavik.
Business Challenges
Internet bookings of airline tickets were rising, and ScandAir’s Web sites felt the impact. This
evolving business channel had the potential to reduce costs and improve efficiencies, but it was
having the opposite effect. Each online sale required an average of 11 minutes of administrative
back-office support, negating any business benefits possible from e-commerce.
At the same time, ScandAir’s IT department was evaluating the need for an integration platform
to help solve the increasingly complex web of enterprise applications that powered the company.
ScandAir’s financial system, for example, interfaced with more than 30 applications. Managing
this had become increasingly challenging. The company was looking for an area to test a new
enterprise integration solution that would demonstrate a measurable ROI. The automation of its
online airline ticketing process provided the ideal pilot project.
How Integration Addressed the Challenges
ScandAir wanted to use enterprise integration to deliver a fully automated
e-ticketing system that would eliminate the time spent on its current manual processes. Using a
local integration partner and components from the webMethods Fabric suite, ScandAir rolled out a
successful initial pilot in less than three months that automated the back-office process associated
with Internet bookings in the Reykjavik sales office.
After this project’s success, the effort quickly expanded to include all bookings out of Reykjavik.
Then the model was implemented in each ScandAir sales region. The project included developing
adapters for the various back-office systems, including the TPOS credit-card clearing system and
the AMADEUS flight reservation system. These adapters were then used across the organization
for other purposes, saving development time and effort.
Bottom-Line Business Benefits
Since ScandAir’s first implementation in 2003, more than 150,000 bookings have been completed.
The automation has saved more than 27,500 hours of manual effort when compared with the
previous processes. In addition, standardization has brought new efficiency gains in business pro-
cesses and IT. ScandAir now processes its invoices electronically—something it did not originally
foresee—and this has produced additional savings as a result of reduced postage and paper use
and reduced storage space for the paper invoices.
20
Chapter 2
Building the Business Case for Integration
Integration software also reduces skills requirements because, typically, the integration
software shields developers from having to deal with many underlying technicalities.
An integration suite offers a standard approach that benefits not just the implemen-
tation process but also operations, further reducing the requirement for special skills.
This may eliminate the need to hire or contract with expensive specialist personnel.
Additionally, since a vendor is now responsible for the integration mechanism be-
tween components, it can be relied on to adapt to new technology advancements (for
example, ensuing that connectivity to new versions of an application such as SAP or
Siebel is maintained).
Finally, integration software can decrease operating costs by reducing transaction er-
ror rates. It automates the transfer of information into systems and eliminates the need
for rekeying, which reduces manual errors and thus offers significant ROI. Finding and
fixing errors increases the cost of a transaction by orders of magnitude.
In fact, in ROI studies conducted by ebizQ, the reduction in error rates alone could
quickly justify the cost of the system. For example, by using integration software, a
major North American transportation company eliminated 70,000 to 80,000 hand-pro-
cessed transactions, reducing the cost per order from $25 to $5. This amounted to a
$1.6 million annual savings. In another example, a high-tech manufacturer using Ro-
settaNet reported an 85% reduction in error rates and achieved a 100% ROI in the first
year. And in 1999, 81% of Cisco’s orders were placed online with 98% to 99% accuracy,
representing $11.7 billion in revenue annually. Organizations that can quantify error
rates in their current systems processing are often able to assign a significant value to
error reduction.
Indirect Benefits
The biggest indirect cost benefit for IT comes from the use of technology and solu-
tions. For example, when Hurricane Katrina hit the Gulf Coast in August 2005, a North
American insurance company wanted to make accommodations for its policyholders
affected by the hurricane, such as providing a grace period on premiums. IT had already
built a collection of Web services to access its mainframe systems in support of another
large project. Now it was asked to quickly build Web pages for the custom site. Even
21
Chapter 2
Building the Business Case for Integration
though the high-level Web services developed for the other project were not suitable
to support the new business requirements, a single member of the IT team was able
to quickly—within a day or two—recombine lower-level worker services to create the
necessary Web services for the Katrina site. This impressed the business managers and
was recognized as a Service Oriented Architecture (SOA) success story up to the CIO
level.
Soft Benefits
The soft benefits of integration for IT include increased responsiveness to the business
and an increased perception of IT as a business enabler. New business processes can
be deployed more quickly because of the reduced effort and the reusability offered by
integration. Further, as new technologies such as Web services take hold, a coherent
integration architecture delivers an enhanced level of system flexibility, enabling these
technologies to be embraced and exploited faster and cheaper. And, of course, the use
of vendor-supported integration technology means that responding to new technol-
ogy challenges is the responsibility of the vendor rather than the IT department.
BusIness BenefITs
The business benefits of integration include enhanced process efficiency and produc-
tivity, increased operational effectiveness, and improved competitive performance. En-
hanced efficiency and productivity often translate into direct cost savings. Increased
operational effectiveness often provides many indirect benefits. Improved competitive
performance can be the most valuable benefit to the organization, but also the most
difficult to quantify.
Direct Cost Benefits
Integration software improves efficiency and productivity in a number of ways. First, in-
tegration can increase levels of automation and reduce human intervention. Examples
include reducing order times by removing manual order reentry at different stages
of the processing cycle, cutting the percentage of failed funds transfer requests due
to input errors, and improving the speed of information sharing between different
22
Chapter 2
Building the Business Case for Integration
departments. Cost reductions would result from decreased staffing levels due to the
increased level of automation or the use of a more productive process. In addition, pro-
cess efficiency may also allow a reduction of inventory through just-in-time processes
that rely on a high degree of automation. This becomes even more feasible when the
process efficiency is extended to encompass suppliers and distributors, and it can have
the side effect of reducing procurement costs as well.
When business process management (BPM) functionality is layered on top of basic
integration capabilities, process efficiency and effectiveness are increased, and this can
have very quantifiable bottom-line results. In the 1950s, Dr. W. Edward Deming, a math-
ematician and statistician, showed that variations created at every step in a process can
cause defects, which are expensive and time-consuming to remediate. Deming’s work
with Japanese manufacturers was directly responsible for the low-cost, high-quality
goods that competed favorably against American manufactured goods.
Deming’s work gave rise to a number of initiatives to manage and measure the effi-
ciency and effectiveness of business, including Total Quality Management (TQM), Bal-
anced Scorecard, Six Sigma, and ISO 9001. Organizations that adopted these practices
realized measurable results. In fact, some integration software products on the market
today enable organizations to apply these quality management and measurement
practices to their business processes in the act of integrating them.
Indirect Benefits
Operational effectiveness creates more indirect benefits. Integration can help make
a business more effective by improving business intelligence and its dissemination.
For example, the ability to pool information about a particular prospect from different
parts of the business may make cross-selling or up-selling possible. Or the ability to
provide comprehensive feedback on product performance directly from customers to
Engineering may enable the development of new processes to improve product qual-
ity. Assessing the ROI depends on the specifics of the situation; for example, benefits
might be realized by reducing the cost of sales or increasing sales volumes.
23
Chapter 2
Building the Business Case for Integration
Case Study 2-2
Company: Annuate, Inc.
Company Description
Annuate, Inc. is a pseudonym for an Australian super-annuation fund administrator. The company man-
ages 5 million member accounts and more than 340,000 employer accounts. It provides administrative
services such as bulk contribution processing, fund take-ons, benefit processing, insurance and claims
management, member communications, and the management of member inquiries.
Business Challenges
For several years, Annuate had been vying for a major new account, the Australian Retirement Fund
(ARF), with more than 600,000 members. Although Annuate had always been competitive in the ten-
der process, it learned that its business process applications did not meet ARF’s requirements. Annuate
identified its technology architecture as the main reason for losing the business. And it realized that
to be competitive it would need to build a world-class technology infrastructure founded on a solid
integration strategy that would enable new systems to be deployed while leveraging existing legacy
systems.
How Integration Addressed the Challenges
Annuate deployed webMethods Fabric to integrate with a new client’s applications and facilitate real-
time dissemination of information across the enterprise. This new enterprise integration infrastructure
allowed it to win the ARF account and to merge 600,000 ARF records in the course of a six-month
project that established the funds’ records on the core administration system, provided data confir-
mation and business rule coding, and made the other applications work off its Oracle-based member
administration system.
As part of Annuate’s integration efforts, it adopted an SOA platform that enabled the company to
reuse many of the services it created. Annuate can now expose Web services through the integration of
applications such as Web sites and internal administration systems, and can repurpose the components
of its existing legacy systems, further leveraging existing technology.
The integration also allows Annuate’s clients to view their portfolios online, get daily estimates of their
account balances, view regular e-newsletters, and learn about investment changes. These online tools
complement the information provided on member statements. The real-time updates of information
ensure accurate information to customers and across the enterprise.
Bottom-Line Business Benefits
The ARF win was a direct outcome of Annuate’s clear enterprise integration strategy. The implementa-
tion of the enterprise integration solution has allowed a number of back-end systems to interact and
provide a complete administration service for the client. As a result, Annuate has already seen a 30%
to 40% reuse of interfaces and services created, as well as reduced maintenance costs. The entire solu-
tion also enables IT to better support Annuate’s business strategy.
24
Chapter 2
Building the Business Case for Integration
Soft Benefits
Being able to serve customers’ needs faster and cheaper improves the business’s
chances of beating the competition. In addition, the ability for process changes to be
implemented faster at the operations level, through underlying IT mechanisms that
are more flexible to change due to integration, enables business operations to respond
to competitive action more quickly. Once an integration infrastructure is in place, it
reduces the cost of implementing change. It could also lead to increased revenue by
enabling a business unit to lead with new differentiating initiatives. These competitive
gains can be evaluated in monetary terms by assessing the likely market share increase
that results from operational efficiency and speed of reaction.
OrganIzaTIOnal BenefITs
Finally, there are the returns at the organizational level. Of course, the two areas already
described—IT benefits and business benefits—already have an effect on the overall
organizational performance. But this section concentrates on areas that apply more
specifically to the organization as a whole, reflecting the brand value across all aspects
of the business. These benefits include:
Compliance with government and industry regulations
Customer satisfaction
Exploitation of the organizational knowledge base
Access to new opportunities
Most organizational returns can be categorized as indirect or soft benefits. Benefits may
result even when integration itself is secondary to the underlying business require-
ment. For example, many organizations are finding unexpected benefits from their
compliance initiatives. The ability to integrate different information sources improves
the understanding of organizational operations and procedures. Moreover, particu-
larly with some of the more recent forms of legislation, such as Sarbanes-Oxley and
Basel II, it enables organizations to build sophisticated audit trails of their operations.
This has the beneficial side effect of providing organizations with a better window into
4
4
4
4
25
Chapter 2
Building the Business Case for Integration
how their business runs. In reality, any integration project that plays a key role in meet-
ing compliance mandates is likely to be justified on a nonfinancial basis (indeed, it is
often a case of organizational survival).
Integration plays a significant role in customer satisfaction. It can provide a complete,
organization-wide single view of the customer in real (or nearly real) time. This picture
may include current product sales, product experiences, reported issues, outstanding
deliveries, and many other facets of customer-related information. This capability al-
lows the organization to provide a much higher level of service to the customer, for
example through its call centers or help desks. Customer satisfaction leads to customer
loyalty, which many organizations can relate to financial performance, for example by
comparing the expense of securing new customers with the cost of retaining existing
ones. Regardless of how meaningfully the benefit can be quantified, customer satisfac-
tion is an important benefit of integration.
Integration can also bring together knowledge from all of an organization’s business
units, making it available for sharing and thereby increasing the overall value of the
organizational knowledge base. This can have paybacks in improved organizational
efficiency, more rational organizational structures, and a more effective workforce. For
example, providing employee portals that offer information about organizational pro-
cesses and procedures, such as health and safety, may simultaneously reduce educa-
tion costs for new employees, help improve employee morale, and achieve compliance
with legal requirements.
Finally, integration can link together the entire value chain, both within the organiza-
tion and outside to partners and even customers to enable to new opportunities. This
wide-reaching integration makes it possible to introduce new business products or
initiatives. In fact, many organizations are using integration strategically to offer new
services to new markets.
Defining Costs
To evaluate ROI, the benefits that accrue from the integration project must be balanced
26
Chapter 2
Building the Business Case for Integration
against the costs incurred. Broadly, IT costs tend to be grouped around the provision
and support of the IT changes required to implement the desired operational changes.
In contrast, business unit and organizational costs tend to be expressed in terms of
the changes required to adapt to the new operational processes and procedures. It is
important to remember that both are relevant, although the costs from the business
side of the organization are harder to evaluate. Organizations that forget the impact to
business personnel generally suffer as a result of the oversight.
IT COsTs
IT costs resulting from an integration project generally fall into three categories:
Infrastructure costs
People costs
Services
Infrastructure costs include the expected software license and maintenance costs.
These cover the integration solution selected, any new application packages, and any
other new pieces of infrastructure software required. It may also be necessary to ac-
quire hardware to handle the additional capacity generated by the solution or to pro-
vide local processing power as needed. While the initial infrastructure costs may be
high, they are easier to justify when amortized across multiple projects. This is another
good reason to take an enterprise approach to integration.
Clearly, a wide range of people costs must be included in any evaluation. These span
the traditional range of project development, encompassing planning, design, devel-
opment, testing, deployment, and support. At the planning and design levels, it is im-
portant to have both IT and business skills, since integration projects deal with both
aspects (for example, business processes have to be defined or modified to produce
the required business return, and this must be explained to the IT team). In addition,
business domain knowledge may be required to help IT understand the business stan-
dards, such as RosettaNet, that need to be taken into account in the solution.
4
4
4
27
Chapter 2
Building the Business Case for Integration
Case Study 2-3
Company: DNA Corp.
Company Description
DNA Corp. is a pseudonym for a life sciences company that specializes in research technolo-
gies, diagnostic products, and targeted therapeutics. Pharmaceutical and biotechnology com-
panies use its products to discover and develop new drugs more effectively. DNA Corp. has an
installed base of approximately 180,000 instrument systems in nearly 100 countries.
Business Challenges
DNA Corp. builds customized assays for its customers. This is a complicated process that
involves performing detailed analyses of DNA sequences. Traditionally, a customer uploaded a
file containing the sequence to DNA Corp.’s e-commerce Web site. The order was then routed
via FTP through DNA Corp.’s back-office systems, which included 13 enterprise systems from
order creation through accounts receivable. This process was error-prone, and recovering from
any problems that developed was labor-intensive. The development time for each new assay
could take 12 to 16 weeks. DNA Corp. needed to streamline the process for building custom-
ized assays to improve its competitive position and reduce costly errors.
How Integration Addressed the Challenges
DNA Corp. wanted to separate the process of managing and accessing the information about
the assays from its order entry system in SAP. To streamline the process, it adopted an SOA-
based enterprise integration solution from webMethods. The new system allows customers
to define assay requirements through a Web front end, which then passes an XML message
to the design engine. The engine attempts to produce an assay design to match customer
requirements. The customer can validate the design and then choose to have the assay built
based on optimizing cost, accuracy, repeatability, or other factors. These additional parameters
determine the process sequences used. Externalizing the assay information from SAP provides
greater flexibility and reuse, and reduces the overall time to build a customized assay.
Bottom-Line Business Benefits
The new enterprise integration solution has had a direct impact on DNA Corp.’s bottom line.
By integrating the 13 back-end systems involved, it reduced the development time for a new
assay type from 16 weeks to two weeks. This enabled DNA Corp. to reduce its order fulfillment
errors by 20% and decrease overall order time from three weeks to less than five days.
28
Chapter 2
Building the Business Case for Integration
The design phase needs to consider not just the technical architecture and approach
but also the operational processes and workflows necessary to support the solution.
Development resources are required to build the interfaces to the components be-
ing integrated, to modify the relevant IT infrastructure, and to supply essential mecha-
nisms, such as audit and problem determination trails. The integration software itself
has to be configured for the operating environment.
The test phase may require specific hardware and software, for instance to validate
performance requirements and to provide a thorough quality assessment of the new
software implementation. Personnel are required to carry out the testing and to resolve
and rework problems as required. Finally, deployment and support costs must include
items such as help desk and operations staffing and training, quite possibly with indi-
viduals who have business as well as technical skills, to ensure successful operations.
Throughout the range of project activities, it may also be necessary to consider third-
party service costs. Some organizations may decide to utilize systems integration ex-
perts to provide valuable guidance in the planning and design phases of the project.
Parts of the project may also deal with a specialized area of technology knowledge,
and in these cases it may be more financially viable to contract for the skills rather
than retrain existing IT staff. Examples include retaining someone to integrate a par-
ticular software package or someone skilled in the particular integration technology
selected.
BusIness ImpaCTs and COsTs
The three categories most likely to be taken into account in the area of business costs
are:
Resources to support project implementation
Education and training of end users
Operational risks
4
4
4
29
Chapter 2
Building the Business Case for Integration
The first category has already been described in the previous section. Because integra-
tion projects span business and technical disciplines, resources to support planning,
design, implementation, and deployment activities are required. These resources are
necessary to ensure not just that the right solution has been built but that the new
solution is deployed in a controlled fashion to minimize disruption and business risk.
The education and training of the end-user community is another important cost to
consider. If the integration project is simply automating some back-room operations,
then the end-user experience and procedures may remain unchanged. But if new or
changed business processes are being implemented, retraining is an essential element
to be considered. Additionally, if the integration project spans business partners as well
as internal departments, the partners may also need education.
The area of operational risks covers aspects such as the loss of productivity during the
retraining and changeover to the new process, and potential service disruptions due
to technical problems or user errors experienced as the solution becomes fully estab-
lished. Since it is difficult to determine the impact of these issues and their associated
cost, decision makers may simply want to note them as part of their overall evaluation
criteria.
Best Practices
The following guidelines will assist you in determining the potential ROI for your inte-
gration project.
Know what you are measuring. Obviously, when measuring the success of any project,
you must first know what you are measuring. Set your objectives for the ROI audit. Be-
fore you can measure improvements, you need to know what your current processes
cost. Take baseline measurements before you begin the project. Measure the initial
performance levels of the processes under consideration for change or optimization
to build a baseline for future comparison. If you have already defined critical success
factors, you can do a baseline measurement of them. We recommend budgeting one
to two weeks for the initial estimate. Then determine how often you want to measure
your progress against your goals. We recommend a quarterly or semiannual cycle.
30
Chapter 2
Building the Business Case for Integration
Get executive commitment. Identify an ROI audit sponsor. This person is usually the
senior manager of the business unit affected by the implementation. Ideally, he or she
should be able to sell the audit findings throughout the organization and be willing to
defend the logic, assumptions, and results confidently.
Focus on improving business processes. Ultimately, improving business processes will
yield the highest return on investment. To begin, identify some initial processes that
are the low-hanging fruit—ripe for automation and integration.
Start small and lay the foundation for future enhancements. Nothing succeeds like suc-
cess. Make sure the first projects have a high probability of succeeding. Keep the scope
relatively small but still pertinent to the business.
Avoid analysis paralysis when doing the ROI study. Set a number of hours or days com-
mitted to ROI analysis, and stick to it.
Build a credible business case for integration. Do your homework to build the analysis
from the ground up, and its credibility will show. Avoid forcing numbers. Be careful
not to work from the goals down to the supporting numbers. When working on the
analysis, many assumptions and variables may change numerous times throughout
the process, and these often will ripple to affect other parts of the analysis. Be sure to
check, double-check, and triple-check all the calculations. One error could shoot your
credibility in the entire analysis.
The checklist on the following page will help you build your business case for investing
in an integrated infrastructure.
31
Chapter 2
Building the Business Case for Integration
32
Executive Overview
M
ost midsize to large corporations and government organizations have a myriad
of computing platforms, business applications, databases, and other technolo-
gies distributed throughout their organizations. These systems were designed to meet
the needs of specific parts of the organization and were not intended to be integrated
with other systems. Business requirements have changed, however, and integration has
become a fundamental necessity for any business process that involves more than one
system or the coordination of work across different groups of employees.
Addressing a particular business need may involve several different integration re-
quirements. For example, implementing a supply chain management solution might
involve connecting electronically to suppliers and customers, managing the manual
processing of order exceptions, synchronizing order details among different internal
systems, and providing a single view of customer transactions and details across differ-
ent parts of the organization.
It is possible to break down almost all business integration requirements—even those
involving the most complex business processes—into a few basic integration scenarios
(or “use cases,” to use a software engineering term). These scenarios can be thought of
as building blocks for implementing more complex, integrated business solutions.
The basic integration scenarios are:
Synchronizing data among different databases and applications
Automating transactions among different internal systems—commonly re-
ferred to as enterprise application integration (EAI)
Integrating mainframe-based applications and data with distributed systems
Connecting businesses electronically and automating transactions with part-
ners, suppliers, and customers—known as business-to-business (B2B) integra-
tion
1.
2.
3.
4.
Common Integration Scenarios
3
33
Chapter 3
Common Integration Scenarios
Providing a single view of information that is distributed across different sys-
tems and databases
Extending access to systems over the Web and via other channels, such as mo-
bile devices and interactive voice response units
Repurposing IT assets by turning existing functions into components that are
then assembled into new composite applications
Automating and managing business processes and workflows
Using integration to optimize business performance
Just as the integration scenarios vary, so do the technologies available to implement
them. In this chapter, we discuss the integration scenarios and mention some of the
technologies. Then, in Chapter 4, we discuss the integration technologies in more de-
tail. Together, these two chapters provide a guide to choosing the right integration ap-
proaches and technologies for an organization’s specific business requirements.
Data Synchronization
An organization usually has multiple systems and platforms, and similar information
is inevitably kept in many of them. Information about customers, orders, and products
can span customer management, order entry, fulfillment, warehousing, and billing sys-
tems. Keeping all this information synchronized is essential to the smooth execution of
many business processes.
Figure 3-1. Sharing data among multiple systems
5.
6.
7.
8.
9.
34
Chapter 3
Common Integration Scenarios
Many organizations synchronize data by rekeying information manually into systems.
However, this method has a high error rate, and fixing an error increases a transaction’s
cost by magnitudes. Automating data transfer provides direct cost benefits by reduc-
ing the personnel costs of both rekeying data and fixing errors. Some organizations
have found that error reduction alone delivers a sizable enough return on investment
(ROI) to justify an automated data integration solution.
Once an organization decides to automate data synchronization, it must choose which
approach to take: transferring flat files, performing database-level updates, or using
an application program interface (API) or application adapter. Because information re-
sides in disparate systems and platforms, data mapping and transformation must be
done so the information is delivered in the right format for the target platform. Data
synchronization patterns, from least to most complex, include one-to-one (synchroniz-
ing data between two systems), one-to-many (synchronizing data from one system
across two or more systems), and many-to-many (synchronizing data from multiple
systems across multiple systems). Data synchronization can also be one-way or bidirec-
tional. If it is bidirectional, a mechanism is needed to prevent an endless loop in which
an update to System A triggers an update to System B, which triggers another update
to System A, and so on.
Enterprise Application Integration
Whereas data synchronization addresses the need to duplicate data across different
systems, application integration addresses the need to link different applications in-
volved in the same business process—for example, an order management process that
spans customer relationship management (CRM) and enterprise resource planning
(ERP) packages. Mergers and acquisitions also create the need for application integra-
tion, for example, to integrate financial and HR systems.
35
Chapter 3
Common Integration Scenarios
Figure 3-2. Exchanging transactions among applications
Application integration is at the core of a variety of business problems. It can be a com-
ponent of B2B applications, composite applications, and business process automation
(as described later in the chapter).
Application integration can sometimes be achieved at the database level—for exam-
ple, extracting an order from System A’s database and inserting it directly into Sys-
tem B’s database. In these cases, the programmer needs to understand the underlying
structure and meaning of the data and ensure that data consistency rules are adhered
to rigorously so the applications receive data in the format they require.
More typically, application integration occurs at the transaction level, such as during
the creation of an order. The transaction—usually in the form of a message sent from
one system to another—is typically processed through an API or an application adapter
that simplifies connectivity with the application. This level of integration enforces the
application’s business logic. However, data translation and transformation are still nec-
essary to ensure that transactions are properly formatted for the receiving systems.
Critical components of application integration include transactional integrity, secu-
rity, the ability to trace what is happening, and exception-handling mechanisms. While
36
Chapter 3
Common Integration Scenarios
exceptions often must be processed manually, they should be managed as part of the
end-to-end automated and integrated process.
Application integration technologies include adapters, message brokers, EAI suites, en-
terprise service buses (ESBs), and application routers. Similar in concept to network
routers, application routers combine hardware and software to address simple integra-
tion needs.
Mainframe Integration
Mainframe integration can be part of various integration scenarios, including data syn-
chronization, application integration, B2B, composite applications, and multichannel
access. However, the unique intricacies of the mainframe environment demand ad-
ditional considerations.
In general, it is difficult to gain access to mainframe applications from the outside be-
cause they were not designed for this. Mainframe operating environments are tightly
controlled and do not lend themselves to external access beyond the originally pre-
scribed mechanisms, such as via terminals or reports.
Figure 3-3. Accessing mainframe systems and data
Mainframe access methods are broadly classified as invasive or noninvasive. Invasive
methods require changes to the mainframe code. They are difficult to implement but
37
Chapter 3
Common Integration Scenarios
may be worth the trouble when the integration needs to support a high transaction
volume. Noninvasive methods require few or small changes to the mainframe. In some
cases, no additional software is required. Options for noninvasive mainframe integra-
tion include the following:
Screen scraping, which replaces a human operator with a computer program
that interacts with the mainframe application through a terminal interface (for
example, 3270).
Report-based interfaces, which use the output from reports to trigger data-
oriented integration processes.
Direct data access via gateways to mainframe-resident databases such as DB2.
These products are sometimes referred to as SQL gateways.
Transaction-level integration to CICS or IMS. This can be noninvasive if the
mainframe application offers that interface, or minimally invasive if imple-
mented through a message-based interface.
In the long run, the best solution for mainframe integration is “wrappering” existing
mainframe transactions—that is, using integration technology to noninvasively make
existing transactions accessible via different protocols—into reusable Web services
that can be accessed from any source. Though it has an initial development cost, wrap-
pering enables greater reusability and flexibility.
Each approach to mainframe integration requires different technologies. The best ap-
proach depends on the characteristics of the mainframe application, the implementa-
tion time frame, and whether the integration is part of a tactical or a strategic imple-
mentation. Other factors may include the available skill sets, budget, and available tool
sets.
Market consolidation has left only a handful of specialized legacy integration vendors.
Many integration platform vendors partner with a specialist on OEM legacy integra-
tion technologies to offer mainframe integration capabilities as part of their broader
suites. Whether an organization is implementing a legacy extension solution as a tacti-
cal solution or building a legacy integration service layer as part of a Service Oriented
Architecture (SOA) strategy, it needs to understand how this capability fits into the
overall architecture.
4
4
4
4
38
Chapter 3
Common Integration Scenarios
B2B Integration
Automating transactions with partners, suppliers, and customers is commonly referred
to as business-to- business, or B2B, integration. Organizations require B2B integration
when they are attempting to streamline the supply chain, connect to an electronic
exchange, or comply with industry and regulatory mandates that require electronic
communications to government or business entities outside the organization.
Figure 3-4. Integrating business processes with trading partners
Processing electronic transactions with business partners can be 10 to 100 times less
expensive than processing the same transactions manually. Organizations typically
have used a value-added network (VAN) solution to implement electronic interchange
of purchase orders and other electronic documents, usually based on electronic data
interchange (EDI) formats and standards. However, VAN- and EDI-based solutions are
expensive to implement. This is due to the complexity of processing EDI and VAN
charges, which are based on the volume of data transacted. As a result, EDI is used
mostly by larger organizations—and by smaller organizations only when they need to
do business with larger organizations.
With the advent of Extensible Markup Language (XML) based integration technologies
that utilize the Internet and open standards rather than expensive proprietary technol-
ogies, alternatives have emerged. EDI is not going away, but EDI systems are evolving
39
Chapter 3
Common Integration Scenarios
to integrate with newer standards-based technologies. Together, the new integration
technologies are making it possible for many more organizations to participate in elec-
tronic commerce.
Integration with systems completely outside the organization’s control pose some
challenging integration requirements. Additional security is necessary to authenticate
partners and ensure that the transactions are legal and cannot be repudiated. Orga-
nizations also need a way to manage service level agreements with each partner and
provide different connectivity options for small, midsize, and large organizations. B2B
integration also requires support for different data formats, including EDI, XML, and
custom flat files, as well as standard industry protocols, such as RosettaNet and the
Health Insurance Portability and Accountability Act (HIPAA).
B2B is also often associated with industry exchanges such Covisint, which started as an
automobile industry exchange and has expanded into healthcare, and the World Wide
Retail Exchange (WWRE), which was set up by 17 international retailers to automate
retail supply chain processes. These exchanges offer supply chain services that enable
suppliers and partners to conduct electronic business using the Internet rather than a
proprietary network as the transport layer. They allow all partners to integrate with the
exchange, rather than integrating point-to-point with all their partners and suppliers.
This saves considerable time and money.
Exchanges may also provide additional value-added services. For example, Covisint of-
fers a Web-based tool to coordinate and manage problem-solving processes between
customers and suppliers. WWRE offers a collaborative planning tool, a logistics tool,
and other facilities that enable on-line trading. Ariba, another B2B exchange, provides
buyers and suppliers with a number of services for conducting electronic business.
Today, B2B integration technologies are generally available as part of a broader integra-
tion platform or in a hosted environment through integration service providers, such
as GXS. Technologies for implementing B2B solutions include basic application capa-
bilities, partner management capabilities (which address the need to manage process-
ing rules and service levels on a partner-by-partner basis), and capabilities for handling
B2B-specific security requirements, such as firewall friendliness and the encryption of
documents exchanged over the Internet.
40
Chapter 3
Common Integration Scenarios
Case Study 3-1
Company: IT Corp.
Company Description
IT Corp. is a pseudonym for a Fortune 50 information technology company with more than
150,000 employees doing business in more than 170 countries. The company offers products
to consumers, small and midsize businesses, enterprises, and the public sector. It invests heav-
ily in R&D, producing more than 10 patents a day worldwide.
Business Challenges
IT Corp. has been dedicated to automating the computer industry supply chain and B2B trans-
actions. Traditionally, all supply chain transactions, from forecasts to invoices, were handled
via paper and email. This process was very inefficient and translated into high costs, increased
turnaround times, and data errors from manual processing. IT Corp. identified the supply chain
process as business-critical and wanted to create a Web-based channel that would enable
supply chain transactions between IT Corp.’s customers and suppliers through a B2B Web site.
How Integration Addressed the Challenges
Using the business integration solution from webMethods, IT Corp. was able to link its Baan
ERP system with an internally developed Active Server Pages Web application. This enabled
the company to use all its existing business logic that resided in Baan for the new Web appli-
cation. With the Web-based supply chain system, business partners can now conduct all trans-
actions on the B2B Web site. Customers can place new orders, request changes to existing or-
ders, view and print order acknowledgments and shipping details, and submit order forecasts
to IT Corp. to streamline the procurement process. Suppliers can easily view and acknowledge
orders, provide updated shipment details, and access procurement forecasts to manage sup-
plies. The new system processes between 10,000 and 15,000 transactions each month.
Bottom-Line Business Benefits
Delivered in just three months, the Web-based B2B portal has become an integral part of IT
Corp.’s supply chain system and supports a growing percentage of the company’s transactions.
The automated process has streamlined and simplified the purchasing process for the com-
pany’s suppliers and customers. By completing simple online transactions, customers receive
immediate responses and acknowledgments of process completion, and this has increased
customer satisfaction. In addition, the automated system has eliminated time-consuming and
error-prone manual processes. The real-time nature of data flows across the system provides
accurate information at all times. The new system has effectively saved thousands of man-
hours of work for IT Corp. and its business partners.
41
Chapter 3
Common Integration Scenarios
Because most organizations do business with many types of partners, they need differ-
ent types of on-ramps, or methods of connecting to partners and suppliers. The larg-
est partners may have full-blown B2B implementations, including full integration with
their back-end systems. Some midsize partners may just need to send and receive XML
messages. And smaller partners may require browser-based connectivity where they
manually submit transactions or check on status. Ultimately, the goal is to make it fast
and easy to connect, because the ROI for B2B implementations is directly related to the
percentage of partners participating.
Single View of Information
With the proliferation of applications and databases in an organization, information
about customers, suppliers, employees, products, orders, and other business entities
becomes spread out, and it is difficult to get a single, consistent view. Integration soft-
ware can help organizations aggregate information from various systems to create a
single view of any business entity. This capability is especially important for business
solutions used in situations such as mergers and acquisitions, CRM, or integrated risk
management.
Figure 3-5. Combining information into a single view
42
Chapter 3
Common Integration Scenarios
One way to gain a single view is to physically aggregate data into one place. Many
organizations have created data warehouses and data marts to accomplish this. How-
ever, data marts and warehouses are generally updated at predefined intervals, such
as nightly or weekly, so they may lack the most current information. An alternative
method is the distributed query, which involves keeping the information in the source
systems and extracting the information as needed into a combined view, as if all the
information were coming from a single source. This ensures that the information pre-
sented is up-to-date and the most current available. The tradeoff with a distributed
query is that it typically takes longer to get a response than it does when making a
request to a single database.
An important dimension of single-view integration is the need to understand both the
underlying data structures of the various source systems and the semantic meaning of
the data in each system so they can be reconciled into the single view (whether in a
data warehouse or in real-time). It is necessary to determine, for example, that the Cus-
tomer field in System A is the same as Name in System B, CustName in System C, and
Client in System D, and to map the transformation of data as it is extracted from the
sources. A major ambition for data integration is to capture the semantic meaning of
information within systems in metadata, which allows systems to be integrated more
easily without having to create each mapping manually.
A number of new solutions are coming to market that enable a real-time single view
of information. These include enterprise information integration (EII) solutions, which
have proprietary technologies and algorithms for aggregating information in real-
time, such as distributed query optimization, indexing, and caching. Some solutions
include semantic integration capabilities. For physical-aggregation-style approaches,
traditional data integration technologies can be used to source data from—and popu-
late data in—the databases involved.
43
Chapter 3
Common Integration Scenarios
Case Study 3-2
Company: FoodCo
Company Description
FoodCo is a pseudonym for one of the leading food and beverage companies in the United
States. It is the largest processor and distributor of milk and other dairy products, and oper-
ates more than 120 plants in the United States and Spain.
Business Challenges
Over the years, FoodCo had acquired several new businesses, which operate as independent
business units. Each acquisition brought more SKUs or UPC codes, and managing them had
become increasingly challenging. Data for identical products was inconsistent among busi-
ness units, and finding and validating information that customers needed took a long time. In
addition, maintaining product information was a complex and error-prone process, especially
when items had multiple levels of descriptions, such as formulas for ice cream flavors. FoodCo
realized that it needed to create an enterprise-wide product data management process to
ensure data consistency across business units and guarantee the delivery of information to the
right place at the right time.
How Integration Addressed the Challenges
Using the webMethods integration solution, FoodCo created a single master-item system that
aggregates data from multiple back-end systems and routes it to users for action. The new
system enables more than 100 employees across 32 business units to create, edit, validate,
and manage product data. The system also generates alerts to resolve exceptions that could
lead to data inconsistency.
Customers now see FoodCo as a single vendor with a consistent product catalog. They can
make purchases in a timely manner because they receive notification beforehand and can
react more efficiently to what the company sends them.
Bottom-Line Business Benefits
The new master-item system enables FoodCo to manage more than 35,000 SKUs in nine
categories and to add more than 200 new items each year. For its 360,000 customers and
partners, the system has reduced the time for new product introductions from five days to
one. Because the enterprise integration platform synchronizes the data within the existing ERP
systems, FoodCo can leverage its investment in internally developed systems and meet cus-
tomer mandates on synchronized items. The webMethods platform has also enabled FoodCo
to implement a complete standards-based EDIINT/AS2 solution between VAN and non-VAN
customers, resulting in a five-figure savings on annual VAN charges.
44
Chapter 3
Common Integration Scenarios
Multichannel Access
The past few years have seen a proliferation of devices through which end users and
consumers can access computing services. No longer tethered to an application run-
ning on a desktop computer—or, before that, a mainframe terminal—people can now
use Web browsers, personal digital assistants (PDAs), mobile phones, interactive voice
response units, and other special-purpose devices to receive and send information. As
organizations start creating reusable business services, they inevitably want to make
these services available through different channels.
The expansion of access channels represents an opportunity for organizations to bet-
ter serve their employees, customers, suppliers, and partners anytime and anywhere,
but it also presents a significant IT challenge. The associated technologies are expand-
ing and changing rapidly, and organizations will increasingly need to provide services
across a variety of devices based on a variety of technology environments.
Rather than altering existing applications to allow for each type of access channel, or
developing solutions that are hard-wired to a specific type of device, several emerging
technologies and standards initiatives allow multichannel access to different back-end
systems.
Figure 3-6. Using different end-user channels
to access back-end systems
45
Chapter 3
Common Integration Scenarios
Case Study 3-3
Company: Comco, Inc.
Company Description
Comco, Inc. is a pseudonym for a U.S.-based Fortune 100 global communications company
that provides mobility products and solutions for the home, automobile, and workplace. The
company’s 2004 sales exceeded $30 billion.
Business Challenges
Comco’s mobile device business unit was losing significant margin because of duty fees for
phones shipped from its U.S. plants to Latin America. The company recognized that divesting
the manufacturing of these phones to a regional manufacturer would reduce its duty fee costs
in the Latin America market by several million dollars each year. Comco’s major challenge was
to define an automated process that would transfer the manufacturing and distribution of
mobile phones to the third-party contractor without reducing on-time shipments to Comco’s
customers. Another challenge was transforming Comco’s existing ERP system into two sepa-
rate but seamless systems, one located at the new contractor’s plant. This would involve a
cutover process with minimal interruptions for each company.
How Integration Addressed the Challenges
Comco had made a formal commitment to RosettaNet, an industry standard for B2B com-
merce. It wanted to build on the RosettaNet work and create an enterprise integration solu-
tion that would provide touchless integration from Comco’s back-end Oracle ERP system to
the third-party contractor’s back-end SAP system. Comco implemented a webMethods-based
solution that enabled the new system to automatically exchange a variety of information
about engineering, purchase orders, and other product information. The system was also
enhanced with business activity monitoring (BAM) capabilities that send alerts to business
managers when process exceptions occur, allowing them to react quickly and proactively.
Bottom-Line Business Benefits
The new system has enabled Comco to relocate its manufacturing to Latin America and avoid
several million dollars in duties annually. The company has also reduced its operating costs by
close to $1 million each year through the elimination of manual processes, effectively reducing
the number of people required to support the business while maintaining on-time shipments
to customers. To date, the touchless system has enabled more than 1.2 million mobile phones
to be shipped to Latin American customers with no direct contact by Comco. In addition, order
tracking and other information is provided in real-time, giving the company added bandwidth
to focus on new business opportunities. This is a significant competitive advantage for Comco.
46
Chapter 3
Common Integration Scenarios
A midtier server is the least invasive and most flexible way to provide interface ser-
vices to multiple types of front-end devices. The midtier server renders the interface to
fit the display characteristics of each device. This way, the back end does not need to
change the way it sends the information for each type of device.
For accessing back-end systems, Web services are a good choice because they enable
access from a broad range of clients-side systems, including Web servers, Java, C#, and
mobile devices. Additionally, the use of XML and style sheets enables information to be
displayed in different formats for different devices without requiring changes to the
underlying data.
Few organizations will be able to ignore the demands for multichannel access from
multiple constituencies, and for many it will soon become a requirement. Organiza-
tions therefore need to create a strategy and an architecture for multichannel access
and integration.
Composite Applications
Organizations seeking to rapidly implement new business solutions soon realize that
they cannot pursue a rip-and-replace strategy. Starting fresh with each new system
simply costs too much and is too disruptive to the organization. Unfortunately, many
applications hinder rather than enable business change. Their rules, processes, data
definitions, and user interfaces are embedded in compiled code that requires program-
ming skills and detailed background knowledge to modify. To meet business needs,
organizations must leverage the systems they have in place but gain flexibility in how
the systems can be repurposed. Composite applications are the answer.
Composite applications are assembled (or composed) from existing components rath-
er than developed from scratch. They also incorporate functionality from a variety of
underlying systems.
This approach to building new systems aligns closely with Service Oriented Architec-
ture. SOA is an architectural approach to creating reusable building-block components
47
Chapter 3
Common Integration Scenarios
as services that interoperate through standardized Web services interfaces and that
can be combined quickly and easily to address new business process needs. A compos-
ite application consists of functionality drawn from several sources within an SOA. The
components may be individual Web services, selected functions from other applica-
tions, or entire systems whose business functions have been enabled as Web services
(often legacy systems).
Figure 3-7. Composing new applications from
existing underlying business services
An example of a composite application is a self-service portal for consumers or busi-
ness partners. Here, a number of existing processes supported by IT services might be
combined within the portal to provide the end user with one-stop access to a range
of the organization’s products and services. Without an SOA approach, this type of
composition would require considerable application integration, programming, and
testing. With an SOA, however, the composite application can be assembled easily,
48
Chapter 3
Common Integration Scenarios
regardless of the location or the technology of the different services being drawn to-
gether. This allows IT to respond faster to business demands, keeps costs to a minimum,
and powers imaginative new ways to address end users’ needs more effectively.
One caveat must be mentioned here. Whereas a monolithic application typically pro-
vides tools to monitor status (for example, to view the status of an order), a composite
application offers no such built-in monitors because the status crosses multiple ser-
vices on potentially different platforms. Therefore, for monitoring and other functions
such as policy enforcement, composite application implementations need an infra-
structure that exists outside the services themselves.
The technologies that enable composite applications include traditional application
development platforms, integration suites, Web services, and Web service repositories
that enforce governance policies, provide security, and enable reuse. The services from
which composite applications are composed can be created by wrappering existing
application functions or by building new ones from scratch. As such, a tool set is need-
ed to catalog, discover, and orchestrate services into a new application, and to deploy
and manage the result. Additionally, a run-time deployment infrastructure is needed to
connect all the components in a scalable and reliable way.
Business Process Management
As integration technologies and solutions have matured, the scope and complexity of
the problems that organizations are looking to solve have increased. Beyond ensuring
data consistency, linking a few applications together, or exchanging documents elec-
tronically with a trading partner, organizations are increasingly looking to integration
software to streamline entire business processes.
Business processes are the lifeblood of an organization. Whether related to customer
service, product development, or service delivery, an organization’s business processes
define its ability to compete successfully in the market. Process-level integration offers
the richest business benefits. Organizations can increase competitiveness through dif-
ferentiated business processes, and IT can better keep pace with changing business
requirements. So, while eliminating manual data entry and automating transactions
can yield direct and measurable cost benefits, automating and optimizing business
49
Chapter 3
Common Integration Scenarios
processes holds the greatest potential for reducing labor costs and cycle times; im-
proving employee, customer, and partner satisfaction; and increasing business agility
Figure 3-8. Coordinating end-to-end business processes
across systems and people.
By nature, business processes are often complex and disjointed. Different organization-
al entities may be responsible for different parts of the process, with no single entity
responsible for the end-to-end process. In addition, business processes often include
both automated and human processes. Some parts of the process may be performed
automatically by IT systems, whereas other parts may involve people performing spe-
cific tasks. In fact, typical enterprise business processes run the gamut from fully au-
tomated to semi-automated to entirely manual. Although organizations may want to
automate as much of the process as possible, people remain an important part of the
equation. Individuals are needed to assess exceptions, apply judgment and experience,
and handle other situations requiring context that is not easily encoded in a system.
The term business process management (BPM) generally refers to the set of activities
that organizations perform to either manage or optimize their business processes and
adapt them to new organizational needs. Because these activities are usually aided by
software tools, BPM also refers to the category of integration software that provides
these capabilities.
50
Chapter 3
Common Integration Scenarios
Since business processes are typically a combination of automated and manual tasks,
BPM solutions should support both types of processes. Human processes require a
user interface. Typical human processes involve delegation and organizational consid-
erations, such as the different roles of managers and employees. For example, manag-
ers need the ability to assign task to employees with certain skill sets, manage their
employees’ workloads, balance the workloads when assigning or reassigning tasks,
and provide management oversight and intervention for manual activities.
BPM solutions typically provide graphical tools for modeling business processes, a plat-
form for automating and executing these processes, and facilities to monitor them. They
may also provide simulation capabilities to test and optimize business processes.
Business Performance Optimization
Henry David Thoreau said, “In the long run, we only hit what we aim at.” When organiza-
tions have tools that deliver real-time visibility into—and performance measurements
of—business processes, they can focus not only on what goods and services they are
delivering but also on how they are delivering those goods and services. This focus is
necessary for business performance optimization.
Business performance optimization is a combination of methodology, best practices,
and technology infrastructure. There are two levels of business optimization. As dis-
cussed above, business process management solutions enable process-level optimi-
zation—in essence, streamlining the execution of business processes. Business per-
formance management solutions monitor relevant business events, which may cross
multiple processes, and relate them to key performance indicators (KPIs).
Both levels of business optimization are required. In fact, it can be argued that business
optimization should be performed at all levels of the organization, from the operational
IT level, through line-of-business managers, up to senior executives. All require visibility
into the KPIs for which they are responsible. Alignment across all organizational levels
is required for overall business optimization. Otherwise, an organization risks optimiz-
ing one process while suboptimizing at an enterprise level.
51
Chapter 3
Common Integration Scenarios
Business activity monitoring (BAM) is an emerging technology for enabling business
performance optimization. In 2002, Gartner coined the term BAM and defined it as
“the concept of providing real-time access to critical business performance indicators,
along with the supporting information to improve the speed and effectiveness of busi-
ness operations.” Whereas BPM solutions provide visibility across individual business
processes (or process-level management), BAM solutions allow organizations to corre-
late events from multiple processes and relate them to measures that are meaningful
to the enterprise. In doing so, BAM provides a “big picture” of business performance
and helps align business strategy with operational execution.
BAM solutions provide business dashboards that correlate real-time business events
with KPIs. Some BAM solutions include predictive capabilities and can identify emerg-
ing patterns that may influence a KPI, giving managers the information necessary to
proactively handle emerging problems rather than simply react to erupting crises.
Figure 3-9. Improving business performance
with BAM technology
52
Chapter 3
Common Integration Scenarios
BAM is often associated with business intelligence (BI) technology. Consider them
complementary sides of a coin, with BAM providing the real-time operational view into
business performance and BI the view for historical analytical purposes. Some BAM
vendors offer the ability to integrate with and feed into existing data warehouses, al-
lowing organizations to use their BI tool of preference.
Best Practices
This chapter illustrates how integration can be applied to a broad range of business
requirements. Chapter 4 provides more detail on the underlying technologies used to
implement the scenarios described in this chapter.
It is important to understand that midsize and large organizations have a variety of
business requirements and therefore require a variety of integration technologies. Be-
low are some best practices to increase the odds that an organization’s integration
initiatives will succeed in the long run.
PIan strategicaIIy, impIement tacticaIIy. Each of the integration scenarios presented
in this chapter could be approached as a tactical implementation or as a necessary
technical task within an overall strategic implementation. While such an approach
might yield short-term benefits for “low-hanging fruit,” in the long run it will increase
redundancy, software license costs, skills training, and management and maintenance
costs. In contrast, an enterprise architecture approach to integration will enable long-
term business agility. It will make different integration technologies available to differ-
ent projects as needed, ensure that the technologies are integrated with one another,
and decrease the time and cost to implement new solutions.
Create an Integration Competency Center (ICC). As an organization strives to de-
velop core competencies for managing integration, an ICC will help it more efficiently
build and maintain the complex integrations that its business demands. The ICC is re-
sponsible for the development and deployment standards that will help facilitate reuse
and interoperability as well support the implementation, maintenance, and continual
evolution of the organization’s integration platform. (See Chapter 6 for a description of
the ICC’s function and roles.)
53
Chapter 3
Common Integration Scenarios
Practice continuous improvement. Business agility is a journey, not a destination. Or-
ganizations need to evolve their infrastructures, business services, and processes to
optimize the business at all levels. Technology is just part of the solution. Real-time
visibility into processes and KPIs is only useful if someone is paying attention and prac-
ticing continuous improvement based on the enhanced information. An organization’s
success depends largely on what it does with the technology. Organizations need to
build a culture of continuous improvement and business optimization. This requires
that executives, line-of-business managers, and business operatives are all aligned
along the same business goals, and that a reward structure is in place to promote the
desired outcomes. The good news is that integration technology can also help solve
this problem through management and operational dashboards.
54
Executive Overview
A
s mentioned in Chapter 3, different business scenarios warrant the use of different
integration technologies and infrastructure services. Therefore, organizations will
inevitably acquire a collection of technologies to meet their diverse business needs. To
maximize agility and reuse, organizations need to understand how the range of inte-
gration solutions available today relate in the overall enterprise architecture.
The ebizQ Integration Roadmap can help organizations match their business require-
ments with the appropriate integration technologies. Becoming familiar with the
Roadmap is a good way to understand both the integration landscape and the types
of business problems addressed by specific technologies.
Application Integration Services
Application integration services offer the basic features and capabilities required to
integrate and connect applications reliably and securely. These services include:
Connectivity to the applications. Connectivity is generally achieved using
adapters, which enable a standard application connectivity approach and
eliminate the need to program against proprietary application program inter-
faces (APIs). If Web services interfaces are available, application connectivity
may be achieved without an adapter. Application connectivity may also be
achieved at the database level using a database adapter or a direct SQL inter-
face.
Data transport and routing capabilities to move information reliably and se-
curely between systems. Application integration solutions typically use a mes-
sage-based approach to moving data and support a variety of topologies that
link systems together in a scalable and flexible manner.
Data transformation and mapping capabilities. These enable data to be con-
verted to and from the required target representations.
4
4
4
Guide to Integration Technologies
4
55
Chapter 4
Guide to Integration Technologies
Infrastructure services to implement the application integration scenarios de-
scribed in the previous chapter. These include support for transactions, appli-
cation-level security, and exception management services.
Figure 4-1. ebizQ Integration Roadmap
The core application integration capabilities often play a role in other integration ser-
vices, such as information integration, interface integration, composite applications,
process integration, and business optimization with business activity monitoring
(BAM). Without the underlying application integration services, these other integration
services require point-to-point connectivity with back-end applications, which limits
agility and reuse.
Technologies that provide application integration services include message brokers,
integration servers, and enterprise service buses (ESBs). In addition, connecting with
mainframe systems and integrating the systems of different organizations in a busi-
ness-to-business (B2B) scenario have special considerations and, consequently, require
additional technology solutions.
4
56
Chapter 4
Guide to Integration Technologies
message BrOkers
The original application integration technology, message brokers provide the basic
connectivity needed for application integration, including messaging, data transla-
tion and transformation, and intelligent message parsing and routing. Message broker
products support different styles of messaging—such as request/reply for sending a
message to an application and receiving a reply, and publish/subscribe for broadcast-
ing information and events to a number of applications. An important function of the
message broker is to provide a secure and reliable communications layer—for exam-
ple, to ensure the guaranteed delivery of a message from one application to another.
Message brokers usually have a hub-and-spoke architecture, with all processing done
at the central broker. More advanced vendor solutions offer distributed processing op-
tions and multiple hubs to improve scalability and availability.
InTegraTIOn servers
As application integration requirements have become more complex, the basic mes-
sage broker-centric architecture has evolved to provide more functionality at the in-
tegration end points, such as supporting more sophisticated application processing
rules, handling B2B standards like RosettaNet, and performing protocol translation (for
example, layering a Web services interface around an existing application interface).
These additional capabilities are typically combined into an integration server. Inte-
gration servers can be used in combination with message brokers to create a scalable
integration backbone, with multiple brokers connecting multiple integration servers.
They can also be used alone for smaller point-to-point integration projects.
InTegraTIOn plaTfOrms
Since introducing their products in the mid-1990s, enterprise application integration
(EAI) vendors have evolved their message brokers, integration servers, and adapter li-
braries into integration platforms, which include additional capabilities and services
to address different integration scenarios and provide more direct business value.
These platforms, also called integration suites, generally include full B2B capabilities
57
Chapter 4
Guide to Integration Technologies
and some level of business process management (BPM) and workflow capability. They
may also include business activity monitoring features, a portal, mainframe integration
capabilities, and other combinations of the integration technologies. These additional
capabilities are discussed later in this chapter.
esBs
The enterprise service bus is an emerging middleware category that is typically viewed
as a key enabling technology for Service Oriented Architecture (SOA). Just as previous
generations of distributed systems depended on middleware such as RPC, CORBA, and
transaction monitors, SOA relies on ESBs to help bind the services together.
ESBs serve as intermediaries between services providers and service consumers to
avoid point-to-point connections. In this role, ESBs can fulfill requirements such as in-
telligent routing, transformation, load balancing, and security while also coordinating
the flow of service calls. Conceptually, ESBs provide much the same functionality in an
SOA as an integration platform does in an application integration scenario. The biggest
difference is that ESBs are generally based on Web services standards.
Some vendors originally positioned ESBs as simpler alternatives to full-blown integra-
tion platforms. In reality, the lines have blurred as ESB vendors incorporate functional-
ity similar to traditional integration platforms and suites.
adapTers
In a typical integration scenario, messages are routed between the various applica-
tions being integrated. Once a message arrives at an application, it must be processed,
usually by making updates or changes to the application or its database. The options
for accessing the application include writing a program that uses the API, using a pre-
built adapter, or using a Web services interface.
Many commercial applications have APIs, which enforce the necessary business rules,
transaction logic, and data checks for programs that interface with the application. Ac-
cessing the application via the API is thus preferred over integrating directly at the
58
Chapter 4
Guide to Integration Technologies
database level. The downside to APIs is that they differ for every application, so the
developer needs to know how to code to each application’s unique APIs. This increases
the time and cost for integrating systems and decreases agility.
Adapters are a desirable alternative to API-level programming. They simplify the inte-
gration effort by eliminating the need for application-specific interface development.
Adapters provide a common set of function calls for different back-end applications.
On the one side, an adapter exposes a standard interface to the developer; on the oth-
er side, the adapter deals internally with the task of translating these interface calls
to the specific APIs of a supported application. The developer needs only learn how
to code the standard adapter interface rather than each application’s interface. Some
vendors reduce the development effort further by offering graphical tools for config-
uring adapters.
There are two types of adapters. Intelligent adapters can perform data translation and
transformation at the source or target, thereby distributing the processing load and
increasing the solution’s overall scalability. Standard (non-intelligent) adapters provide
only basic connectivity services, thus requiring a message broker or integration server
for translation and transformation.
Off-the-shelf adapters are not always available for every application that an organi-
zation might want to integrate. So vendors frequently provide a development kit or
library for building custom, one-off adapters that interface with the application while
providing the niceties of a commercial adapter, such as graphical configuration or
built-in logging and error handling.
WeB servICes
Web services are an attractive integration alternative because they provide a standard
interface protocol that eliminates the need for adapters. However, Web services only
replace adapters, not the rest of the application integration services. For example, they
do not perform functions such as data translation and transformation, which remain
an important feature of the overall integration solution.
59
Chapter 4
Guide to Integration Technologies
B2B InTegraTIOn
Additional requirements exist when integrating with applications beyond the firewall.
These requirements include:
Support for a variety of electronic commerce standards and formats,
including Electronic Data Interchange (EDI) and newer XML-based
standards such as RosettaNet, as well as support for custom document and
flat-file formats.
Support for connectivity to proprietary communications networks and over
the Internet using protocols such as FTP, HTTP, and Web services.
More stringent security than what is typical for internal application integra-
tion. This includes the need to manage data confidentiality and integrity and
the identity of the parties involved.
Management of configuration settings and service levels on a partner-by-
partner basis.
B2B integration solutions usually include a variety of connectivity options—sometimes
called on-ramps—for different types of partners. The challenge of B2B commerce is to
provide a range of access technologies to meet the needs of all partners and suppli-
ers. For larger partners, it might make sense to fully automate the B2B transactions,
including integrating with the back-end systems on both sides of the firewall. Smaller
partners with minimal IT resources require simpler solutions, such as partner portals
for inputting information or just sending XML messages. Tools that enable organiza-
tions to connect faster, such as integration and process templates, accelerate the time
to value.
A major evolution of B2B integration is in the area of EDI. Electronic Data Interchange
encompasses a set of standard protocols for conducting structured interorganization
exchanges, such as making purchases or initiating loan requests. EDI transactions are
typically conducted through a value-added network (VAN) provider. These VANs are
essentially private networks for conducting B2B transactions.
4
4
4
4
60
Chapter 4
Guide to Integration Technologies
The explosive use of the Internet has led to the creation of a new standard—EDIINT—
that enables EDI transactions to be integrated with Internet transactions using XML.
EDIINT has several variations, which are based on the transfer protocol used. EDIINT ASI
uses SMTP, while EDIINT AS2 uses HTTP and HTTPS for securely exchanging EDI files
over the Internet. Some integration solutions on the market today combine EDI and
XML standards support in the same framework and tool set, allowing a single solution
to be used for all B2B-related integration requirements.
B2B solutions also demand different—and in many cases greater—levels of security
than does internal application integration. Authentication services are required to con-
firm the user’s or organization’s identity. Authorization services are required to specify
a user group’s levels of access to various applications and services. And nonrepudiation
services are required to ensure the validity and enforceability of electronic contracts.
The type of nonrepudiation needed depends on the type of application. Nonrepudi-
ated communication sessions may suffice for simple data exchanges, but nonrepudiat-
ed transactions are generally required when the solution includes more complex B2B
transaction processing.
B2B integration exchanges and service providers have become viable alternatives for
essentially outsourcing the B2B solution. Each partner needs only connect with the ex-
change rather than manage the details of connecting with every partner. This reduces an
organization’s burden when connecting to multiple partners with different technology
infrastructures.
maInframe InTegraTIOn
Mainframe systems usually do not have exposed APIs and therefore present unique
application integration challenges. There are several ways to integrate with these
legacy systems. Options include screen scraping or report scraping, integrating with
the legacy database, integrating at the transaction level through specialized gateways
(for example, for IBM CICS or IMS transactions), wrappering legacy code as a service
with a standardized interface, such as a Web service, or front-ending (that is, creating
a message-oriented interface to an application), which requires some changes to the
61
Chapter 4
Guide to Integration Technologies
application. Specialized legacy integration solutions are often used in combination
with other integration software to connect the mainframe systems with applications
running on distributed platforms.
Information Integration Services
Information integration services consolidate and integrate structured and unstruc-
tured information from multiple sources, and manage it at an enterprise level to pro-
mote data consistency. The value of the information in an enterprise system is in direct
proportion to its quality and consistency with other systems. When data is managed as
an important enterprise resource, it provides greater value.
The key to managing distributed information effectively is metadata, which is informa-
tion about data represented independently from the underlying systems. Metadata
plays an important role in enabling interoperability among systems because it allows
data to be understood and exchanged consistently across the systems. Enterprise
metadata repositories that provide real-time data services enable reuse and agility.
The next leap forward in data management services and integration productivity will
come from semantic integration. Semantic integration captures the meaning and con-
text of data within source systems in a metadata repository. Every integration project
requires an understanding of the underlying information in the systems. It is crucial to
know, for example, that CustName in one system is the same as Company in another
system. Generally, this knowledge is put into proprietary mapping tools. By captur-
ing it as semantic metadata, organizations can maximize reuse and ultimately enable
systems to automate the rules for translation and transformation. This also increases
return on investment (ROI) because the effort and cost of discovering the semantic
meaning of data within systems can be more readily reused across multiple projects.
Enterprise information integration (EII), enterprise content management (ECM), and
metadata repositories are the newer technologies that focus on information integra-
tion. Additionally, organizations will continue to use traditional data integration ap-
proaches—such as data warehouses and extract, transform, and load (ETL) tools—to
deliver the full set of information integration services.
62
Chapter 4
Guide to Integration Technologies
eTl
Extract, transfer, and load technology was developed for data warehousing implemen-
tations. ETL tools operated on large data sets in batch mode, extracting the informa-
tion from systems on a nightly or weekly basis, cleansing it, transforming it into the
desired target format, and then loading it into the data warehouse or data mart. ETL
technology is still used today for applications where it is preferable to physically move
the information to create a unified view and where real-time updates are not neces-
sary, such as for historical analytics. However, the newer information integration tech-
nologies discussed below offer the option of leaving the data in the source systems
and retrieving it in real-time as needed.
eII
An emerging integration market sector, enterprise information integration provides an
aggregated view of data. EII solutions can provide the equivalent of a virtual data ware-
house or midtier data store for real-time reporting or fast implementation of a portal
solution. EII tools have technologies for distributed query optimization, indexing, and
caching that enable real-time information access and aggregation. Graphical views
make it easier for programmers and business managers to access the information they
need without knowing the underlying data structures of the source systems. Whereas
data warehousing requires that the information be moved to a central database, EII
solutions access the data from the source systems themselves. Emerging EII solutions
are also incorporating semantic integration capabilities.
eCm
Enterprise content management provides integrated access to and management of
unstructured data, such as documents and images. ECM solutions can provide work-
flow and process management as well as integration capabilities. ECM services are re-
quired when the business process is primarily focused on documents being passed
around the organization, or when unstructured content needs to be part of an overall
business process integration solution.
63
Chapter 4
Guide to Integration Technologies
Interface Integration Services
Interface integration services provide access from a variety of front-end devices to a
variety of back-end systems. The technologies that provide interface integration ser-
vices include portals (for delivering the user interface via a browser) and mobile inte-
gration gateways (for enabling the use of mobile devices).
pOrTals
Portals provide an integrated user interface to multiple back-end systems. Many EAI
solutions focus on automating transactions across systems. But a portal solution inte-
grates information into a browser for a particular purpose or set of users, providing a
single touch point for the delivery of application services.
Various types of portals exist. Some—such as enterprise information portals, which pro-
vide information on corporate policies or news—are informational only. That is, they
aggregate information and functionality from multiple systems, but the information
flows only one way: to the portal. Transactional portals provide greater interoperability;
for example, enabling customers to place and track orders, or allowing employees to
enter expenses or manage their 401k’s. There are also management portals for tracking
business processes or key performance indicators (KPIs), and operational management
portals for monitoring system-level performance.
Portal technology often includes an environment for developing the portal applica-
tion. This application can be composed of multiple portlets, which are pluggable user
interface components. For example, there may be a log-in portlet that manages log-ins
to multiple back-end services. Portals may also include personalization features that al-
low users to customize the look and feel of the portal application—for example, choos-
ing what types of information to display, such as local weather or a customized stock
ticker. Although portal technology provides a way to develop an integrated user inter-
face, back-end integration is still required. This can be done point-to-point or using the
application integration technologies discussed above.
64
Chapter 4
Guide to Integration Technologies
Several important standards address portals. Web Services for Remote Portlets (WSRP)
defines a set of standard interfaces for creating portlets that can be accessed remotely
using Web services protocols. This allows applications to consume portlet components
without having to write unique code for interacting with each component. And JSR
168 is a Java API for portlets that enables content sources and application front ends to
be aggregated. It also addresses how security and personalization are handled.
mOBIle InTegraTIOn gaTeWays
Mobile devices—including cell phones, PDAs, and specialized mobile units such as
package tracking systems—all have different screen sizes, resolutions, and interface
requirements. These mobile devices are proliferating rapidly. Today, people commonly
use their cell phones to receive email, surf the Web, and conduct other aspects of busi-
ness.
IT departments are coming under increasing pressure to provide enterprise application
services to a variety of mobile devices. Mobile applications are likely to require back-
end integration to present information from multiple sources. In addition, because
mobile devices operate on inherently unreliable and insecure networks, management
and security need to be part of the integration solution. And mobile applications of-
ten must accommodate occasionally connected users, who need to work offline and
then synchronize with source systems and reconcile the offline transactions across the
systems.
Mobile integration solutions usually include a specialized server that provides the
transformation services for each target device, along with security. This server inte-
grates with back-end systems either directly (point-to-point) or through an integra-
tion platform. The mobile integration server typically uses message queues and logs
to guarantee that messages will be delivered once, and only once, even in the event of
a system failure. To support occasional users, it controls message delivery by organiz-
ing messages into transactional groups, and determines when messages are delivered
based on the type of network connection. To address the insecurity of mobile commu-
nications, the ability to encrypt data is important.
65
Chapter 4
Guide to Integration Technologies
Composite Application Services
The term “composite application” refers to an application assembled from components
that may originate from various underlying systems. Composite applications enable
new business solutions to be created by flexibly combining new and existing appli-
cation functionality. This allows organizations to deliver systems faster than building
them from scratch. The benefits of a composite application approach are maximized
when organizations create reusable business services—which then serve as building
blocks for a composite application—as part of a Service Oriented Architecture. As a
result, composite applications are typically associated with SOA and Web services.
Composite application services are the technical infrastructure services that support
the development, deployment, and management of composite applications. These in-
clude the ability to:
Develop new services (typically based on Web services standards)
Create service abstractions by combining several low-level services into a
more useful, reusable high-level business service
Design and model the behavior of the composite application, including how
information and messages flow across services, and how services need to be
orchestrated in support of a business process
Discover services or browse from a registry of available services
Manage policies and service level agreements to engender trust and enable
the reuse of services
Deploy the resulting solution into a managed run-time environment
The technologies most commonly associated with composite applications include
Web services, service orchestration capabilities, and service registries for governance.
Software vendors are introducing composite application platforms to offer a collection
of these technologies and services in an integrated suite.
4
4
4
4
4
4
66
Chapter 4
Guide to Integration Technologies
WeB servICes
Web services and their related standards have emerged as the de facto building blocks
for composite applications. Web services define a standardized interface (Web Services
Description Language, or WSDL), a standardized communication protocol (Simple Ob-
ject Access Protocol, or SOAP), a standardized repository for registering and discover-
ing Web services (Universal Description, Discovery, and Integration, known as UDDI),
and standardized message encoding using XML. These standards enable a Web service
to reside anywhere and be accessed from everywhere, making Web services well suit-
ed to the role of providing aggregated functionality within a composite application
and being the standard for interapplication interfaces. As a result, many organizations
today base their application development and integration efforts on Web services.
OrChesTraTIOn
To create a composite application from a set of Web services, it is necessary to define
the flow of control across services. Doing this in programming code reduces the flex-
ibility to change the application, so an externalized approach to service orchestration
is preferable. This latter approach basically allows developers to modify the way ser-
vices are linked together without changing the services themselves. Business Process
Execution Language (BPEL), a vendor-led initiative, is gaining mind share and market
share as the orchestration standard. A number of software vendors now support BPEL,
making it possible to share process definitions more easily among tools.
servICe regIsTrIes fOr gOvernanCe
UDDI is the registry standard for dynamically discovering and invoking Web servic-
es. UDDI was originally created to manage the cataloging of public Web services, but
registries are also gaining ground within the enterprise. Organizations that embrace
SOA will experience a proliferation of services—and versions of services. To reap the
benefits of reusability, organizations must manage these services effectively through-
out their life cycle. Beyond basic UDDI support, commercial registry products typically
incorporate some level of Web services management capability to make the registry
more useful within the enterprise.
67
Chapter 4
Guide to Integration Technologies
In addition, some vendors use the registry as the basis for policy and governance man-
agement in their solutions so that policies—such as service level agreements—can be
enforced at run-time. This approach allows the policies to be enforced across all ap-
plications that utilize the service, rather than having each application implement the
policy independently. That means policies can be set at an enterprise level. As compos-
ite applications grow in number, registries and service governance will become critical
aspects of the integration infrastructure.
COmpOsITe applICaTIOn plaTfOrms
The development paradigm of composite applications involves assembling existing
components, or services, into a new system. However, it will often be necessary to de-
velop new services and to integrate composite applications with other enterprise sys-
tems. Therefore, both the integration vendors and the application server vendors are
beginning to position their platforms as—and evolve their offerings into—composite
application platforms. Naturally, the integration vendors take an integration-centric
view, while the application server vendors focus on the development aspects. Never-
theless, a common set of composite application platform characteristics is emerging.
These characteristics include custom development capabilities, integration capabili-
ties, process orchestration, and user interface functionality.
Depending on the vendor, the composite application platform may be based on an ap-
plication server that also includes integration and orchestration capabilities, or it may
have an integration platform as its foundation, with process management capabilities
and a registry for managing services. Because it is an emerging technology segment,
the offerings are extremely varied.
Process Integration Services
Process integration services enable organizations to automate and optimize business
processes that cross application and organizational boundaries. This helps organiza-
tions reduce operating costs, gain real-time visibility into end-to-end business process-
es, and increase business agility.
68
Chapter 4
Guide to Integration Technologies
While underlying integration technologies and composite application development
can help make IT more productive and agile, the optimization of business processes
can deliver competitive advantage and significant ROI. As mentioned earlier, in the
1950s, Dr. W. Edward Deming proved that improvements in managing business pro-
cesses lead to improvements in overall business and a significant competitive advan-
tage.
A process-driven approach to integration also improves the alignment between IT and
the organization. It starts with the development of business process models that busi-
nesspeople can understand and review. These models depict a shared understanding
of the end-to-end business process. Using process integration technology, the models
can then be automated. This significantly reduces the chance of misinterpreting the
business requirement or getting the process wrong. Since processes can span depart-
ments, business units, and organizations, no single individual may own or even un-
derstand the end-to-end process. Therefore, process integration tools need to foster
collaboration on a process model and an iterative approach to its design and imple-
mentation.
An organization’s business processes may be automated, manual, or collaborative. Fre-
quently, an end-to-end business process includes both automated and manual pro-
cesses. From an integration perspective, each type of process has different require-
ments and therefore calls for different process technologies. The technologies that
deliver process-level integration include business process management for handling
automated (and sometimes manual) processes, workflow management for manual
processes, and groupware or collaboration platforms. Some vendors offer solutions
that combine these capabilities into a single product offering. Some also offer process
simulation and analytics to provide feedback for optimizing processes.
Bpm
The business process management solutions on the market today represent a range
of process integration capabilities to support process automation and optimization.
These capabilities include modeling business processes, automating the processes
69
Chapter 4
Guide to Integration Technologies
directly from the model, and monitoring running processes. Available solutions have
varying levels of support for managing manual processes as part of the overall process
and for using simulation capabilities to optimize processes. Another evolving capabil-
ity is the ability to externalize business rules rather than having them hard-coded into
the process integration software. That way, businesspeople can directly control their
portion of the process and thereby increase business agility. For example, during the
busy season, a manager could raise the dollar amount that would trigger an approval
for an order.
Unlike BPEL, which is gaining mind share as an underlying execution language, no
widely supported standards currently exist for process modeling. One candidate, Busi-
ness Process Modeling Notation (BPMN), is working its way through the standards
process within the Object Management Group (OMG). However, few vendors currently
support BPMN. Most have their own modeling notation with BPEL support for process
execution.
As mentioned, a number of BPM solutions also include simulation capabilities to facili-
tate process analysis and optimization. Process designers can perform what-if analysis
to test and compare variations of the process for better productivity and efficiency.
Some tools also allow performance statistics from running processes to be fed back
into the simulation engine to analyze and optimize processes dynamically. This makes
continuous process improvement possible and is a key to gaining competitive advan-
tage.
BPM solutions can also be used to create composite applications in the form of busi-
ness processes that span multiple systems. The BPM tool can model and implement
the flow of control across services and applications. BPM products on the market today
generally provide additional capabilities beyond simple process orchestration in the
areas of process monitoring, management, and optimization.
WOrkflOW managemenT
Workflow management solutions focus on the efficient coordination of manual tasks.
These solutions have been available for many years and predate most BPM solutions.
70
Chapter 4
Guide to Integration Technologies
However, because end-to-end business processes frequently include automated as well
as manual processes, many integration vendors now offer BPM and workflow manage-
ment in a single platform. Vendors that focus exclusively on workflow management are
rapidly decreasing in number.
BPM solutions vary in the level of support they provide for manual processes. Robust
workflow management involves process modeling as well as the design and develop-
ment of user interfaces to manage task lists, accept or change assignments, balance
workloads, and account for changes in resource availability due to vacations, illness,
etc. Manual workflows often involve approvals, which need to be logged and audited
and require the workflow solution to understand the organizational hierarchy and ap-
proval chain. In addition, different interfaces should be available for managers and the
employees receiving the tasks.
COllaBOraTIOn plaTfOrms
Also known as groupware, collaboration software facilitates interactions among peo-
ple. In contrast with workflow, where the interactions follow established processes with
predefined paths and steps, the interactions in a collaboration scenario are generally
free-form and ad hoc. Although collaboration is not usually considered a mainstream
component of business integration, it is relevant to organizations seeking to imple-
ment integration solutions as part of a strategic initiative to optimize business process-
es. This is especially true as project teams increasingly become physically distributed.
Collaboration is also relevant to B2B, where more specialized solutions enable partners
to work together on product design or development, and other joint initiatives. Col-
laboration platforms include repositories for managing team artifacts, such as docu-
ments and design specifications; version control and security capabilities; and features
for managing email threads and discussion groups.
Business Optimization Services
Business optimization services enable organizations to align business strategies with
coordinated actions. The goal of BPM is to streamline processes, but optimizing indi-
71
Chapter 4
Guide to Integration Technologies
vidual processes does not necessarily lead to optimization at an organizational level.
The goal of business optimization is to optimize at the organizational level and to align
all parts of the organization to enhance business performance.
Business optimization services include technologies that enable business performance
management. As a discipline, business performance management includes methodol-
ogy—such as Six Sigma, Balanced Scorecard, and Lean—best practices, and technol-
ogy infrastructure.
Several developments have emerged on the technology front to support business
performance management. The most significant of these are dashboards for execu-
tives, line-of-business managers, and IT operations staff that provide real-time visibility
into the organization’s operations and associated KPIs. This capability is provided by an
emerging technology called business activity monitoring.
The well-known analyst firm Gartner coined the term business activity monitoring in
2002 and defines BAM as “the concept of providing real-time access to critical busi-
ness performance indicators, along with the supporting information to improve the
speed and effectiveness of business operations.” Advances in integration technology
have allowed organizations to instrument applications to deliver real-time informa-
tion associated with significant business events, such as the receipt of an order. BAM is
the technology that monitors the events, correlates them to identify trends—such as
a sudden, unexpected drop in order volumes—provides real-time analytical capabili-
ties and management dashboards, and alerts users when exceptions occur. Some BAM
solutions include capabilities to predict when exceptions are imminent by comparing
current operating metrics with historical norms.
BAM solutions are emerging from BPM, integration, database, business intelligence,
and systems management vendors. The market for BAM is a quickly evolving sector,
and one that is not yet fully defined. The key feature requirements for business perfor-
mance optimization include dashboards, statistical analysis of user-defined KPIs, auto-
mated and dynamic performance thresholds, and advanced correlation and predictive
monitoring to help managers identify developing concerns and react before they be-
come full-blown crises.
72
Chapter 4
Guide to Integration Technologies
Ultimately, business performance optimization must encompass all levels of the orga-
nization, both on the business side and within IT. For example, an emerging category
of solutions from software management vendors is business transaction management.
This technology enables systems administrators to view the status of end-to-end trans-
actions as they cross systems on a business process level so they can better understand
the context of the transaction.
Compliance and Industry Solutions
The previous categories all represent integration infrastructure services. They are the
technologies to use when creating business solutions. The Compliance and Industry
Solutions section of the Roadmap encompasses business applications that utilize the
integration infrastructure technologies to provide specific business applications.
The difference between these solutions and traditional business applications lies in
their process-centric nature and in the fact that the processes typically span multiple
underlying systems, organizational groups, and other entities. In other words, these are
inherently integrated applications. Compliance and industry solutions require an inte-
grated infrastructure—including process monitoring, tracking, and auditing—because
none of the individual participating systems has the total end-to-end view.
Compliance is an example of a business requirement that involves integration across a
diverse set of areas and has a strong monitoring and auditing requirement. Many inte-
gration vendors and systems integrators now offer Sarbanes-Oxley, Basel II, and Health
Insurance Portability and Accountability Act (HIPAA) solutions that make it faster and
easier for organizations to implement solutions to comply with government regula-
tions. Similarly, integration vendors and consultants are offering solutions for specific
industries—including healthcare, manufacturing, finance, and retail—as well as solu-
tions for supply chain management and customer relationship management.
Organizations seeking to implement new business solutions that require integration
will focus on vendors in this category.
73
Chapter 4
Guide to Integration Technologies
Management and Security
Every level of the integration infrastructure requires management and security. Emerg-
ing management offerings include third-party solutions for application integration
platform management, Web services management, and end-to-end transaction man-
agement at a business process level.
With the growing adoption of SOA, the need for Web services management has be-
come increasingly important. Web services management frameworks help users deal
with issues such as registering Web services across the enterprise, monitoring those
Web services in operation, and determining problems, as well as related areas such as
governance and security. Without management, the ad hoc creation of Web services
across the enterprise can quickly result in chaos, with no one, for example, having a
complete understanding of what is available, who is using what service, and which
services are secure.
Security is another extremely important aspect of integration, and integrated security
solutions are beginning to emerge. Solutions that include digital certificates and en-
cryption are especially important for B2B security. Where standards are lacking, vendor
solutions are filling in the holes, especially in the area of Web services.
Architecture and Development
The ebizQ Integration Roadmap helps explain the types of integration technologies
available. Some vendors offer only one type of technology, while others offer platforms
that incorporate many aspects of the Roadmap in their solutions.
Whether an organization is partnering with one vendor or several, taking a strategic
approach to architecture and development is a key element for success in implement-
ing the technologies represented in the Roadmap. Organizations that take a tactical
approach to developing integration solutions risk suffering the same problems expe-
rienced with developing distributed applications: multiple redundant technologies
used for different implementations, resulting in increased maintenance costs and a
more fragile architecture. Organizations that take a strategic approach to developing
74
Chapter 4
Guide to Integration Technologies
the architecture start with an overall plan and then implement it on a project-by-proj-
ect basis. This prevents the purchase of redundant technologies for different projects,
avoids overlaps in integration technologies, reduces training and maintenance costs,
leverages IT investments, and increases business agility. Rather than requiring individ-
ual project teams to research and choose their own integration technologies, they can
simply use what is already in place and focus instead on adding business functionality
and value.
A well-defined and integrated technical architecture also increases flexibility in the de-
velopment phase. If the business services conform to standard interfaces, they can be
written in a variety of languages, enabling organizations to leverage skill sets within
the organization. For example, services written in Java can be part of the same compos-
ite application as services written in .NET.
Organizations that wish to take a strategic approach to developing an integrated in-
frastructure should establish a centralized group. Some organizations may place this
group within their Enterprise Architecture Group, if they have one. However, since inte-
gration requires specialized skills, many organizations are establishing an Integration
Competency Center (ICC). Chapter 6 focuses on best practices for creating an ICC.
Conclusion
Integration is a journey consisting of many tactical implementations. Organizations
that take a strategic approach to building an integration architecture will be better
able to avoid integration mistakes, control redundancy, lower the cost of ownership,
increase ROI, and ultimately achieve greater business agility. A key element of this is
understanding the role of different integration technologies and their context within
the overall integration infrastructure. The ebizQ Integration Roadmap can guide you in
this process.
75
Executive Overview
S
ervice Oriented Architecture (SOA) is transforming IT, so it is important to under-
stand the relationship between integration and SOA. This is not a straightforward
task, however, because definitions and interpretations of SOA abound.
From a software engineering perspective, SOA is simply an approach to software de-
sign that involves assembling systems from reusable components (services) that may
originate from different sources and underlying technology environments. At this lev-
el, SOA is often associated with Web services. However, Web services address only the
low-level interactions between services; they do not address the grander architectural
and design philosophies of SOA. In other words, SOA is about much more than just
Web services.
From a broader IT perspective, SOA suggests a fundamental shift in how an organiza-
tion implements business systems—with attendant changes in technology, method-
ology, and organizational structure. An even broader interpretation is that SOA marks
the end of monolithic enterprise applications and the beginning of more flexible and
adaptable business process-centric applications.
While definitions of SOA vary, there is unanimous agreement on SOA’s value and prom-
ise. The past decade has seen unprecedented leaps in computing. More and more as-
pects of a business use computers. Additionally, the Internet has extended system inter-
faces to include connections with trading partners. Not surprisingly, this has resulted in
an unruly explosion of stovepipe applications, databases, and electronic connections.
The challenge for IT today is to create and support an infrastructure that can harness
these diverse assets and deliver on new business requirements more effectively. SOA
provides the basis for this infrastructure and offers the promise of tangible business
benefits. SOA also makes it possible for organizations to leverage IT assets so they can
rapidly deploy new business solutions as needed.
Integration and SOA
5
76
Chapter 5
Integration and SOA
With its promise of transforming the way systems are developed, SOA will benefit how
organizations approach integration. At a basic level, the growing adoption of Web ser-
vices will improve system-to-system interoperability—making it simpler to connect
systems—while the componentization of business services will make it easier to cre-
ate new business solutions from existing applications and retrieve information as the
business demands. At the same time, integration technologies play an important role
in the SOA portfolio, addressing needs such as service orchestration, service- and pro-
cess-level monitoring, service management, and the ability to make legacy systems
part of the SOA environment. Clearly, integration and SOA are highly complementary,
with SOA making it easier to integrate systems and business processes, and integration
technologies providing the essential technical infrastructure services required for SOA
implementations.
Unlike previous IT revolutions that were accompanied by massive change, SOA enables
an evolutionary approach and the opportunity for organizations to move forward in-
crementally. The key to success is to plan strategically at an enterprise level and then
implement tactically, with the appropriate best practices and governance policies and
procedures in place. This strategy delivers benefits along the way, making it easier to
justify SOA investments.
This chapter explores the benefits, design concepts, technologies, and best practices of
SOA, and shows how they relate to integration.
77
Chapter 5
Integration and SOA
Case Study 5-1
Company: WRA Electronics
Company Description
WRA Electronics is a pseudonym for a Fortune 500 global provider of electronic components
and computer products. The company serves as a supply channel partner for more than 500
suppliers and 150,000 original equipment manufacturers, contract manufacturers, and com-
mercial customers. Headquartered in New York, WRA employs more than 10,000 people in
more than 200 locations around the world.
Business Challenges
As a player in the global electronic equipment market, WRA is required to comply with RoHS,
a European industry directive restricting the use of six hazardous substances in electrical and
electronic equipment. As part of the directive, WRA needed to make all relevant information
about its products accessible to its customers for quoting, ordering, and purchasing decisions.
However, different business processes (e.g., quoting and ordering) employed different sub-
systems running on different platforms and diverse technologies, and the global components
data was located in an Oracle database.
How Integration Addressed the Challenges
WRA adopted an SOA strategy to address its requirements. Using webMethods as its SOA
platform, WRA was able to link its disparate systems using Web services and to provide a
service-oriented interface for consumer applications. The webMethods solution also provides
connectivity to WRA’s mainframe-based systems that currently cannot interoperate with Web
services. The RoHS Web service provides data from the Oracle database and plugs into the ex-
isting business processes for quoting and ordering. Any differences between the platforms are
bridged using the Web service interface, and the webMethods platform performs all necessary
mappings and transformations.
Bottom-Line Business Benefits
WRA is now able to comply with the RoHS mandate. The new system provides global en-
terprise visibility into the company’s products and parts along with RoHS information. Each
month, more than 500,000 transactions pass through the company’s internal Web services
integration environment, and the RoHS Web service is utilized more than 80,000 times.
In addition to complying with the directive, WRA has achieved significant cost savings and
efficiency improvements with its SOA approach. The system has enabled the automation of
numerous process steps, saving time and errors. Company sales personnel have also been able
to help customers make more informed purchasing decisions, increasing customer satisfaction.
By providing its customers with quick and easy access to RoHS-related information, WRA has
gained a competitive edge in the market.
78
Chapter 5
Integration and SOA
Why SOA?
SOA provides many benefits to both IT and the orga-nization at large. The most im-
portant benefit—and the one most often cited in relation to SOA—is reusability. SOA
allows organizations to do more with the resources at their disposal. It promotes reus-
ability by enabling services to be shared by different applications running on different
platforms. Reusability creates a competitive advantage by providing greater flexibility
in the way computer systems can be used to support the business.
Figure 5-1. Flexibility of SOA through reuse
For example, many organizations have stovepipe applications that address specific ar-
eas of business functionality, such as order processing, invoice handling, or customer
relationship management (CRM). Each application has its own customer data and its
own view of the customer. And each has local mechanisms for retrieving information,
such as customer details, that were developed in the environment and technology
specific to the application platform. In an SOA, these duplicated routines can be re-
79
Chapter 5
Integration and SOA
placed by a common reusable service (for example, FindCustomer) that delivers the
same functionality. Because there is just one copy of the FindCustomer service, chang-
es need only be made in one place. This reduces application maintenance costs and
enables new changes to be accommodated more quickly and easily.
Another benefit of SOA is increased agility when addressing business opportunities or
changes in business requirements. By reusing common steps and developing only the
new steps that are required, organizations can more easily modify or enhance business
processes and applications when the need arises. For example, a company that offers
home loans may have a loan approval process consisting of numerous SOA services.
If the company now wants to offer car loans, it can reuse many components from the
existing implementation, such as checking current account details or market interest
rates. This increased agility translates directly into increased competitiveness. Thus, or-
ganizations that master SOA will gain an advantage over those that do not.
A third benefit of SOA is the reduced complexity from a more orderly enterprise archi-
tecture and development approach. Modular services enable a less-complex, divide-
and-conquer approach to software development. This enables SOA to accelerate the
deployment of new application functionality, decreases the testing effort by making it
easier to test individual components independently, and lowers implementation risks
through the reuse of functionality that is already quality-tested and proven.
Another benefit of SOA is increased IT adaptability. Packaged application implementa-
tions, custom-built solutions, and system changes resulting from mergers and acquisi-
tions can be integrated more quickly and easily. SOA’s inherent platform-independent
approach provides organizations with more flexibility to use the software and hard-
ware of their choice. This allows them to engage in a multisource strategy and reduces
the threat of vendor lock-in. For solutions developed in-house, it enables programmers
to use their existing technical skill sets.
80
Chapter 5
Integration and SOA
Figure 5-2. Adding new functionality easily
Best of all, these benefits can be realized quickly. SOA is ideally suited for incremental
deployment, with investments made step-by-step for individual projects. Returns can
also be realized incrementally. This eliminates the need for costly and risky “big-bang”
software implementations, reduces investment risk, and delivers value more quickly.
Thus, SOA becomes a far more palatable investment for organizational decision mak-
ers.
SOA Explained
Although the term Service Oriented Architecture first emerged in the 1990s, the spirit
of SOA existed long before that. The underlying principles of SOA—componentiza-
tion, encapsulation, separation of interface from implementation, loose coupling, and
so on—have been promoted since computing’s infancy. These design principles were
used in the days of centralized computing, when the focus was on IT resource optimi-
zation.
81
Chapter 5
Integration and SOA
Then the landscape changed. On the technology side, the arrival of distributed com-
puting offered great productivity and price performance, and the emergence of the In-
ternet promised ubiquitous connectivity. On the business side, IT was increasingly seen
as something that could be used for business advantage rather than just back-office
operations. But these changes introduced considerable challenges in linking together
different technologies and platforms to utilize the increasingly diverse IT infrastructure
to meet new business needs. The advent of object- oriented computing paved the
way for SOA by introducing the concept of having discrete program components with
standard inputs and outputs that could be used to assemble applications.
So what exactly is Service Oriented Architecture? From a software engineering stand-
point, SOA is an IT architecture based on the delivery of reusable, well-defined busi-
ness services underpinned by IT components in such a way that the providers and the
consumers of the business services are loosely coupled and have no knowledge of one
another’s technologies, platforms, or locations.
An explanation of the term “business services” is helpful here. With SOA, IT compo-
nents are formed into services that are relevant to a particular business operation. For
example, the service for FindCustomer might involve code that locates the customer
number by searching for the customer’s name in one system, then more code that
uses the customer number to query the customer’s service subscriptions in another
application, and yet more code to retrieve the organization’s interaction history with
the customer from a third database.
A service consists of a well-defined service interface and the service implementation.
The service interface describes how to call the service. It specifies, among other things,
the location of the service and the information required to make a request to the ser-
vice and get a response. This is where Web services provide an ideal fit for SOA. The use
of Web services ensures a standard protocol for communication between the service
consumer and provider and a standard way to define a service’s interface. It is basically
an electronic description of how to call the service from other programs.
The service implementation is the actual code that fulfills the functionality of the ser-
vice. It is the logic that resides on computers somewhere on the network and executes
82
Chapter 5
Integration and SOA
when the service is called, subject to appropriate security constraints. The service im-
plementation is inherently platform-dependent. However, SOA is not concerned with
how services are implemented. Architecturally, it does not matter, for example, whether
the FindCustomer service is written in Java, C#, or COBOL. The only thing that matters
is that the service fulfills the behavior specified by its interface.
Another important term used when defining SOA is “loose coupling.” Services that are
loosely coupled can be invoked by any application on any platform in any location. All
the calling application needs to know is how to invoke the service, and any inputs and
outputs required. This enables the interoperation of different applications and services
written in different languages with different development tools and platforms. Loose
coupling also means that one service can be replaced by another—provided it has the
same signature (inputs and outputs)—without impacting any of the other connected
services or requiring them to be modified. That means incremental changes to a sys-
tem can be made without having to redevelop the entire application. The dynamic
nature of SOA is another important distinction from more traditional application de-
velopment approaches.
A final consideration when thinking about SOA is “granularity.” Granularity refers to
the level of functionality provided by a service. A fine-grained service (such as Veri-
fyZipCode) performs a very specific and discrete function, while a course-grained ser-
vice (such as ProcessInvoice) incorporates more functionality and typically maps more
closely to the notion of a business service. This distinction sets SOA apart from previous
computing paradigms by providing a common language for IT and the line of busi-
ness to discuss computing requirements. Rather than talking about low-level bits of
programming code, SOA allows application requirements to be expressed in terms of
modular, reusable business services. At a technical level, SOA enables services to be
made up of other services, which makes it possible to assemble course-grained servic-
es that map to useful business functions. An important goal when designing services
is to encapsulate business functionality at an appropriate level of granularity so it can
be reused across different solutions.
83
Chapter 5
Integration and SOA
The Role of Web Services
As mentioned, SOA is often discussed in conjunction with Web services, but the two
are not synonymous. In theory, SOA does not require Web services, and simply imple-
menting Web services does not result in an SOA. However, Web services are the first
standard for service-oriented computing to gain the near-universal support of soft-
ware vendors, end-user organizations, and standards organizations. By standardizing
essential elements of SOA, Web services remove the risk of having to bet on a particular
technology (as was the case with CORBA in the days of distributed computing). As a
result, Web services and their related standards are driving the widespread adoption
of SOA.
The Web services standards define XML-based protocols and interface descriptions
that enable services to interact. Basic Web services standards include Simple Object
Access Protocol (SOAP), Web Services Description Language (WSDL), and Universal De-
scription, Discovery, and Integration (UDDI).
XML, or Extensible Markup Language, is a World Wide Web Consortium (W3C)-recom-
mended standard for defining different types of data. XML provides a text-based way
to describe and apply a tree-based structure to information. It has several advantages
over previous approaches to describing data: It is readable by humans and machines, it
is platform-independent, and it has the flexibility to handle arbitrary types of data and
data structures. Propelled by the adoption of the Internet and open standards, XML has
emerged as the critical standard for data definition and document markup.
SDAP is a lightweight protocol intended for exchanging XML-based messages in a
decentralized, distributed environment, normally using HyperText Transfer Protocol
(HTTP), the protocol used between Web servers and browsers on the Web. HTTP was
chosen as the primary application-layer protocol for SOAP because it works well with
today’s Internet infrastructure and is firewall friendly—unlike other distributed com-
puting protocols, which are generally filtered out by firewalls.
84
Chapter 5
Integration and SOA
A SOAP message consists of a one-way transmission from a SOAP sender to a SOAP
receiver. Applications can use SOAP to create more complex interaction patterns, in-
cluding request/response and multiple, back-and-forth (conversational) exchanges.
The most common pattern is the remote procedure call (RPC) or request/reply pat-
tern, where one program (the client) sends a request message to another program (the
server), and the server immediately sends a response message to the client.
WSDL is an XML-based format for describing Web services. WSDL defines the public
interface to a Web service, which describes how to communicate with the Web service
as well as the operations and messages supported by the Web service. WSDL is typi-
cally used in combination with SOAP. A client program reads a Web service’s WSDL to
determine what functions are available on the server, and then uses SOAP to actually
call one of the functions listed in the WSDL.
UDDI is an XML-based standard for a Web services registry (i.e., directory) that allows
service providers to publish information about their services and enables potential
service consumers to browse listings of available services (using SOAP messages) to
identify the ones most suited to their application requirements. As such, UDDI is often
described as a Yellow Pages for Web services. Originally intended as a public resource,
UDDI is now more commonly used within organizations as a private registry of their
internal Web services.
Additional Web services standards are being developed in areas such as security, reli-
able message delivery, and transactions. Standards are also being created to define
the business process flow across services (for example, BPEL, which was discussed in
Chapter 4). Unfortunately, as history has taught us, standards alone do not guarantee
interoperability. Vendors may implement subsets of the standards for expediency, or
supersets of the standards for enhanced functionality, but these subsets and supersets
may actually limit interoperability.
85
Chapter 5
Integration and SOA
Figure 5-3. How Web services standards interact
To guard against interoperability issues and the potential slowdown in Web services
adoption that might result, the vendor community has formed the independent Web
Services Interoperability Organization (WS-I). WS-I’s charter is to promote the interop-
erability of Web services implementations from different vendors. WS-I creates profiles,
which are implementation guidelines for interoperability, and testing tools to verify
conformance with a profile and WS-I guidelines. Currently available profiles include the
Basic Profile, Attachments Profile, and Simple SOAP Binding Profile. In addition, work is
under way on a Basic Security Profile. WS-I profile conformance is an important con-
sideration for organizations that want to ensure interoperability of their Web services
across different vendor platforms.
86
Chapter 5
Integration and SOA
Case Study 5-2
Company: U.S. Power & Light
Company Description
U.S. Power & Light is a pseudonym for one of the country’s largest power providers. The
company manages a distribution grid that spans 11 states and provides electricity to 5 million
customers.
Business Challenges
Several years ago, the Federal Energy Regulatory Commission mandated that U.S. Power &
Light become a member of a regional transmission organization (RTO). RTOs are indepen-
dent, government-regulated organizations focused on energy transmission and distribution
for regions of the country. U.S. Power & Light needed to meet the RTO’s stringent technical
requirements in 10 months, or it risked federal fines of $1 million a day. These requirements
included providing real-time and batch information on capacity, number of units online, power
produced per hour, emissions, and outages directly from the company’s power plants. At the
same time, U.S. Power & Light needed to improve its operational efficiency to meet Sarbanes-
Oxley requirements. Redundancies in the existing system required employees to enter the
same information into 15 different systems for some transactions.
How Integration Addressed the Challenges
More than 100 new interfaces were required for the types of data and forms that U.S. Power
& Light would be exchanging with the RTO. The company chose webMethods as the anchor
of an SOA strategy that enables it to publish data in one place to support the RTO require-
ment. With the webMethods platform, all subscribing systems can now instantly retrieve the
information they require. The solution is based on XML schemas, which are translated from
the source system into U.S. Power & Light’s internal canonical forms. This enables the data to
be integrated into multiple systems without point-to-point data transformations. With the new
system, U.S. Power & Light sends its RTO hourly schedules of where it will be delivering power
and where it will be pulling power off.
Bottom-Line Business Benefits
The SOA-based webMethods platform has enabled U.S. Power & Light to be development
platform-agnostic for application interfaces. Using Web services, the company can easily con-
nect disparate systems and reuse code when automating new processes. U.S. Power & Light
met its 10-month schedule and avoided $1 million per day in potential fines. In addition, it can
now meet the Sarbanes-Oxley requirements. With all the integration points in its systems con-
nected through a common platform, U.S. Power & Light can easily monitor and document the
system and processes, errors, and other key areas as required for compliance. SOA and Web
services were two key ingredients for this success.
87
Chapter 5
Integration and SOA
Integration’s Role in SOA
Although SOA is commonly associated with the creation of new systems, SOA and in-
tegration are also closely linked. A key driver of SOA is interoperability, which of course
is at the core of integration. In driving a greater degree of system-to-system interoper-
ability through the adoption of Web services standards, SOA will ease the challenge
of integration. For reasons explained below, however, it will not replace the need for
higher-level integration capabilities. Still, organizations can increase flexibility and re-
use by taking an SOA approach to implementing integration projects. In essence, SOA
makes integration easier, and integration technologies enable SOA to achieve greater
productivity and utility.
So how does integration serve an essential role in SOA?
For one thing, as stated earlier, Web services on their own do not provide the full spec-
trum of functionality required in an enterprise SOA. Beyond the interoperability en-
abled by Web services, additional capabilities are required to assemble, orchestrate,
and coordinate Web services into more solutions in real-world business scenarios.
For example, it will be a very long time, if ever, before all enterprise data is accessible
in XML format, the data format for Web services. Different applications—built with dif-
ferent business requirements for different purposes—define business entities (such as
customers, products, or financial securities) their own way. Consequently, translation
and transformation of data into the native formats of each application is an essential
component of integration today, and will remain an essential component of SOA. Dif-
ferent data representations—in terms of syntax (the format of the data) and semantics
(the meaning of the data)—will exist as long as more than one system is involved. Since
so much data is represented in non-XML formats, XML-based translation and transfor-
mation technologies such as XSLT do not suffice. Therefore, integration technologies—
with their robust support for complex legacy data formats—are needed to provide
SOA with a wider range of translation and transformation capabilities.
Additionally, forward-migration mechanisms are needed to bring the legacy environ-
ment—that is, the non-XML, non-Web-services-based systems that will continue to
88
Chapter 5
Integration and SOA
constitute the majority of enterprise systems for a long time—into the SOA fold, since
starting with a clean slate is not a viable option. This forward migration is often ex-
pressed in terms of putting Web services wrappers around legacy interfaces to make
them available to the SOA world.
Though jumping from an existing legacy interface to one based on Web services may
be technically feasible, it may not be desirable from a financial or risk-management
perspective. Creating Web services interfaces for existing systems requires consider-
able development and investment. If the service will be reused often, the investment
will pay off over time. For some implementations, however, getting a new solution up
and running quickly is the prime requirement. When connecting to enterprise resource
planning, packaged, and legacy applications, it may be faster and easier to use existing
application adapters provided by an integration vendor than to recreate access via
Web services interfaces. Using application adapters for tactical solutions while taking
an evolutionary approach to creating Web services from existing applications will al-
low an organization to realize an early return on investment (ROI) from SOA imple-
mentations. Some integration vendors enable a more seamless transition by making
operations exposed through an adapter automatically available as Web services.
Finally, Web services do not provide the requisite infrastructure services for organi-
zations focused on using SOA to support a strategic initiative to optimize business
processes. Process optimization requires managing the end-to-end business process,
which crosses application and organizational boundaries. This is what business pro-
cess management (BPM) systems do. Web services orchestration defines the flow of
control for a composite application, but BPM goes further with real-time monitoring
management, and process optimization through process automation, human work-
flow management, and process simulation.
The relationship between SOA and integration can be summed up as follows:
SOA and integration are highly complementary. On the one hand, SOA simpli-
fies integration by driving interoperability between systems via Web services,
and by componentizing application functionality into business services, mak-
ing systems more accessible to integration. On the other hand, integration
technologies and solutions provide a migration path to SOA from the existing
installed base of pre-SOA systems.
4
89
Chapter 5
Integration and SOA
Interoperability and integration are not equivalent. Web services promote
cross-platform interoperability, but true business integration demands higher-
level capabilities such as process orchestration, process monitoring, data rec-
onciliation, and exception management. These capabilities are necessary ele-
ments of SOA-based business solutions, but they are not features provided by
Web services. Integration technologies and solutions exist to fill these needs.
The disciplines behind integration and SOA-based development are similar,
and best practices developed for integration can be applied to SOA. Organiza-
tions that take a strategic approach to integration understand that success lies
in appropriate methodologies, governance on the use of organizational IT as-
sets, and dedicated organizational structures and roles. These same principles
apply in practice to SOA. For example, the expertise and processes developed
to manage integration interfaces—to ensure that they are well specified, meet
required service levels, and so on—can be applied directly to a registry of busi-
ness services. In short, the expertise and competencies developed for integra-
tion give organizations a head start with SOA.
Enterprise-Class SOA
As stated earlier, simply making Web services available on the network does not create
an SOA. For an SOA to be effective in the real world, numerous infrastructure capabili-
ties are needed—such as data translation and transformation, process orchestration,
and the other integration capabilities described in the previous chapter—as well as
capabilities for security, versioning, auditing, management, monitoring, and ensuring
predictable quality of service. The latter capabilities are especially important for en-
abling SOA to address mission-critical business requirements.
It is useful therefore to think of an enterprise-class SOA as one that not only incorpo-
rates the business services underlying the applications delivered via the SOA, but also
provides important infrastructure capabilities for delivering the reliability, scalability,
and manageability that global organizations expect. A coherent SOA infrastructure will
also help manage the complexity of the environment, without which sustainable reus-
ability would not be possible.
4
4
90
Chapter 5
Integration and SOA
Case Study 5-3
Company: State Bank
Company Description
State Bank is a pseudonym for a European bank based in Germany. With a focus on midsize
enterprises, the bank offers its domestic and international customers the financing services
of an internationally oriented credit institution and the investments and services of an invest-
ment bank. State Bank has more than 1,700 employees and over €70 billion in revenues.
Business Challenges
State Bank was introducing SAP’s Bank Analyzer, which includes a financial database (FDB).
The FDB was to be the bank’s new core database, and all enterprise applications needed to be
connected to it. This requirement made State Bank’s previous interface model obsolete be-
cause it was time-consuming and led to a single point of failure.
How Integration Addressed the Challenges
State Bank decided that adopting an SOA would shorten the development time of the in-
terfaces, making them reusable and easier to maintain while eliminating the single point of
failure. The bank implemented webMethods Fabric as the SOA-based integration layer for
connecting its legacy systems and databases to the FDB, and used XML to map all the data
entering and leaving the integration layer. As a result, all of the bank’s systems can now com-
municate with the FDB for the SAP solution.
Bottom-Line Business Benefits
The SOA-based integration solution has replaced State Bank’s obsolete interface architecture
with a flexible, reusable system. After only a three-month implementation phase, the new
platform is already processing 170,000 transactions each month. As a result of its approach,
State Bank can more easily synchronize customer information with the customer master data,
obtain foreign exchange prices in real-time, and update new securities holdings after every
transaction.
91
Chapter 5
Integration and SOA
sOa InfrasTruCTure
Deploying an SOA infrastructure as part of the overall SOA implementation strategy is
vital if organizations are to realize the benefits promised by SOA. Although a tactical
approach might be possible in the early stages of an SOA implementation, long-term
ROI requires a proactive approach to creating the SOA infrastructure.
An enterprise-class SOA infrastructure:
Promotes architectural consistency by ensuring that Web services are de-
ployed in a manner consistent with a service-oriented model
Facilitates the assembly of large-scale, dynamic systems from Web services
Provides a central point for managing and monitoring deployed services
Delivers the performance and robustness required for business-critical sys-
tems through features such as load balancing and automatic failover between
services
Provides mechanisms whereby services can easily locate one another, regard-
less of where they are deployed in the network, and promotes loose coupling
of business-oriented services through location transparency and dynamic
binding
Provides metadata management and registry services to support the reusabil-
ity of services
Provides a framework for creating and deploying infrastructure services—ser-
vices that function between business-level services, such as transformation
and filtering
Allows services to be provisioned dynamically as business needs fluctuate
Provides security across services, addressing needs such as client authoriza-
tion
Offers various levels of logging—for example, Web services requests and re-
sponses, session-related information, and security events—to support trend
analysis, exception handling, and problem diagnosis
4
4
4
4
4
4
4
4
4
4
4
92
Chapter 5
Integration and SOA
An important requirement of an enterprise-class SOA infrastructure is that it exist in-
dependent of the business services and applications it supports. As the figure on the
next page illustrates, different Web services containers may have different capabilities
(including no native Web services capabilities at all). In addition, even when different
containers have similar capabilities (for example, support for a specific WS-* standard),
the actual implementations may vary or be incompatible because of version differ-
ences.
As a result, the only way to achieve the appropriate consistency across different service
containers for functions such as security, transactions, service monitoring, and quality
of service is to externalize the SOA infrastructure from the service providers. Conceptu-
ally, this means that instead of consumers calling Web services providers directly, calls
are made through an interception layer that provides a consistent set of infrastructure
capabilities across service providers. In addition to enabling consistency, this layer en-
ables the SOA infrastructure to do things like load-balance across different services
and provide Web services capabilities to systems that are not natively built on Web
services.
Figure 5-4. Enterprise SOA infrastructure
93
Chapter 5
Integration and SOA
As explained, the SOA infrastructure should be non-intrusive to the business services
themselves. It should also be capable of operating natively within supported platforms,
such as an application server, and as an intermediary for nonsupported platforms, such
as a packaged application or a service hosted by a third party). With this flexibility, an
SOA infrastructure can bring all the services in an organization’s environment under a
single management scope.
sOa gOvernanCe
Providing the capabilities of an enterprise-class SOA infrastructure requires a combi-
nation of standards, technologies, policies, and procedures. Although the appropriate
technologies are important, corresponding policies and procedures are just as crucial
to realizing the benefits of SOA. As Web services begin to proliferate in organizations
and services are reused in different applications and different areas of the organiza-
tion, governance becomes an essential part of the SOA strategy. So does the need to
enforce policies and procedures throughout the development and maintenance life
cycle of a Web service.
One important area of SOA governance is managing reuse. As experience with object-
oriented programming has shown, if ROI is to be realized, the discipline of reuse must
be designed into the process. Reuse does not happen by accident. When the prolif-
eration of Web services is unmanaged, it leads to duplication, overlap, and an over-
all loss of control that erodes the reusability benefits of SOA. Managing reuse means
architecting services so they can be reused, monitoring how services are reused, and
providing appropriate incentives for reuse. Although these issues are primarily ones of
methodology and process, some vendors provide capabilities to help manage reuse.
For example, by profiling the performance characteristics of a given Web service (such
as average response time), a developer can make an informed decision about whether
to use the service for a specific business requirement.
An additional governance issue relates to policy definition, implementation, and en-
forcement. Consider the FindCustomer service. A company may need to implement a
new business policy that requires customer credit card information to be transmitted
94
Chapter 5
Integration and SOA
in an encrypted format to protect customer confidentiality. The developer of the ser-
vice can easily ensure that the service receives and delivers only encrypted informa-
tion. The challenge is communicating and enforcing the policy change with individuals
who are using the earlier, unencrypted version of that service.
A centralized registry helps support the definition, implementation, and enforcement
of service policies. Putting policy rules in a centralized governance repository makes
them much easier to enforce across applications and platforms. For example, develop-
ment policies can be enforced when a service is registered, and run-time policies can
be enforced when a service is invoked. Because of this, the UDDI registry is increasingly
viewed as more than just a place to catalog services; it is also a strategic asset in the
SOA infrastructure from a governance perspective.
Enforcing security policies across services is another important governance issue. A
Web service may be created for use within a single department but then adopted by
other users, including users outside the organization. Security must be maintained
across users. An ad hoc approach to Web services management provides no mecha-
nism for implementing a security policy and enforcing compliance across the organi-
zation.
A final issue relates to the operational monitoring and management of Web services
from a service-level perspective. A Web service may be used by applications across
the enterprise that have different quality-of-service expectations regarding availability,
performance, and security. It may be possible to monitor the performance of the ser-
vice in isolation, but the more important requirement is whether the service satisfies
the service-level expectations of a particular user.
When issues such as these are not addressed at an enterprise level, and tactical Web-
services-based solutions begin to proliferate without effective governance, the ben-
efits of SOA rapidly decline. As such, the more progressive software vendors offer not
only technology solutions but also a broader portfolio of capabilities that includes best
practices and methodologies for helping organizations realize their SOA strategies ef-
fectively.
95
Chapter 5
Integration and SOA
Best Practices in SOA
By now it should be clear that SOA is an enterprise strategy, not a technology or a tacti-
cal solution. To reap the rewards of SOA, organizations must implement an enterprise-
level approach. Best practices include the following:
Create an enterprise SDA group. This group is responsible for defining the enterprise
SOA architecture and roadmap, and managing the SOA’s incremental implementation
by working with project groups across the organization. The enterprise SOA group is
similar in function and purpose to the Integration Competency Center (ICC), as de-
scribed in Chapter 6, and may evolve as an outgrowth of the ICC or the Enterprise Ar-
chitecture Group, if one exists.
Manage reuse. As mentioned earlier, reuse must be managed if any benefits are to
be realized. Among other things, managing reuse requires planning and designing for
reuse, maintaining appropriate documentation and metadata about reusable assets,
having a registry of the services available for reuse, and providing active administra-
tion. Policies and incentives for reuse are also important.
ControI redundancy. Duplication of technical and business services drastically in-
creases maintenance costs and complexity. Redundancy usually results from poor
governance and a temptation to take shortcuts. These can be avoided with strong en-
terprise controls and best practices.
DeveIop SDA competency within the organization. Loosely coupled Service Ori-
ented Architectures provide maximum business agility. However, it is not enough to
provide programmers with an integrated development environment for creating Web
services. Programmers need to understand how to design and build systems that fol-
low and take advantage of the new paradigm. For example, developers should under-
stand how to design services at the right level of granularity to maximize reuse. For
long-term success, organizations need to invest in developing a core competency in
SOA design and implementation.
96
Chapter 5
Integration and SOA
Conclusion
SOA is the undisputed path for creating an agile IT infrastructure. Although there are
many important technology considerations—including the implementation of the in-
tegration technologies represented in the ebizQ Integration Roadmap (Chapter 4)—
SOA is about more than technology. The broad, successful adoption of SOA requires a
proactive, managed approach that has implications for an organization’s development
practices, its organizational structures, and its approach to investment in applications
and software infrastructure. Taking the proper strategic approach requires an initial
upfront investment, but the benefits accrue as the number of deployments grows.
Over the long run, a strategic approach offers a much higher ROI than what might be
experienced with quick and easy tactical solutions.
97
Executive Overview
A
major theme of this book is that integration and Service Oriented Architecture
(SOA) are not shrink-wrapped solutions, nor are they simply about technology. If
an organization wants to reap all the rewards that integration has to offer, it needs to
follow best practices, avoid common pitfalls, and understand how to structure the or-
ganization for success. Risks include ignoring data quality issues, lacking clear require-
ments or well-defined project plans, not having a cohesive organizational structure,
and lacking processes and procedures for implementing integrated solutions.
Integration skills are specialized, in demand, and in short supply. One of the more im-
portant best practices to emerge is the notion of an Integration Competency Center
(ICC) within the IT organization. An ICC provides an environment in which critical skills
can be honed, knowledge and experience from previous projects can be consolidated
and reused, integration processes can be developed and tested, and governance can
be applied to reusable integration assets and business services.
Most organizations will spend considerable sums on implementation and consulting
services. However, integration needs to become a core competency within the orga-
nization so that new solutions can be easily implemented on an ongoing basis. This
chapter addresses some common integration pitfalls, explains the importance of tak-
ing a strategic approach to integration, examines various models for establishing an
ICC in an organization, defines roles and responsibilities, and identifies the critical re-
quirements for successful integration projects.
Common Pitfalls
By taking a few precautions, organizations can avoid the pitfalls that jeopardize an
integration project’s success.
Best Practices for Integration
6
98
Chapter 6
Best Practices for Integration
Data quaIity issues. Integration is pointless if the underlying data quality is poor. As
the adage goes: “garbage in, garbage out.” Before integrating systems, organizations
should first assess the quality of the original data to determine whether it meets the
usage needs elsewhere. Some level of data cleansing in the source systems is often
necessary to avoid downstream exceptions that reduce the efficiency of an automated
integration process.
Lack of cIear requirements. When an integration project is started while an appli-
cation is still being developed or customized, the integration requirements become
a moving target that is much harder to hit. Before beginning an integration project,
organizations should first ensure that the source systems and interface requirements
have solidified. With integration, the devil is in the details, meaning that even minor
changes to the requirements can entail significant rework of the design and solution.
Poor project pIanning. Organizations frequently fail to plan for the long lead times of-
ten associated with hardware and software procurement or firewall and other security
changes. Consequently, dependency-related delays and scheduling issues are com-
mon. This pitfall can be avoided by working closely with procurement, security, and
operations management to ensure a smooth process. Another aspect of poor planning
is failing to coordinate and establish realistic deadlines when integrating with trading
partners. Organizations should work closely with their trading partners to establish
realistic project timelines, milestones, and testing processes that factor in the compli-
cations of dealing with external entities. They should also communicate frequently to
ensure that all parties are on track with the schedule and milestones. When kicking off
a project, it is a good idea to meet key trading partner contacts in person. This often
reduces the communication barriers during the project.
Poor coordination between I7 and business stakehoIders. Inadequate communi-
cation between IT personnel and key stakeholders in the business opens the door for
future scope changes. This is a potential issue on all projects, but integration projects
are particularly vulnerable because of the number of related parties involved. Organi-
zations should identify the key stakeholders and decision makers for all the interested
parties, and establish a formal process to obtain sign-off.
99
Chapter 6
Best Practices for Integration
Lack of deveIopment standards and guiding principIes. Naming standards, config-
uration management standards, and code promotion standards are critical for agility
and reuse. Organizations need to establish standards and guiding principles early to
ensure a consistent and orderly solution.
Taking a Strategic Approach to Integration
Historically, most integration projects have been undertaken as tactical projects, or as
tasks within a larger implementation project. This is quickly changing because of the
inefficiency and cost of taking a piecemeal approach to integration. Treating each inte-
gration requirement on a project-by-project basis does not afford the opportunity for
reuse from one application to the next. It also increases maintenance costs, since there
is no opportunity for reuse and no way to gain economies of scale.
A tactical approach also inhibits the development of integration skills in an organi-
zation. Although application developers have been doing integration for many years,
most of this has been done in a point-to-point, application-to-application manner us-
ing custom code, application- specific programming interfaces, or flat-file data trans-
fers. As a result, integration skills tend to be specialized and scattered throughout an
organization’s IT community rather than centralized in one group. Organizations today
need individuals with the skills required by more sophisticated software and integra-
tion platforms that offer built-in support for integration-specific requirements such as
transactions, auditing, business process modeling, and process automation. However,
few organizations have invested in the resources required to support these new tech-
nologies.
The increasing complexity of integration requirements is also fueling the push toward
a strategic approach. Rather than focusing simply on linking two or more applications
together, integration today seeks to automate process flows both within the organiza-
tion and across organizations (for example, integration with customers and suppliers).
If organizations want to more efficiently build and maintain the complex integration
processes that their businesses demand, they can no longer treat integration as a series
100
Chapter 6
Best Practices for Integration
of heterogeneous, project-driven tasks. Organi-zations that take a strategic approach
to integration by creating a separate organizational entity to focus on integration is-
sues and promote best practices throughout the organization will benefit through
lower maintenance costs, greater skills development, and increased business agility.
Organizing for Success
When it comes to creating an organizational structure to support integration, there is
no “one-size-fits-all” approach. An integration team can range from a few resources to
several dozen and can vary in the level of services offered—from simply developing
guidelines and best practices to serving as a full-scale shared services provider that
offers hosting and administration for all of the organization’s integration needs. Re-
gardless of the team’s size or the services offered, best practice recommends that an
organization create an Integration Competency Center. The actual composition and
services of the ICC will depend in large part on the IT organization’s infrastructure. That
is, the ICC model can be centralized, distributed, or a hybrid of the two.
In the distributed model, an organization’s IT resources are divided along functional
areas, product lines, business units, or geographical units. IT groups operate fairly au-
tonomously, and there is little coordination among them. In this model, the ICC estab-
lishes best practices that standardize processes (such as development methodologies
and naming conventions) across project teams.
Although the ICC may not be able to enforce policies and standards in a distributed
IT organization, it can provide a set of integration guidelines for the independent IT
organizations. This increases the likelihood that individual integration projects are ar-
chitected properly. The ICC may also make technology recommendations, though ul-
timately the project teams implement the technology and thus retain a fair degree of
autonomy. The benefit of this model is that knowledge is shared and some procedural
reusability is achieved.
101
Chapter 6
Best Practices for Integration
In a variation of the distributed model, the ICC defines not only the standard proce-
dures but also the standard integration technologies that project teams should use. As
before, however, the actual implementations are done by the project teams. Because
the integration platform is not under the ICC’s control, the ICC needs to offer services
such as capacity planning, disaster recovery planning, and deployment assistance to
the external integration teams. The ICC basically acts as an internal consulting group to
the distributed IT organizations. This scenario provides a greater degree of consistency
than the previous one.
In the hybrid model, some of the integration implementation workload shifts to a cen-
tralized group of development resources. As before, the ICC establishes the methodol-
ogy and software standards. However, in this model it also contributes to the develop-
ment of shared integration assets, with resulting benefits to reusability.
In the centralized model, the ICC acts as a full-service provider. Not only does it set
the standards, but it also delivers architecture, design, development, and operations
support services. All or most of an organization’s application development and IT infra-
structure is consolidated into one organizational IT group. A full-service ICC typically
performs the following functions:
Running the integration pIatform as a pubIic utiIity. A centrally run and
administered integration backbone has been likened to a public utility. Just as
a telephone can be plugged into a wall jack and receive phone service, devel-
opers can plug systems into the integration backbone and tap into the stream
of data flowing through the integration platform. In a highly centralized IT or-
ganization, the ICC is responsible for building and maintaining the integration
backbone.
Dffering standard interfaces to organizationaI systems. The ICC builds and
maintains a library of standard interfaces into enterprise systems that applica-
tion developers can reuse in their projects.
Providing data center operations for integration projects. The ICC main-
tains the shared integration infrastructure and allocates development, test,
and production environments for integration projects. It also assists the IT
operations staff with production support and monitoring once the integra-
tion project is in production.
4
4
4
102
Chapter 6
Best Practices for Integration
Providing integration deveIopment services. The ICC acts as an internal
consulting group to project teams, providing architecture, design, and devel-
opment services either directly through its own staff or by contracting with
preferred systems integration partners and off-shore development compa-
nies.
Maintaining the enterprise information modeI. In an environment where
multiple integration processes publish and subscribe to data, the ICC must
develop and maintain a business object model (also referred to as a canonical
data model) of all the documents and data to be exchanged through the in-
tegration platform. This role is critical to maintain data consistency and a high
level of reusability.
A centralized model provides several benefits. Policies, procedures, and standards are
easily set and administered. Additionally, having all the services under one roof and
point of control maximizes the opportunity for reuse and consistency. The downside of
this model, as with any centralized service organization, is the increased effort required
to effectively manage and respond to the project teams’ commitments and priorities.
In the absence of a formal ICC, project teams operate entirely independently, establish-
ing their own development practices and making one-off technology choices. Allow-
ing project teams to innovate without constraints may have some benefits. But this
scenario runs counter to the SOA philosophies of interoperability, governance, and re-
use. As such, it might be applied to a select subset of projects but seldom across-the-
board.
4
4
103
Chapter 6
Best Practices for Integration
Case Study 6-1
Company: MK Healthcare Products
Company Description
MK Healthcare Products is a pseudonym for a Fortune 50 manufacturer of healthcare prod-
ucts. The company was founded over a century ago and currently has more than 200 operat-
ing companies grouped into segments focusing on pharmaceuticals, consumer products, and
medical devices and diagnostics. MK Healthcare Products employs approximately 109,000
people in 57 countries and sells products throughout the world.
Business Challenges
MK Healthcare Products is well known for its highly decentralized management structure. A
combination of organic growth, acquisitions, and a global market has increased the need for
collaboration of business processes, applications, and data across the organization’s multiple
operating companies. In the past, the operating companies developed new processes and
services individually, without a common platform, common technology, or business processes.
Each operating company had its own integration strategies, architecture, tools, and standards.
This often resulted in systems that could not integrate easily with one another, and it signifi-
cantly limited the organization’s ability to maximize knowledge sharing, reduce redundancies,
and improve business processes.
How Integration Addressed the Challenges
MK Healthcare Products created an Integration Services Group, whose charter was to create
a strategy and platform for integrating business processes and applications. It adopted a total
business integration methodology based on an enterprise application integration solution to
connect the operating companies with one another and with external business partners. The
methodology uses standards, processes, a common design approach, templates, and a philoso-
phy of reuse and covers the entire integration life cycle, from project initiation to deployment.
By moving toward a centralized approach to enterprise integration, MK Healthcare Products
was able to standardize on an integration platform, consolidate hardware, share code and
components, combine education and training initiatives, and leverage best practices across
the operating companies.
Bottom-Line Business Benefits
The centralized Integration Services Group has made tremendous gains for MK Healthcare
Products. To date, more than 120 projects have been implemented using the total business
integration methodology and architecture, resulting in a cost avoidance savings of $9.1 mil-
lion. The new, integrated systems process 300 million transactions per week across all projects,
which equates to $30 billion in transactions annually—approximately half of MK Healthcare
Products’ annual revenues.
104
Chapter 6
Best Practices for Integration
Integration Roles
Regardless of the model chosen, an integration team should incorporate the following
roles and responsibilities:
ßusiness anaIyst. With integration requirements increasingly being driven by busi-
ness process automation, the integration team or ICC should include resources with
strong business domain expertise and analysis skills. The primary responsibilities of
the business analyst are to understand the business needs, define the related busi-
ness processes and integration requirements, and perform the associated data-level
analysis.
AppIication speciaIist. The application specialist builds the individual components
and Web services from which higher-level integration applications and processes can
be assembled. Typically, the application specialist develops business-level services that
are associated with one or more applications, so he or she needs to have some level
of application domain expertise. Services may be created from scratch (for example,
writing a new customer lookup service), by recombining existing services into new
services, or by wrappering existing functionality with Web services interfaces.
Integration speciaIist. While the application specialist focuses on developing reus-
able business services, the integration specialist is the middleware expert. The inte-
gration specialist develops custom adapters and interfaces, reusable infrastructure
services to facilitate different integration scenarios, and other utility integration ser-
vices to enable the implementation of more complex business process and integration
solutions. The integration specialist also handles the detailed technical engineering
of the implementation, conducts capacity planning and other system-level tests, and
provides some operational support. The integration specialist’s expertise is generally
centralized within the ICC. In some organizations, the application and integration spe-
cialist roles are filled by the same individuals.
Process-oriented deveIoper. The process-oriented developer bridges the gap be-
tween the business analyst and the application and integration specialists, turning
105
Chapter 6
Best Practices for Integration
business process models into executable business processes that link together the ser-
vices developed by the application and integration specialists. The process-oriented
developer needs both a business-level understanding of the business process and the
technical expertise to assemble and orchestrate the underlying services into a pro-
cess.
Integration architect. The integration architect plays a critical role within the ICC. This
individual defines the overall integration strategy and architecture, performs design
reviews of integration implementations, and mentors the rest of the IT organization
to ensure that the integration software is appropriately leveraged and properly imple-
mented and that integration solutions are architected to take best advantage of the
integration infrastructure. In addition to driving standard integration practices and
methodologies, the integration architect is often responsible for educating the IT com-
munity on the merits of the integration platform and evangelizing the adoption of the
software throughout the organization.
DperationaI support speciaIist. The operational support tasks that the ICC takes on
can vary widely, depending on its scope. Therefore, the number and composition of
the operational support team will vary accordingly. At a minimum, the ICC should in-
clude resources that can provide integration platform deployment guidance to project
teams and data center personnel.
Project manager. The staffing of project managers within the ICC also depends on its
charter. If the ICC is responsible for providing hands-on consulting to the IT commu-
nity, then project managers need to be on board.
ReIationship manager. A strong vendor relationship is critical to the successful adop-
tion of new technologies. The relationship manager acts as the central contact between
the organization and the vendor. He or she keeps the vendor informed of issues that
arise within the organization and notifies the organization about the vendor’s current
and future product and service offerings.
106
Chapter 6
Best Practices for Integration
Responsibilities of the ICC
The individual responsibilities that an ICC takes on will depend on the organization’s
stated goals for the ICC as well as the nature of the organization’s IT group. This section
describes the major areas of responsibility for most ICCs and lists possible objectives
for each area.
develOpmenT sTandards and praCTICes
An important function performed by an ICC is developing standards to facilitate reuse
and interoperability. Responsibilities in this area include:
Creating and maintaining integration best practices—for example, standards
for documenting an integration, naming standards and conventions, proce-
dures for promoting code to production, and instructions for building security
into integration processes
Creating interoperability techniques to standardize the integration of hetero-
geneous platforms—for example, integrating between the webMethods Fab-
ric platform and IBM MQ Series-based applications
Developing architectural patterns for common integration scenarios—for ex-
ample, real-time data lookups, flat-file integration processes, batch integration
processes, and transactional business processes
Developing and maintaining an organizational business object model to al-
low disparate integration processes to interoperate easily
Developing, harvesting, and maintaining reusable integration components
gOvernanCe
Another important function of an ICC, especially in relation to SOA, is in the area of gov-
ernance. Governance—essentially, processes for managing and controlling the use of
enterprise assets—is essential for ensuring reusability, long-term maintainability, and
productivity. Governance responsibilities of the ICC include:
Establishing policies for how an organization’s services are built and main-
tained.
4
4
4
4
4
4
107
Chapter 6
Best Practices for Integration
Defining and managing SOA domains, which are sets of services related by
a common business context. The domain owner brokers the relationship be-
tween line-of-business users and the owners of the applications on which the
domain services are built.
Managing the organization’s business services repository and the associated
metadata, and ensuring that development practices incorporate the appropri-
ate use of the repository.
Establishing service level agreements for business services.
COnsulTIng and TraInIng
The ICC can act as an internal consulting group to an organization’s IT community, pro-
viding expertise on the use of integration tools and techniques. Services include:
Providing architectural guidance and review for individual projects
Developing and delivering custom training materials
Providing in-house first- and second-tier support for integration tools
Assigning individual developers to project teams to build integration processes
CusTOmer advOCaCy and vendOr managemenT
As mentioned earlier, continued success with a chosen technology is often predicated
on a strong relationship with that technology’s vendor. To facilitate the relationship,
the ICC should act as the main contact between the organization and the vendor. Re-
sponsibilities include:
Working with the vendor to help market the benefits of both the products and
the ICC’s services to the organization
Prioritizing the organization’s enhancement and feature requests, and negoti-
ating with vendors
4
4
4
4
4
4
4
4
4
108
Chapter 6
Best Practices for Integration
Participating in vendor early-release programs and planning for the rollout
and migration of new and upgraded projects
TeChnOlOgy leadershIp
The ICC should constantly look for advances in integration technology that can benefit
its organization. The rest of the organization should view the ICC as an expert body for
advice on implementing and deploying integration technologies. Services in support
of technology leadership include:
Writing white papers on emerging technologies and trends
Hosting internal information-sharing sessions
Staying up-to-date on new vendor features and capabilities
Developing prototype applications using new technologies
OperaTIOnal suppOrT
The area of operational support is where the ICC’s responsibilities are most likely to
differ from one organization to the next. At one end of the spectrum, the ICC may own
and administer the hardware on which the integration platform is deployed. It would,
in essence, create the physical infrastructure on which the enterprise integration back-
bone is deployed. Project teams would request development and production space on
the shared platform and then turn over their integration processes to the ICC to deploy
and monitor in production. In this scenario, ICC services include:
Procuring and maintaining the physical integration platform infrastructure
Providing production support for integration processes built by project
teams
Allocating development and testing environments for project teams
At the other end of the spectrum, the ICC could simply provide integration platform
deployment guidance to the data center team, which would deploy the integration
processes. ICC services in this scenario might include:
4
4
4
4
4
4
4
4
109
Chapter 6
Best Practices for Integration
Planning capacity for integration platform components
Planning for disaster-recovery scenarios
Developing fail-over and load-balancing environments
Providing production support for integration platform software—for exam-
ple, data center personnel would be responsible for hardware and operations
system support, integration developers would support the integration logic
they have developed, and the ICC team would be responsible for supporting
the integration platform software
Creating an ICC
There are two general approaches to forming an Integration Competency Center. An
ICC can be created independent of any one integration project. Or it can be established
in concert with an organization’s first integration project and then evolved. We have
found that it is better to take an evolutionary approach and allow the ICC to develop
from the pilot integration project. This increases the opportunities for success, for sev-
eral reasons:
The quality of the standards, practices, and reusable frameworks produced by
the ICC tends to be higher because they reflect the team’s actual experience
working with the integration tools in a project setting.
Members of the IT community are more willing to accept the ICC’s guidance
because the staff has real-world experience.
Management is more likely to approve funding for the ICC because it is con-
tributing immediately and directly to project deliverables. As a result, the ICC’s
return on investment can be achieved quickly.
Once the original project has been completed, the ICC can be transitioned to an inde-
pendent body and can continue refining the methods, tools, reusable architectures,
and infrastructures that were established for the pilot project.
The following example illustrates how to build a full-service ICC using an evolutionary
approach.
4
4
4
4
4
4
4
110
Chapter 6
Best Practices for Integration
pIlOT prOjeCT
In this phase, the ICC team may consist of only one or two resources working with
the pilot project team. The ICC’s aim is to help develop standard architectures, devel-
opment and deployment techniques, and the beginning of an enterprise information
model. The ICC staff should include a professional services resource from the integra-
tion platform vendor to facilitate the knowledge transfer of best practices and archi-
tectural techniques to the ICC team.
TransITIOn TO The IndependenT OrganIzaTIOnal unIT
Once the pilot project has been completed, the ICC moves to an independent organi-
zation, gaining resources by transitioning some of the pilot project team members into
the ICC. As this phase begins, it is important to formalize the ICC’s charter and clearly
spell out the services that the team plans to offer the IT community. In this phase, the
ICC:
Assesses best practices and standards developed in the pilot project, makes
any changes necessary to render them generic, and publishes them for use on
future integration projects.
Builds architectural frameworks around common patterns of integration, or-
ganizational systems that are widely integrated to, and common technologies
(for example, Web services, MQ Series, and CICS mainframe applications). This
will allow future integration projects to be more productive as well as ease
maintenance and support, since all integration processes will be based on a
common framework.
Begins building the reuse library by harvesting integration components from
the pilot project and making them more generic, if necessary, for reuse on
future projects.
OngOIng suppOrT
As the ICC becomes well established and its standards and practices are deployed, the
team continues to provide support to the organization and:
4
4
4
111
Chapter 6
Best Practices for Integration
Stays current on integration industry trends and practices
Participates in vendor early-release programs to plan for upgrades and to ad-
vise project teams on the use of new features
Continues educating the IT community on integration technologies and best
practices
Continues building the reuse library by gathering components from project
teams as well as assessing the need for common integration processes and
developing them from scratch to offer on future projects
Maintains and grows the enterprise information model to ensure that the in-
tegration backbone provides a coherent canonical model for all the applica-
tions deployed on it
Maintains the business services repository and expands the inventory of ser-
vices available for reuse
Critical Success Factors
Integration can be a complex subject because multiple technologies may be applied
to different business scenarios. Regardless of the type or scope of the integration proj-
ect, the following factors are critical for success:
Link the ICC mission and charter to business objectives. Even when based on sound
technology and executed flawlessly, many a project has been seen as a failure because
it did not deliver on the business objectives that the organization was striving to meet.
Linking mission to business objectives is even more critical when forming an ICC, which
is essentially a cost center to the organization. The ICC’s charter and deliverables must
be aligned with the organization’s overall objectives. For example, if the orga-nization
is focused on cost savings, the ICC should strive to make integration processes more
efficient, reduce implementation costs, and so on. If the organization is focused on im-
proving customer service, the ICC should concentrate on creating standard integration
processes across all organizational systems that store customer data, building out the
enterprise information model to define a single view of customer data, and so on.
4
4
4
4
4
4
112
Chapter 6
Best Practices for Integration
Identify and communicate integration and ICC successes. For each project, specific
key performance indicators—such as reducing cost or implementation time, or im-
proving customer service—should be defined. Then the successes should be tracked
and communicated. This will build confidence in the ICC and encourage further orga-
nizational investment in integration infrastructure projects.
ßuiId a core competency in integration within the organization. ICC resources
should be involved regularly in integration projects so they can continue to increase
their skills and add to the collective knowledge of the organization. Systems integra-
tors and consultants on an integration project must transfer their knowledge to ICC re-
sources. If the ICC does not actively focus on these tasks, integration experience could
become scattered throughout the organization and be difficult to leverage across proj-
ects. Worse, valuable expertise could leave the organization altogether as the systems
integration partners complete their assignments and move on. It’s critical to invest in
training to continue building skill sets and knowledge.
Start smaII and buiId on successes. Integration is inherently complex. An organiza-
tion should not begin by trying to boil the ocean. Instead, it should start small and add
integration processes incrementally. Taking on too much too quickly can lead to fail-
ure. It is better to underpromise and overdeliver. In the early stages, the ICC should be
supplemented with experienced professional services resources from the vendor or a
systems integrator to satisfy key project team needs.
Define governance poIicies and procedures. Many benefits of SOA and integration
come from reusing existing organizational assets. However, enabling others to utilize
existing functionality for new solutions requires proactive governance of the reusable
assets. Policies and procedures should be defined, as should service level agreements.
This is a critical success factor for managing reuse.
Dbtain executive support. Integration projects routinely cross organizational bound-
aries. Automating, optimizing, and managing end-to-end business processes or provid-
ing unified access to distributed information can mean stepping on toes. Management
support is critical for helping to break down any barriers to acceptance that may exist
within the organization.
113
Chapter 6
Best Practices for Integration
Conclusion
As organizations invest more strategically in integration and SOA, many decisions and
obstacles are sure to present themselves. To successfully navigate these obstacles, it is
critical to develop and maintain integration competency within the organization. Es-
tablishing and evolving an ICC is a proven approach to promoting best practices; mak-
ing efficient use of resources and skills; and improving the consistency, reusability, and
agility that enable organizations to reap the rewards of integration. Finally, because
of the synergy between SOA and integration and the commonality of best practices,
organizations that have made a strategic investment in an ICC will find themselves
poised for success with SOA.
114
Gary So
VP, Offce of the CTO, webMethods, Inc.
Gary So is Vice President, Offce of the Chief Technology
Offcer, at webMethods, Inc, where he is responsible for
advancing the company’s status as a recognized industry
thought leader. Gary has over 10 years experience in
the integration feld, serving previously as a system
architect in corporate IT and as a director of professional
services at Active Software, Inc., before joining webMethods in 2000 and spear-heading
the development of the company’s integration methodology. Formerly VP of Strategic
Marketing, he was in charge of driving webMethods’ agenda through media and analyst
relations. Gary has a Masters degree in Computer Engineering from the University of
Toronto.
Beth Gold-Bernstein
VP, ebizQ
Beth Gold-Bernstein is a recognized expert in integration
technologies and SOA. She has over 20 years experience
working with fnancial institutions, retail chains and
manufacturers on planning, designing and implementing
large scale distributed systems and enterprise
architectures. She has developed training programs for
business analysts, architects, and implementers, and speaks nationally and internationally
at conferences and seminars. Beth is the author of “Enterprise Integration: the Essential
Guide to Integrated Solutions,” published in July 2004 by Addison Wesley. Her frst book,
“Designing Enterprise Client/Server Systems,” was published by Prentice Hall in August
1997. Beth holds an M.S. in Computer Information Systems from Bentley College.

About ebizQ
ebizQ – The Insiders Guide to SOA and Integration
ebizQ is the leading community of business and IT executives, managers, architects and developers focused on business integration and SOA. ebizQ’s analyst-centric multimedia content includes interactive training, webinars, and analysis of industry trends and best practices. You can access daily aggregation of in-house and industry-wide blogs, breaking news, feature articles and white papers. Learn more by visiting www.ebizQ.net and gain access to the following great content! Webinars: www.ebizQ.net/webinars/ White Papers: www.ebizQ.net/white_papers/ Online Curriculum: www.ebizQ.net/trainingcenter Blogs: www.ebizQ.net/blogs CIO audio: www.ebizQ.net/executive_corner/cioaudio/ Buyers Guide: www.ebizQ.net/vendors Bring the rest of your team up to speed. Attend our online companion course, a Manager’s Guide to Integration and SOA: www.ebizQ.net/managersguide or call 1-866-55-EBIZQ (1-866-55-32497)

About webMethods
Get There Faster.
webMethods provides total business integration to the world’s largest corporations and government agencies. webMethods flagship product suite, webMethods Fabric, leverages leading SOA and BPM technologies and methodologies to deliver rapid ROI to our 1,300 customers around the globe. With webMethods, customers can take a process-centric approach to their business problems, allowing them to re-use their existing IT assets, dramatically improve business process productivity and ROI, and rapidly create competitive advantage by making their business processes work harder for their company. webMethods (NASDAQ: WEBM) is headquartered in Fairfax, VA, USA and has offices throughout the United States, Europe, Asia Pacific and Japan. For more information on our company and our products, we invite you to visit our website, www.webMethods.com View our SOA webinar series – www.webMethods.com/Events/Web/GetRealwithSOA Learn more about us – www.webMethods.com/About Hear our vision – www.webMethods.com/Vision Meet our customers – www.webMethods.com/Customers View our products – www.webMethods.com/Products If you would like additional copies of the book, please email soabook@webMethods.com 

Tables of Contents
Chapter 1 The Business Imperative for Integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . Executive Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Challenges to Business Agility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Mandate for Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Integration and SOA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Evolution of Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Integration: A Business Imperative . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How to Use This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 2 Building the Business Case for Integration . . . . . . . . . . . . . . . . . . . . . . . . . . Executive Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tangible and Intangible Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Areas for ROI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 3 Common Integration Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Executive Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enterprise Application Integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mainframe Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B2B Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Single View of Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multichannel Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Composite Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Business Process Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Business Performance Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Best Practices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 5 6 6 10 11 12 14 15 15 16 18 25 27 32 32 33 34 36 38 41 44 46 48 50 52 

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Management and Security . . . . . . . . . . The Role of Web Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Compliance and Industry Solutions. . . . . . . . . . . . . . . . . . . . . . Why SOA? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Process Integration Services . . . . . . . . . . . . Executive Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Taking a Strategic Approach to Integration . . Creating an ICC . . . . . Common Pitfalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Chapter 4 Guide to Integration Technologies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 5 Integration and SOA . . . . . . . . . . . . . . . . . Business Optimization Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Best Practices in SOA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 6 Best Practices for Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Architecture and Development . . . . . . . . . . . . . . Executive Overview . . . . . . . . . . . . . . . . . . . . . Information Integration Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . Critical Success Factors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interface Integration Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Application Integration Services. . . . . . . . . . . . . . . Organizing for Success . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SOA Explained . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Executive Overview . . . . . . . Conclusion . . Responsibilities of the ICC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 54 54 61 63 65 67 70 72 73 73 74 75 75 78 80 83 89 95 96 97 97 97 99 100 106 109 111 113  . . . . . . . . . . . . . . . . . . . . . . . . . . . . Composite Application Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enterprise-Class SOA. . . . . . . . . . . . . . . . . .

(See Chapter 5. how they manufacture goods. “Nothing endures but change.. The notion of an agile business has long captured the imagination of business executives. This can only be achieved when business processes are able to be changed nimbly. Rapid changes are only possible when the organization itself is agile. The desire for greater business agility is making integration increasingly important. and how they are organized and managed. and the regulatory environment without missing a beat. Integration also enables organizations to leverage existing IT investments to streamline their processes for greater efficiency and productivity. the Greek philosopher Heraclitus said. The way business was conducted even a decade ago is no longer acceptable if an organization wishes to remain competitive.” While change may have been a constant since time immemorial. This requires an integrated infrastructure along with end-to-end visibility of the business processes across disparate systems.1 The Business Imperative for Integration Executive Overview S ometime around 500 B. The agile business is able to embrace changes in market conditions. Organizations have had to change how they interact with customers. or SOA.C. the rate of change is accelerating far faster than ever before. Agility is the combination of speed and adaptability: speed to bring new solutions to market more quickly. It also requires an approach that supports the implementation of business solutions from existing components. Integration and SOA. And that in turn is possible only when the underlying IT systems are integrated flexibly to accommodate the speed of change required. This is the idea behind Service Oriented Architecture. and this is having a profound effect on business. Business cycles are shrinking rapidly. organizational structure. and adaptability to new business requirements and competitive pressures. IT infrastructures need to utilize existing applications and systems to the extent possible. An agile business empowers the management team to focus its collective acumen on delivering substantially increased value to its stakeholders.)  .

and differentiate how their organizations do business.Chapter 1 The Business Imperative for Integration Challenges to Business Agility Currently. business agility is constrained by a number of obstacles. they are usually batch systems and are unable to respond flexibly to business needs for realtime information or new functionality. The legacy systems still in place today generally run an organization’s core back-end operations in a robust and reliable way. Often. Business responsiveness has become a function of an organization’s ability to rapidly marshal the underlying IT systems in alignment with business needs. In fact. but not business agility. and regulations. However. non-integrated stovepipe applications. This means leveraging existing assets while creating new business functionality. That unfulfilled business need gave rise to the first wave of distributed computing with the emergence of minicomputers. Department managers could purchase them with their budgets and select them based on their particular needs. the availability of  . organizational growth may be inhibited without integration. In the beginning. The Mandate for Integration One only needs to look at the history of computing and the evolution of business software to see why integration is a priority for most chief information officers (CIOs) today. and regulatory requirements. These systems were built to optimize expensive computing resources. respond faster to business opportunities. competitive pressures. challenges brought about by mergers. back-office operations were run by mainframe systems. inefficient business processes. acquisitions. get to market more quickly. in today’s business climate. and a lack of alignment between an organization’s strategic objectives and daily operations. Integration technology can play a role in removing these obstacles. They were—and remain—difficult to change and brittle to boot: Changing one thing could easily break something else. These include inflexible applications that cannot be easily changed or enhanced. a lack of visibility into business processes and operations. Integration technology is a key enabling factor in helping business and IT executives transform their organizations.

so the organizations had to find ways to keep the information in synch across the systems. which also proliferated and often became more strategic than they were originally designed to be. PCs. the result was reduced visibility and control. and Visual Basic—supported ad hoc solutions. and prone to errors that could be costly to trace and resolve. This includes improving the process for initiators such as customers. Unfortunately. it also led to islands of automation: hundreds of applications spread across the organization. These applications used organizational information and were in turn used to report on business operations. Access. But they were not under organizational IT management and were certainly not integrated consistently. Another problem was that each packaged application was designed to focus on specific department processes. Unix.  . Individual business units established their own computing facilities and application development capabilities. IT—to become more responsive to market dynamics. along with reduced economies of scale. Business agility requires the optimization of business processes end-to-end. So an easier way to integrate the disparate stand-alone systems was needed. in effect setting up shadow IT organizations.Chapter 1 The Business Imperative for Integration a packaged application that met 50% to 80% of the department’s specific needs led to the introduction of these new platforms. This gave them the desired level of agility—at least at first—to respond more rapidly and independently to new business needs. The emergence of the Internet and the associated rush to e-business further punctuated the need for organizations—and. But with different departments launching their own initiatives and often duplicating one another’s work. However. The widespread adoption of distributed systems often meant that large organizations had multiple platforms running hundreds of applications that managed similar information through different portions of various business processes. by association. resourceintensive. Rekeying the data—a means of last resort. but all too often the approach taken—was slow. and they began to proliferate throughout organizations. many of them on desktops. Low-cost desktop productivity and development tools—such as Microsoft Excel. and client server software further reduced the cost of department systems. these applications were not designed to integrate with one another.

 . Such integration “spaghetti” was difficult to manage and change. The challenge is akin to upgrading the wings of a plane while it’s in flight. and upgrading to a new version of any of the applications meant going through the process all over again. organizations are increasingly challenged to close the gap between their business needs and the lack of flexibility in their IT infrastructures. Packaged software systems were far from being turnkey solutions. Perhaps the biggest obstacle to closing this gap is that change has to be implemented while the core IT operations that support the business continue to function smoothly and seamlessly. While some emerging technologies look for a business problem to address. and suppliers as well as people in different business roles. It also took a great deal of time and was inflexible to change. it is ultimately viewed as an IT problem. integration was a business problem long before it was a technology. The number of interfaces rose exponentially with the number of systems being integrated. Although the current state of affairs is largely the result of short-term business decisions.Chapter 1 The Business Imperative for Integration partners. That’s because business change is intimately tied to the underlying IT systems’ flexibility—or lack thereof. This is where Service Oriented Architecture enters the picture. The problem was that the integration involved point-to-point hand-coding. This brings us to the present. As markets move quickly and new opportunities and competitors emerge. and developing systems that support different parts of the business. This required an understanding of the application program interfaces (APIs) of all the systems to be integrated and a high level of expertise. The integration costs of implementing the typical enterprise resource planning (ERP) system could be three to five times the cost of the software.

 . making the customer experience much more positive. for example. expanded anti-money-laundering requirements through provisions to deter illicit financial activities that support the financing of terrorism. and decisions are returned to the customer in real-time. Applications are now routed through the integration platform to RBank’s financial systems for processing. RBank also faced challenges complying with an increasing number of federal laws. retirement. internationally. Business Challenges RBank had more than 60 different customer response systems and applications. consumer finance. retail and commercial banking. The Patriot Act. By automating processes and eliminating error-prone manual steps. including customer loan applications. RBank employees can make changes to customer information. RBank can now connect directly to its agency to transmit all the required customer information and manage the verification process in real-time. for certain businesses. It provides investment management. improving customer satisfaction and growing sales opportunities. RBank has realized substantial cost savings. RBank also used the new integration solution to automate previously manual processes. RBank needed to ensure that it was submitting the appropriate customer information to its agency in a timely manner to meet the compliance requirements. Using the integration solution. in one place and have them propagated and synchronized across all customer systems. How Integration Addressed the Challenges RBank’s enterprise architecture division adopted an SOA and the webMethods Fabric integration platform to build applications faster and cheaper. and all customer service requests had to be routed to specific lines of business—a process involving several steps—before the customer was directed to the appropriate representative.Chapter 1 The Business Imperative for Integration Company Description Case Study 1-1 Company: RBank RBank is a pseudonym for the retail banking subsidiary of one of the nation’s largest financial services companies. such as address updates. Bottom-Line Business Benefits The 360-degree customer view has enabled RBank to customize services to customers as well as to cross-sell and up-sell offerings. The SOA approach and integration platform solution has also played a critical role in compliance efforts. RBank representatives now have comprehensive information on all customer interactions with the bank on one screen. and investment banking products and services to individuals and organizations throughout the United States and.

integration should play a central role in any organization’s SOA strategy. standardization is largely responsible for removing the barriers to SOA and fostering a future where systems can be more easily assembled and incrementally modified. Now. Web services simplify the task of integration. Integration solutions also contribute mature technologies—such as messaging. allowing organizations to leverage existing software assets while managing their transition to SOA. While SOA involves much more than Web services. A distributed architecture requires integration. the adoption of SOA was hindered by a lack of widely accepted standards for putting together these building blocks. in the face of growing competitive pressure and the accelerating pace of business. do not suffice. routing. Web services alone. and event management—along with organizational disciplines that are necessary for full-fledged. are long-standing best practices. Today. SOA is inherently about a distributed architecture.Chapter 1 The Business Imperative for Integration Integration and SOA At the core. such as isolating functionality to promote reusability. (See Chapter 4. they are highly complementary. however. enterprise SOA. organizations realize that they run a risk if they do not move toward SOA. with systems that span computing platforms. data translation and transformation. The concept is not new. SOA is about creating systems out of standard building blocks. In short.) Moreover. Integration software provides the bridge between the legacy systems and SOA. In the past. Organizations need an evolutionary approach to SOA that incorporates legacy (non-Web-services-based) systems. integration capabilities such as business process management (BPM) and business activity monitoring (BAM) allow organizations to realize a higher level of business productivity from SOA by enabling the optimization of business processes and the alignment of strategic objectives with operational actions. So how are SOA and integration related? In fact. Many of the principles underlying SOA. and technologies. the pressing demands of business—which necessitate increased system flexibility and adaptability—have inspired nearly all organizations to align on a single set of standards for SOA: Web services. By standardizing how systems interoperate. 10 . data sources. Guide to Integration Technologies. however.

So new or modified business services need to be delivered rapidly without affecting current operations. Integration software has emerged as an effective and viable approach to bridging organizational silos. the role of integration now also encompasses quickly assembling new solutions for the business. 4 Assemble. Not only does it enable organizations to leverage the investments they have already made.Chapter 1 The Business Imperative for Integration The Evolution of Integration An organization’s IT history remains its present. The new world order. They also need to maximize the use of in-place assets. Putting in place an SOA-based integration foundation enables the rapid assembly of new solutions from the catalog of available services. This is the concept behind SOA. 4 Integrate. Organizations seldom have the luxury of starting with a clean slate. organizations must evolve their definition of integration and its application to their business objectives. Business agility requires the ability to extend existing IT assets as reusable services. At best. When combined with BPM 11 . With the vast investments that organizations have made in systems and infrastructures. but it promotes reuse. This ability to decompose monolithic applications into reusable business services. with all the associated benefits of reduced cost and improved quality. CIOs have an overriding financial imperative to leverage existing resources to support new business requirements. with all their associated constraints. To successfully deliver business agility. To manage the full scope of the business and to increase efficiencies. But so far organizations have had only modest success in truly leveraging their existing in-place application assets—whether legacy or packaged applications—to support new business initiatives. and monitoring and optimizing an organization’s business processes and operations. organizations must integrate previously autonomous business processes and their underlying applications. contributes directly to business agility by enabling organizations to rapidly change and assemble new business processes to support changing business priorities. No longer only about integrating systems. and then to recompose them as needed. demands an SOA-based approach to integration. Integration software on its own is insufficient. therefore. it is very risky to rip out and replace complex systems that serve mission-critical business operations. it will only allow organizations to streamline their existing processes. In addition.

Again. As discussed in this chapter. Organizations need to create core competencies around integration and SOA. cheaper. The bottom line is that IT must be able to respond more quickly to changing business needs. average order amount. Integration: A Business Imperative Business agility is becoming a strategic goal to organizations in every industry. by helping identify and reduce the bottlenecks in an order-to-cash workflow. by comparing these real-time performance metrics against previously defined key performance indicators. 4 Monitor. This ability to iteratively improve the way the business operates ultimately results in greater business process productivity for the organization. or implement the necessary system solutions. much the way they did with databases and data centers in the 1980s and ’90s. opportunities. 4 Optimize. Integration is nothing less than a core requirement for enabling business agility. The glue that binds together all these business goals and implementation strategies is integration. organizations can gain unprecedented visibility into the metrics—such as order processing times.Chapter 1 The Business Imperative for Integration to enable processes to be composed from existing building blocks. to drive these improvements. 1 . organizations can create a feedback loop that facilitates iterative improvement of their processes—for example. By using the services within an SOA-based system as measurement points. SOA allows operational changes to be delivered faster. SOA also makes it uniquely possible for organizations to monitor every aspect of their key business processes and gain insight into how they can optimize these business processes. agility is enabled through an integration infrastructure implemented with SOA principles. and so on—that reflect the operational and business performance of a process. SOA plays a role by simplifying and speeding an organization’s ability to make the desired business process changes. and threats. Then. and in both the private and public sectors. It is a matter of survival. with a higher degree of quality and with a higher return on assets.

The EDI modules were installed into the infrastructure and utilized for document translation and data transformation. however. The company leveraged its existing webMethods integration infrastructure. it needed to quickly put in place a solution that would connect the two ends of the supply chain by allowing each partner to leverage its existing EDI solution. and they needed to select and migrate their current EDI ordering processes to a new partner within 60 days. The company is positioned between multiple tiers of customers—valueadded resellers (VARs). How Integration Addressed the Challenges Using an out-of-the-box EDI solution from webMethods.Chapter 1 The Business Imperative for Integration Company Description Case Study 1-2 Company: Electro Electro is a pseudonym for a leading global distributor of electronic components and computer products. To capitalize on the opportunity. This included an estimated 60% of reuse from existing integration assets. incremental revenue with a minimal investment ($50. The solution enabled Electro to not only target new customers and their potential revenues but also create entirely new business models that otherwise could not have been addressed. distributes. and end users—and a large supplier base. the project yielded productivity gains of 30% to 45%. The resulting framework was reusable and enabled Electro to provide drop-in integration capability to the targeted VARs. enterprise computing products. which sped timeto-market. it markets. 1 . Most of the VARs used EDI for business. it quickly added a number of key VARs to its customer list.000). and embedded systems. With the new EDI integration solution. Bottom-Line Business Benefits Electro created more than $200 million in new. With more than 100. Business Challenges One of Electro’s major suppliers mandated that its software customers buy through the distribution channel. which contained many key integration points and core services necessary to construct a solution rapidly. Electro saw the opportunity to provide drop-in integration capability and capture the substantial revenue stream. This forced multiple VARs to quickly choose a distribution partner for order fulfillment because they could no longer buy directly from the manufacturer. contract manufacturers.000 customers and 300 suppliers. Electro accelerated the creation of an EDI framework so potential VAR customers could connect for fulfillment. and adds value to a wide variety of electronic components. In addition.

1 . 4 Chapter 3. helps you identify the business initiatives that can be driven by integration technology. This chapter helps you build a business case for integration projects. more deeply defines the business benefits and return on investment that can be achieved through integration. Although we have changed the companies’ names to maintain anonymity. nor is it even a single project. Common Integration Scenarios. Best Practices for Integration. 4 Chapter 6. It is not a single technology that comes in a shrink-wrapped box.Chapter 1 The Business Imperative for Integration How to Use This Book This book is designed as a primer for those who need to understand integration and how to apply it in their organizations to achieve business goals. Integration and SOA. Guide to Integration Technologies. offers guidelines to ensure that enterprise integration becomes a core competency in your organization. we have provided case studies that illustrate the benefits of integration in specific situations. 4 Chapter 4. As you read this book. it will become increasingly clear that integration is a journey. process. and shows how to optimize your IT infrastructure to enable business agility. further explains the technologies available and how they relate to different business solutions. Success requires technology. remember that integration is not an end point. Building the Business Case for Integration. they are actual organizations that have used integration products to achieve their goals. and people. In closing. 4 Chapter 5. The book is organized as follows: 4 Chapter 2. Throughout the book. We sincerely hope that this book will be a valuable guide on your integration journey. defines the relationships between SOA and integration.

Before the availability of off-the-shelf integration technology. More intangible benefits include the ability to enhance the organization’s brand value by enabling modern. It can help businesses streamline operations. offer sustainable business value. With this type of integration. Organizations also need to understand that integration was a requirement long before packaged integration middleware solutions became available. and enable faster response to competitive threats. because they are hand-coded.2 Building the Business Case for Integration Executive Overview M any past investments in infrastructure technology have failed to deliver real business benefits and return on investment (ROI). the proprietary nature of each interface makes it impossible to leverage existing work when adding new systems. integration can reduce the cost and time required to implement new solutions while improving efficiency. provide better customer service through access to more complete information. In IT. And. typically meant hard-wiring the systems together in a point-to-point fashion using hand-coded interfaces. Business cases for integration should reflect a combination of tangible returns in hard dollars and more intangible returns that. implementing a software package with interfaces to other applications. although difficult to enumerate. making some organizations leery of investing in integration software. Progressive organizations are leveraging integration strategically to fundamentally differentiate their products and services and to gain competitive advantages that drive revenue growth. upgrading a software package can require integration costs several times the purchase price of the package. The net result is that 1 . consumer-oriented solutions and a higher degree of service quality. However. As a result. Furthermore. the number of interfaces involved rises exponentially for each additional system being integrated. the interfaces tend to be brittle and resistant to change. or combining systems because of a merger or acquisition. integration is a key component in the creation of a more agile enterprise.

they often provide the largest long-term benefits. it is rarely possible to build an integration value case that is purely financially based. off-the-shelf integration software solutions deliver a better ROI than internally developed alternatives. While indirect benefits may be difficult to quantify in the short term. build decision for integration software. and reduced resource needs.Chapter 2 Building the Business Case for Integration a significant percentage of IT organizations’ budgets are being consumed by the high ongoing maintenance and support costs for their home-grown integration solutions. Instead. if improved integration 1 . the overall business case for investing in an integration infrastructure is likely to have three parts: 4 Dollars representing the direct cost benefits of integration 4 Dollars representing the indirect benefits of integration 4 Soft or intangible factors that must be considered as additional weight in any judgment Direct cost benefits from an integration solution represent actual dollars saved. When making a buy vs. reduced maintenance costs. such as reduced development costs for new projects. In nearly all cases. For example. Assigning estimated values to indirect benefits is more challenging but equally essential. since the vendor can invest more R&D dollars into development and provide support for the software. Off-the-shelf integration software is also typically more functional and robust. Tangible and Intangible Benefits Because of the complexity of integration and the mixture of tangible and intangible benefits. organizations need to factor in the total cost of ownership. reduced errors and the cost of fixing errors. This chapter helps you to identify the benefits of integration and to build a business case and calculate the ROI you can expect from your investment in a packaged integration solution. The key here is to ensure that the affected business units agree with the assessed benefit value.

And later refinement will make the case more accurate. Though this may be a key organizational goal. The third part of the justification—soft or intangible benefits—is the most difficult to address. Figure 2-1. It should be possible to validate this figure as the project is deployed. One benefit should be improved customer satisfaction and. even if some benefits remain intangible and subjective. such as Sarbanes-Oxley requirements. though noncompliance would have significant implications for the business. one benefit will be reduced inventory. decision makers can still obtain a meaningful broad-brush feel as to whether the investment makes sense. in that a value judgment is required in addition to hard numbers. On this basis. but it is nevertheless an important part of the consideration. hence.Chapter 2 Building the Business Case for Integration delivers a more efficient just-in-time process for a manufacturing company. Soft or intangible factors make evaluating integration projects an art as well as a science. Nevertheless. Again. Suppose the integration project delivers a more responsive customer self-service system. the inventory manager can provide a value estimate. Or suppose the integration project is part of fulfilling legislation. it is hard to place a dollar value on this. assigning a dollar value to the benefit is difficult. Examples of benefits to consider in an integration investment decision 1 . customer loyalty.

Increased agility is more difficult to quantify. then adding the cost of maintaining those interfaces. Cost reduction can be measured in real dollars. Direct Cost Benefits The direct cost savings for IT through integration include reductions in three areas: 4 Development and implementation costs 4 Skills requirements 4 Operating costs Integration software provides a single interface point to individual applications. the number of interfaces that need to be developed. business investments are usually made on a project-by-project basis. This drastically reduces the number of interfaces required—and. To justify integration infrastructure investments. It is often easier to justify investments across projects if an organization takes an enterprise approach to integration and the recommendations come from an Enterprise Architecture Group or Integration Competency Center (ICC) within the Architecture Group. integration platforms and Web services provide a standard interface to the integration software that acts as a broker. and finally comparing the total cost with the cost of purchasing the vendor-developed and maintained adapters. look for both IT and business benefits. tested. thus. as well as benefits to the organization as a whole. but it does present challenges for justifying investments. and maintained.Chapter 2 Building the Business Case for Integration Areas for ROI Integration is a key infrastructure technology for delivering a new breed of more agile enterprise applications. IT BenefITs IT can benefit from integration through decreased costs and increased agility. 1 . First. while infrastructure investments span multiple projects. The savings can be calculated by first determining the cost of developing interfaces for each application that needs to be integrated. Instead of programming to a different application program interface (API) for each application. but it can provide potentially much larger long-term benefits and ROI.

the effort quickly expanded to include all bookings out of Reykjavik. The automation of its online airline ticketing process provided the ideal pilot project. Using a local integration partner and components from the webMethods Fabric suite.Chapter 2 Building the Business Case for Integration Company Description Case Study 2-1 Company: ScandAir ScandAir is a pseudonym for an Iceland-based airline that carries 1. standardization has brought new efficiency gains in business processes and IT. The automation has saved more than 27. Bottom-Line Business Benefits Since ScandAir’s first implementation in 2003. ScandAir’s IT department was evaluating the need for an integration platform to help solve the increasingly complex web of enterprise applications that powered the company. and ScandAir’s Web sites felt the impact. for example. These adapters were then used across the organization for other purposes. more than 150. ScandAir now processes its invoices electronically—something it did not originally foresee—and this has produced additional savings as a result of reduced postage and paper use and reduced storage space for the paper invoices. 1 . ScandAir rolled out a successful initial pilot in less than three months that automated the back-office process associated with Internet bookings in the Reykjavik sales office. After this project’s success.5 million passengers each year on its fleet of Boeing 757 and 767 jets. including the TPOS credit-card clearing system and the AMADEUS flight reservation system. This evolving business channel had the potential to reduce costs and improve efficiencies. saving development time and effort. ScandAir has sales offices and flight destinations across Europe and North America. In addition. At the same time. Business Challenges Internet bookings of airline tickets were rising.000 bookings have been completed. Managing this had become increasingly challenging. The project included developing adapters for the various back-office systems. but it was having the opposite effect. interfaced with more than 30 applications. Then the model was implemented in each ScandAir sales region. negating any business benefits possible from e-commerce. The company was looking for an area to test a new enterprise integration solution that would demonstrate a measurable ROI.500 hours of manual effort when compared with the previous processes. How Integration Addressed the Challenges ScandAir wanted to use enterprise integration to deliver a fully automated e-ticketing system that would eliminate the time spent on its current manual processes. ScandAir’s financial system. Each online sale required an average of 11 minutes of administrative back-office support. and serves tourist and business traffic to Iceland as well as international traffic via its hub in Reykjavik.

Additionally. in ROI studies conducted by ebizQ. For example.000 hand-processed transactions. which reduces manual errors and thus offers significant ROI. further reducing the requirement for special skills. In another example. a major North American transportation company eliminated 70.7 billion in revenue annually. In fact. integration software can decrease operating costs by reducing transaction error rates. by using integration software. Indirect Benefits The biggest indirect cost benefit for IT comes from the use of technology and solutions. Even 0 .6 million annual savings. An integration suite offers a standard approach that benefits not just the implementation process but also operations. the reduction in error rates alone could quickly justify the cost of the system. since a vendor is now responsible for the integration mechanism between components. when Hurricane Katrina hit the Gulf Coast in August 2005. And in 1999. a high-tech manufacturer using RosettaNet reported an 85% reduction in error rates and achieved a 100% ROI in the first year. Finding and fixing errors increases the cost of a transaction by orders of magnitude.Chapter 2 Building the Business Case for Integration Integration software also reduces skills requirements because. This may eliminate the need to hire or contract with expensive specialist personnel. a North American insurance company wanted to make accommodations for its policyholders affected by the hurricane.000 to 80. For example. such as providing a grace period on premiums. typically. reducing the cost per order from $25 to $5. It automates the transfer of information into systems and eliminates the need for rekeying. ensuing that connectivity to new versions of an application such as SAP or Siebel is maintained). Organizations that can quantify error rates in their current systems processing are often able to assign a significant value to error reduction. Finally. IT had already built a collection of Web services to access its mainframe systems in support of another large project. This amounted to a $1. representing $11. the integration software shields developers from having to deal with many underlying technicalities. 81% of Cisco’s orders were placed online with 98% to 99% accuracy. it can be relied on to adapt to new technology advancements (for example. Now it was asked to quickly build Web pages for the custom site.

Enhanced efficiency and productivity often translate into direct cost savings. New business processes can be deployed more quickly because of the reduced effort and the reusability offered by integration. and improved competitive performance. a coherent integration architecture delivers an enhanced level of system flexibility. a single member of the IT team was able to quickly—within a day or two—recombine lower-level worker services to create the necessary Web services for the Katrina site. First. the use of vendor-supported integration technology means that responding to new technology challenges is the responsibility of the vendor rather than the IT department. BusIness BenefITs The business benefits of integration include enhanced process efficiency and productivity. cutting the percentage of failed funds transfer requests due to input errors. Examples include reducing order times by removing manual order reentry at different stages of the processing cycle. and improving the speed of information sharing between different 1 .Chapter 2 Building the Business Case for Integration though the high-level Web services developed for the other project were not suitable to support the new business requirements. increased operational effectiveness. This impressed the business managers and was recognized as a Service Oriented Architecture (SOA) success story up to the CIO level. Direct Cost Benefits Integration software improves efficiency and productivity in a number of ways. Improved competitive performance can be the most valuable benefit to the organization. integration can increase levels of automation and reduce human intervention. And. Soft Benefits The soft benefits of integration for IT include increased responsiveness to the business and an increased perception of IT as a business enabler. enabling these technologies to be embraced and exploited faster and cheaper. Increased operational effectiveness often provides many indirect benefits. but also the most difficult to quantify. as new technologies such as Web services take hold. Further. of course.

In addition. In the 1950s.  . benefits might be realized by reducing the cost of sales or increasing sales volumes. high-quality goods that competed favorably against American manufactured goods.Chapter 2 Building the Business Case for Integration departments. Edward Deming. Six Sigma. Dr. for example. Organizations that adopted these practices realized measurable results. This becomes even more feasible when the process efficiency is extended to encompass suppliers and distributors. including Total Quality Management (TQM). process efficiency may also allow a reduction of inventory through just-in-time processes that rely on a high degree of automation. Assessing the ROI depends on the specifics of the situation. When business process management (BPM) functionality is layered on top of basic integration capabilities. Integration can help make a business more effective by improving business intelligence and its dissemination. the ability to pool information about a particular prospect from different parts of the business may make cross-selling or up-selling possible. W. showed that variations created at every step in a process can cause defects. Indirect Benefits Operational effectiveness creates more indirect benefits. Or the ability to provide comprehensive feedback on product performance directly from customers to Engineering may enable the development of new processes to improve product quality. and this can have very quantifiable bottom-line results. For example. In fact. a mathematician and statistician. and it can have the side effect of reducing procurement costs as well. which are expensive and time-consuming to remediate. some integration software products on the market today enable organizations to apply these quality management and measurement practices to their business processes in the act of integrating them. Deming’s work gave rise to a number of initiatives to manage and measure the efficiency and effectiveness of business. and ISO 9001. Deming’s work with Japanese manufacturers was directly responsible for the low-cost. Cost reductions would result from decreased staffing levels due to the increased level of automation or the use of a more productive process. process efficiency and effectiveness are increased. Balanced Scorecard.

Chapter 2 Building the Business Case for Integration

Case Study 2-2 Company: Annuate, Inc.
Company Description Annuate, Inc. is a pseudonym for an Australian super-annuation fund administrator. The company manages 5 million member accounts and more than 340,000 employer accounts. It provides administrative services such as bulk contribution processing, fund take-ons, benefit processing, insurance and claims management, member communications, and the management of member inquiries. Business Challenges For several years, Annuate had been vying for a major new account, the Australian Retirement Fund (ARF), with more than 600,000 members. Although Annuate had always been competitive in the tender process, it learned that its business process applications did not meet ARF’s requirements. Annuate identified its technology architecture as the main reason for losing the business. And it realized that to be competitive it would need to build a world-class technology infrastructure founded on a solid integration strategy that would enable new systems to be deployed while leveraging existing legacy systems. How Integration Addressed the Challenges Annuate deployed webMethods Fabric to integrate with a new client’s applications and facilitate realtime dissemination of information across the enterprise. This new enterprise integration infrastructure allowed it to win the ARF account and to merge 600,000 ARF records in the course of a six-month project that established the funds’ records on the core administration system, provided data confirmation and business rule coding, and made the other applications work off its Oracle-based member administration system. As part of Annuate’s integration efforts, it adopted an SOA platform that enabled the company to reuse many of the services it created. Annuate can now expose Web services through the integration of applications such as Web sites and internal administration systems, and can repurpose the components of its existing legacy systems, further leveraging existing technology. The integration also allows Annuate’s clients to view their portfolios online, get daily estimates of their account balances, view regular e-newsletters, and learn about investment changes. These online tools complement the information provided on member statements. The real-time updates of information ensure accurate information to customers and across the enterprise. Bottom-Line Business Benefits The ARF win was a direct outcome of Annuate’s clear enterprise integration strategy. The implementation of the enterprise integration solution has allowed a number of back-end systems to interact and provide a complete administration service for the client. As a result, Annuate has already seen a 30% to 40% reuse of interfaces and services created, as well as reduced maintenance costs. The entire solution also enables IT to better support Annuate’s business strategy. 

Chapter 2 Building the Business Case for Integration

Soft Benefits

Being able to serve customers’ needs faster and cheaper improves the business’s chances of beating the competition. In addition, the ability for process changes to be implemented faster at the operations level, through underlying IT mechanisms that are more flexible to change due to integration, enables business operations to respond to competitive action more quickly. Once an integration infrastructure is in place, it reduces the cost of implementing change. It could also lead to increased revenue by enabling a business unit to lead with new differentiating initiatives. These competitive gains can be evaluated in monetary terms by assessing the likely market share increase that results from operational efficiency and speed of reaction.

OrganIzaTIOnal BenefITs
Finally, there are the returns at the organizational level. Of course, the two areas already described—IT benefits and business benefits—already have an effect on the overall organizational performance. But this section concentrates on areas that apply more specifically to the organization as a whole, reflecting the brand value across all aspects of the business. These benefits include: 4 Compliance with government and industry regulations 4 Customer satisfaction 4 Exploitation of the organizational knowledge base 4 Access to new opportunities Most organizational returns can be categorized as indirect or soft benefits. Benefits may result even when integration itself is secondary to the underlying business requirement. For example, many organizations are finding unexpected benefits from their compliance initiatives. The ability to integrate different information sources improves the understanding of organizational operations and procedures. Moreover, particularly with some of the more recent forms of legislation, such as Sarbanes-Oxley and Basel II, it enables organizations to build sophisticated audit trails of their operations. This has the beneficial side effect of providing organizations with a better window into 

Chapter 2 Building the Business Case for Integration

how their business runs. In reality, any integration project that plays a key role in meeting compliance mandates is likely to be justified on a nonfinancial basis (indeed, it is often a case of organizational survival). Integration plays a significant role in customer satisfaction. It can provide a complete, organization-wide single view of the customer in real (or nearly real) time. This picture may include current product sales, product experiences, reported issues, outstanding deliveries, and many other facets of customer-related information. This capability allows the organization to provide a much higher level of service to the customer, for example through its call centers or help desks. Customer satisfaction leads to customer loyalty, which many organizations can relate to financial performance, for example by comparing the expense of securing new customers with the cost of retaining existing ones. Regardless of how meaningfully the benefit can be quantified, customer satisfaction is an important benefit of integration. Integration can also bring together knowledge from all of an organization’s business units, making it available for sharing and thereby increasing the overall value of the organizational knowledge base. This can have paybacks in improved organizational efficiency, more rational organizational structures, and a more effective workforce. For example, providing employee portals that offer information about organizational processes and procedures, such as health and safety, may simultaneously reduce education costs for new employees, help improve employee morale, and achieve compliance with legal requirements. Finally, integration can link together the entire value chain, both within the organization and outside to partners and even customers to enable to new opportunities. This wide-reaching integration makes it possible to introduce new business products or initiatives. In fact, many organizations are using integration strategically to offer new services to new markets.

Defining Costs
To evaluate ROI, the benefits that accrue from the integration project must be balanced 

although the costs from the business side of the organization are harder to evaluate. While the initial infrastructure costs may be high. that need to be taken into account in the solution. These cover the integration solution selected. Broadly. design. since integration projects deal with both aspects (for example. it is important to have both IT and business skills. encompassing planning. IT COsTs IT costs resulting from an integration project generally fall into three categories: 4 Infrastructure costs 4 People costs 4 Services Infrastructure costs include the expected software license and maintenance costs. In addition. IT costs tend to be grouped around the provision and support of the IT changes required to implement the desired operational changes. testing. This is another good reason to take an enterprise approach to integration. such as RosettaNet. It may also be necessary to acquire hardware to handle the additional capacity generated by the solution or to provide local processing power as needed.  . a wide range of people costs must be included in any evaluation. deployment. business unit and organizational costs tend to be expressed in terms of the changes required to adapt to the new operational processes and procedures. and support. business domain knowledge may be required to help IT understand the business standards. Organizations that forget the impact to business personnel generally suffer as a result of the oversight. they are easier to justify when amortized across multiple projects. At the planning and design levels. and any other new pieces of infrastructure software required. Clearly. any new application packages. business processes have to be defined or modified to produce the required business return. In contrast. These span the traditional range of project development. and this must be explained to the IT team).Chapter 2 Building the Business Case for Integration against the costs incurred. It is important to remember that both are relevant. development.

Traditionally. How Integration Addressed the Challenges DNA Corp.’s e-commerce Web site. has an installed base of approximately 180. The engine attempts to produce an assay design to match customer requirements. Pharmaceutical and biotechnology companies use its products to discover and develop new drugs more effectively. accuracy. and targeted therapeutics. and recovering from any problems that developed was labor-intensive. Business Challenges DNA Corp.’s bottom line. DNA Corp. This enabled DNA Corp. repeatability. a customer uploaded a file containing the sequence to DNA Corp. To streamline the process. These additional parameters determine the process sequences used. Externalizing the assay information from SAP provides greater flexibility and reuse.000 instrument systems in nearly 100 countries. The new system allows customers to define assay requirements through a Web front end. This is a complicated process that involves performing detailed analyses of DNA sequences. DNA Corp. DNA Corp.  . This process was error-prone. needed to streamline the process for building customized assays to improve its competitive position and reduce costly errors. which then passes an XML message to the design engine. The order was then routed via FTP through DNA Corp. The customer can validate the design and then choose to have the assay built based on optimizing cost. to reduce its order fulfillment errors by 20% and decrease overall order time from three weeks to less than five days. wanted to separate the process of managing and accessing the information about the assays from its order entry system in SAP. it adopted an SOAbased enterprise integration solution from webMethods. is a pseudonym for a life sciences company that specializes in research technologies. builds customized assays for its customers. it reduced the development time for a new assay type from 16 weeks to two weeks. The development time for each new assay could take 12 to 16 weeks. By integrating the 13 back-end systems involved. which included 13 enterprise systems from order creation through accounts receivable. and reduces the overall time to build a customized assay. diagnostic products. or other factors.’s back-office systems.Chapter 2 Building the Business Case for Integration Company Description Case Study 2-3 Company: DNA Corp. Bottom-Line Business Benefits The new enterprise integration solution has had a direct impact on DNA Corp.

BusIness ImpaCTs and COsTs The three categories most likely to be taken into account in the area of business costs are: 4 Resources to support project implementation 4 Education and training of end users 4 Operational risks  . Development resources are required to build the interfaces to the components being integrated. and to supply essential mechanisms. such as audit and problem determination trails. Examples include retaining someone to integrate a particular software package or someone skilled in the particular integration technology selected. The integration software itself has to be configured for the operating environment. Finally. Personnel are required to carry out the testing and to resolve and rework problems as required. Throughout the range of project activities. to modify the relevant IT infrastructure. Parts of the project may also deal with a specialized area of technology knowledge. deployment and support costs must include items such as help desk and operations staffing and training.Chapter 2 Building the Business Case for Integration The design phase needs to consider not just the technical architecture and approach but also the operational processes and workflows necessary to support the solution. to ensure successful operations. and in these cases it may be more financially viable to contract for the skills rather than retrain existing IT staff. Some organizations may decide to utilize systems integration experts to provide valuable guidance in the planning and design phases of the project. quite possibly with individuals who have business as well as technical skills. The test phase may require specific hardware and software. for instance to validate performance requirements and to provide a thorough quality assessment of the new software implementation. it may also be necessary to consider thirdparty service costs.

if the integration project spans business partners as well as internal departments. The education and training of the end-user community is another important cost to consider. Take baseline measurements before you begin the project. Additionally.Chapter 2 Building the Business Case for Integration The first category has already been described in the previous section. These resources are necessary to ensure not just that the right solution has been built but that the new solution is deployed in a controlled fashion to minimize disruption and business risk. you can do a baseline measurement of them. Know what you are measuring. you need to know what your current processes cost. Because integration projects span business and technical disciplines. The area of operational risks covers aspects such as the loss of productivity during the retraining and changeover to the new process. design. Since it is difficult to determine the impact of these issues and their associated cost. Best Practices The following guidelines will assist you in determining the potential ROI for your integration project. you must first know what you are measuring. Set your objectives for the ROI audit. the partners may also need education. resources to support planning. Obviously. when measuring the success of any project. and potential service disruptions due to technical problems or user errors experienced as the solution becomes fully established. retraining is an essential element to be considered. Before you can measure improvements. decision makers may simply want to note them as part of their overall evaluation criteria. then the end-user experience and procedures may remain unchanged. and deployment activities are required.  . We recommend budgeting one to two weeks for the initial estimate. We recommend a quarterly or semiannual cycle. But if new or changed business processes are being implemented. implementation. If the integration project is simply automating some back-room operations. Measure the initial performance levels of the processes under consideration for change or optimization to build a baseline for future comparison. Then determine how often you want to measure your progress against your goals. If you have already defined critical success factors.

Chapter 2 Building the Business Case for Integration Get executive commitment. he or she should be able to sell the audit findings throughout the organization and be willing to defend the logic. and triple-check all the calculations. Ideally. double-check. assumptions. One error could shoot your credibility in the entire analysis. Be sure to check. and stick to it. Keep the scope relatively small but still pertinent to the business. Be careful not to work from the goals down to the supporting numbers. Avoid analysis paralysis when doing the ROI study. Ultimately. Nothing succeeds like success. many assumptions and variables may change numerous times throughout the process. Avoid forcing numbers. identify some initial processes that are the low-hanging fruit—ripe for automation and integration. The checklist on the following page will help you build your business case for investing in an integrated infrastructure. 0 . When working on the analysis. Focus on improving business processes. To begin. Build a credible business case for integration. improving business processes will yield the highest return on investment. and its credibility will show. Make sure the first projects have a high probability of succeeding. Start small and lay the foundation for future enhancements. This person is usually the senior manager of the business unit affected by the implementation. Set a number of hours or days committed to ROI analysis. and results confidently. Identify an ROI audit sponsor. and these often will ripple to affect other parts of the analysis. Do your homework to build the analysis from the ground up.

Chapter 2 Building the Business Case for Integration 1 .

” to use a software engineering term). These scenarios can be thought of as building blocks for implementing more complex. and other technologies distributed throughout their organizations. Synchronizing data among different databases and applications 2. Addressing a particular business need may involve several different integration requirements. implementing a supply chain management solution might involve connecting electronically to suppliers and customers. It is possible to break down almost all business integration requirements—even those involving the most complex business processes—into a few basic integration scenarios (or “use cases. business applications. Integrating mainframe-based applications and data with distributed systems 4. Connecting businesses electronically and automating transactions with partners. These systems were designed to meet the needs of specific parts of the organization and were not intended to be integrated with other systems. For example. however. and integration has become a fundamental necessity for any business process that involves more than one system or the coordination of work across different groups of employees. and customers—known as business-to-business (B2B) integration  . integrated business solutions. and providing a single view of customer transactions and details across different parts of the organization. Business requirements have changed. synchronizing order details among different internal systems. managing the manual processing of order exceptions. databases. suppliers.3 Common Integration Scenarios Executive Overview M ost midsize to large corporations and government organizations have a myriad of computing platforms. Automating transactions among different internal systems—commonly referred to as enterprise application integration (EAI) 3. The basic integration scenarios are: 1.

we discuss the integration technologies in more detail. Keeping all this information synchronized is essential to the smooth execution of many business processes. Data Synchronization An organization usually has multiple systems and platforms. orders. and similar information is inevitably kept in many of them. Figure 3-1. in Chapter 4. so do the technologies available to implement them. Then. Repurposing IT assets by turning existing functions into components that are then assembled into new composite applications 8. fulfillment. and billing systems. Providing a single view of information that is distributed across different systems and databases 6. these two chapters provide a guide to choosing the right integration approaches and technologies for an organization’s specific business requirements. Information about customers. In this chapter. such as mobile devices and interactive voice response units 7. Automating and managing business processes and workflows 9. order entry. and products can span customer management. we discuss the integration scenarios and mention some of the technologies.Chapter 3 Common Integration Scenarios 5. Together. warehousing. Sharing data among multiple systems  . Extending access to systems over the Web and via other channels. Using integration to optimize business performance Just as the integration scenarios vary.

Because information resides in disparate systems and platforms. from least to most complex. it must choose which approach to take: transferring flat files.  . application integration addresses the need to link different applications involved in the same business process—for example. Some organizations have found that error reduction alone delivers a sizable enough return on investment (ROI) to justify an automated data integration solution. for example. an order management process that spans customer relationship management (CRM) and enterprise resource planning (ERP) packages. and fixing an error increases a transaction’s cost by magnitudes. Data synchronization patterns. to integrate financial and HR systems. Mergers and acquisitions also create the need for application integration.Chapter 3 Common Integration Scenarios Many organizations synchronize data by rekeying information manually into systems. or using an application program interface (API) or application adapter. Enterprise Application Integration Whereas data synchronization addresses the need to duplicate data across different systems. data mapping and transformation must be done so the information is delivered in the right format for the target platform. Once an organization decides to automate data synchronization. and many-to-many (synchronizing data from multiple systems across multiple systems). If it is bidirectional. this method has a high error rate. one-to-many (synchronizing data from one system across two or more systems). Automating data transfer provides direct cost benefits by reducing the personnel costs of both rekeying data and fixing errors. which triggers another update to System A. However. performing database-level updates. and so on. a mechanism is needed to prevent an endless loop in which an update to System A triggers an update to System B. Data synchronization can also be one-way or bidirectional. include one-to-one (synchronizing data between two systems).

extracting an order from System A’s database and inserting it directly into System B’s database. composite applications. Application integration can sometimes be achieved at the database level—for example. the ability to trace what is happening. Exchanging transactions among applications Application integration is at the core of a variety of business problems. application integration occurs at the transaction level. the programmer needs to understand the underlying structure and meaning of the data and ensure that data consistency rules are adhered to rigorously so the applications receive data in the format they require. data translation and transformation are still necessary to ensure that transactions are properly formatted for the receiving systems. While  . It can be a component of B2B applications. security. In these cases. and business process automation (as described later in the chapter). The transaction—usually in the form of a message sent from one system to another—is typically processed through an API or an application adapter that simplifies connectivity with the application. This level of integration enforces the application’s business logic. Critical components of application integration include transactional integrity. such as during the creation of an order. More typically. However. and exception-handling mechanisms.Chapter 3 Common Integration Scenarios Figure 3-2.

In general. and multichannel access. application integration. Mainframe operating environments are tightly controlled and do not lend themselves to external access beyond the originally prescribed mechanisms. Figure 3-3. application routers combine hardware and software to address simple integration needs. Application integration technologies include adapters. Mainframe Integration Mainframe integration can be part of various integration scenarios. Invasive methods require changes to the mainframe code. including data synchronization. enterprise service buses (ESBs). the unique intricacies of the mainframe environment demand additional considerations.Chapter 3 Common Integration Scenarios exceptions often must be processed manually. they should be managed as part of the end-to-end automated and integrated process. They are difficult to implement but  . However. EAI suites. message brokers. Accessing mainframe systems and data Mainframe access methods are broadly classified as invasive or noninvasive. composite applications. and application routers. it is difficult to gain access to mainframe applications from the outside because they were not designed for this. Similar in concept to network routers. B2B. such as via terminals or reports.

which use the output from reports to trigger dataoriented integration processes. which replaces a human operator with a computer program that interacts with the mainframe application through a terminal interface (for example. the implementation time frame. Noninvasive methods require few or small changes to the mainframe.Chapter 3 Common Integration Scenarios may be worth the trouble when the integration needs to support a high transaction volume. and whether the integration is part of a tactical or a strategic implementation. 4 Report-based interfaces. 4 Direct data access via gateways to mainframe-resident databases such as DB2. no additional software is required. 3270). This can be noninvasive if the mainframe application offers that interface.  . it needs to understand how this capability fits into the overall architecture. The best approach depends on the characteristics of the mainframe application. using integration technology to noninvasively make existing transactions accessible via different protocols—into reusable Web services that can be accessed from any source. or minimally invasive if implemented through a message-based interface. 4 Transaction-level integration to CICS or IMS. These products are sometimes referred to as SQL gateways. In the long run. In some cases. wrappering enables greater reusability and flexibility. Other factors may include the available skill sets. budget. Each approach to mainframe integration requires different technologies. Market consolidation has left only a handful of specialized legacy integration vendors. Though it has an initial development cost. the best solution for mainframe integration is “wrappering” existing mainframe transactions—that is. and available tool sets. Options for noninvasive mainframe integration include the following: 4 Screen scraping. Whether an organization is implementing a legacy extension solution as a tactical solution or building a legacy integration service layer as part of a Service Oriented Architecture (SOA) strategy. Many integration platform vendors partner with a specialist on OEM legacy integration technologies to offer mainframe integration capabilities as part of their broader suites.

Organizations require B2B integration when they are attempting to streamline the supply chain. connect to an electronic exchange. but EDI systems are evolving  . which are based on the volume of data transacted. EDI is used mostly by larger organizations—and by smaller organizations only when they need to do business with larger organizations. alternatives have emerged. However. With the advent of Extensible Markup Language (XML) based integration technologies that utilize the Internet and open standards rather than expensive proprietary technologies. Integrating business processes with trading partners Processing electronic transactions with business partners can be 10 to 100 times less expensive than processing the same transactions manually. suppliers. VAN.and EDI-based solutions are expensive to implement. This is due to the complexity of processing EDI and VAN charges. EDI is not going away. As a result. or comply with industry and regulatory mandates that require electronic communications to government or business entities outside the organization. integration. usually based on electronic data interchange (EDI) formats and standards. or B2B.business. Figure 3-4. and customers is commonly referred to as business-to.Chapter 3 Common Integration Scenarios B2B Integration Automating transactions with partners. Organizations typically have used a value-added network (VAN) solution to implement electronic interchange of purchase orders and other electronic documents.

rather than integrating point-to-point with all their partners and suppliers. partner management capabilities (which address the need to manage processing rules and service levels on a partner-by-partner basis). They allow all partners to integrate with the exchange. which started as an automobile industry exchange and has expanded into healthcare. and the World Wide Retail Exchange (WWRE). These exchanges offer supply chain services that enable suppliers and partners to conduct electronic business using the Internet rather than a proprietary network as the transport layer. B2B integration technologies are generally available as part of a broader integration platform or in a hosted environment through integration service providers. another B2B exchange. For example. such as firewall friendliness and the encryption of documents exchanged over the Internet. as well as standard industry protocols. WWRE offers a collaborative planning tool. Technologies for implementing B2B solutions include basic application capabilities. including EDI. the new integration technologies are making it possible for many more organizations to participate in electronic commerce. and other facilities that enable on-line trading. which was set up by 17 international retailers to automate retail supply chain processes. Today.  . This saves considerable time and money. and capabilities for handling B2B-specific security requirements. XML. such as GXS. and custom flat files. such as RosettaNet and the Health Insurance Portability and Accountability Act (HIPAA). Additional security is necessary to authenticate partners and ensure that the transactions are legal and cannot be repudiated. a logistics tool. Organizations also need a way to manage service level agreements with each partner and provide different connectivity options for small. B2B is also often associated with industry exchanges such Covisint. Exchanges may also provide additional value-added services. Together. Covisint offers a Web-based tool to coordinate and manage problem-solving processes between customers and suppliers. provides buyers and suppliers with a number of services for conducting electronic business. midsize. and large organizations.Chapter 3 Common Integration Scenarios to integrate with newer standards-based technologies. B2B integration also requires support for different data formats. Ariba. Integration with systems completely outside the organization’s control pose some challenging integration requirements.

business partners can now conduct all transactions on the B2B Web site. and submit order forecasts to IT Corp. How Integration Addressed the Challenges Using the business integration solution from webMethods. and access procurement forecasts to manage supplies. and data errors from manual processing.000 transactions each month. By completing simple online transactions. all supply chain transactions.000 and 15. view and print order acknowledgments and shipping details. In addition. and the public sector. IT Corp. The new system has effectively saved thousands of manhours of work for IT Corp. The new system processes between 10. request changes to existing orders.’s customers and suppliers through a B2B Web site.Chapter 3 Common Integration Scenarios Company Description Case Study 3-1 Company: IT Corp. 0 . Suppliers can easily view and acknowledge orders. and this has increased customer satisfaction. enterprises. Business Challenges IT Corp. The real-time nature of data flows across the system provides accurate information at all times. identified the supply chain process as business-critical and wanted to create a Web-based channel that would enable supply chain transactions between IT Corp. has been dedicated to automating the computer industry supply chain and B2B transactions. increased turnaround times. The company offers products to consumers. customers receive immediate responses and acknowledgments of process completion. This enabled the company to use all its existing business logic that resided in Baan for the new Web application. is a pseudonym for a Fortune 50 information technology company with more than 150. the automated system has eliminated time-consuming and error-prone manual processes. Customers can place new orders. This process was very inefficient and translated into high costs. Bottom-Line Business Benefits Delivered in just three months. was able to link its Baan ERP system with an internally developed Active Server Pages Web application. the Web-based B2B portal has become an integral part of IT Corp.’s supply chain system and supports a growing percentage of the company’s transactions.000 employees doing business in more than 170 countries. IT Corp. It invests heavily in R&D. provide updated shipment details. IT Corp. were handled via paper and email. from forecasts to invoices. The automated process has streamlined and simplified the purchasing process for the company’s suppliers and customers. small and midsize businesses. With the Web-based supply chain system. and its business partners. producing more than 10 patents a day worldwide. Traditionally. to streamline the procurement process.

orders. The largest partners may have full-blown B2B implementations. the goal is to make it fast and easy to connect. including full integration with their back-end systems. or methods of connecting to partners and suppliers. Figure 3-5. Some midsize partners may just need to send and receive XML messages.Chapter 3 Common Integration Scenarios Because most organizations do business with many types of partners. or integrated risk management. they need different types of on-ramps. This capability is especially important for business solutions used in situations such as mergers and acquisitions. Integration software can help organizations aggregate information from various systems to create a single view of any business entity. Ultimately. products. consistent view. employees. because the ROI for B2B implementations is directly related to the percentage of partners participating. information about customers. suppliers. and it is difficult to get a single. and other business entities becomes spread out. And smaller partners may require browser-based connectivity where they manually submit transactions or check on status. Combining information into a single view 1 . Single View of Information With the proliferation of applications and databases in an organization. CRM.

indexing. so they may lack the most current information. such as nightly or weekly. and Client in System D. such as distributed query optimization. This ensures that the information presented is up-to-date and the most current available. and to map the transformation of data as it is extracted from the sources. Some solutions include semantic integration capabilities. which have proprietary technologies and algorithms for aggregating information in realtime. for example. that the Customer field in System A is the same as Name in System B. For physical-aggregation-style approaches. traditional data integration technologies can be used to source data from—and populate data in—the databases involved.  . as if all the information were coming from a single source. A number of new solutions are coming to market that enable a real-time single view of information. and caching. which involves keeping the information in the source systems and extracting the information as needed into a combined view. CustName in System C. The tradeoff with a distributed query is that it typically takes longer to get a response than it does when making a request to a single database.Chapter 3 Common Integration Scenarios One way to gain a single view is to physically aggregate data into one place. However. Many organizations have created data warehouses and data marts to accomplish this. A major ambition for data integration is to capture the semantic meaning of information within systems in metadata. An important dimension of single-view integration is the need to understand both the underlying data structures of the various source systems and the semantic meaning of the data in each system so they can be reconciled into the single view (whether in a data warehouse or in real-time). It is necessary to determine. which allows systems to be integrated more easily without having to create each mapping manually. These include enterprise information integration (EII) solutions. data marts and warehouses are generally updated at predefined intervals. An alternative method is the distributed query.

It is the largest processor and distributor of milk and other dairy products. They can make purchases in a timely manner because they receive notification beforehand and can react more efficiently to what the company sends them. The new system enables more than 100 employees across 32 business units to create. and manage product data. Bottom-Line Business Benefits The new master-item system enables FoodCo to manage more than 35. maintaining product information was a complex and error-prone process. FoodCo can leverage its investment in internally developed systems and meet customer mandates on synchronized items. Business Challenges Over the years. For its 360.Chapter 3 Common Integration Scenarios Company Description Case Study 3-2 Company: FoodCo FoodCo is a pseudonym for one of the leading food and beverage companies in the United States. FoodCo had acquired several new businesses. resulting in a five-figure savings on annual VAN charges. The system also generates alerts to resolve exceptions that could lead to data inconsistency. Because the enterprise integration platform synchronizes the data within the existing ERP systems. FoodCo realized that it needed to create an enterprise-wide product data management process to ensure data consistency across business units and guarantee the delivery of information to the right place at the right time. Data for identical products was inconsistent among business units. How Integration Addressed the Challenges Using the webMethods integration solution. and finding and validating information that customers needed took a long time. Each acquisition brought more SKUs or UPC codes. In addition. edit. FoodCo created a single master-item system that aggregates data from multiple back-end systems and routes it to users for action.  . validate. especially when items had multiple levels of descriptions. and managing them had become increasingly challenging. Customers now see FoodCo as a single vendor with a consistent product catalog.000 customers and partners. which operate as independent business units. the system has reduced the time for new product introductions from five days to one. such as formulas for ice cream flavors.000 SKUs in nine categories and to add more than 200 new items each year. and operates more than 120 plants in the United States and Spain. The webMethods platform has also enabled FoodCo to implement a complete standards-based EDIINT/AS2 solution between VAN and non-VAN customers.

No longer tethered to an application running on a desktop computer—or. The associated technologies are expanding and changing rapidly. suppliers. personal digital assistants (PDAs).Chapter 3 Common Integration Scenarios Multichannel Access The past few years have seen a proliferation of devices through which end users and consumers can access computing services. a mainframe terminal—people can now use Web browsers. customers. As organizations start creating reusable business services. Rather than altering existing applications to allow for each type of access channel. but it also presents a significant IT challenge. mobile phones. Figure 3-6. Using different end-user channels to access back-end systems  . and partners anytime and anywhere. and organizations will increasingly need to provide services across a variety of devices based on a variety of technology environments. before that. and other special-purpose devices to receive and send information. or developing solutions that are hard-wired to a specific type of device. The expansion of access channels represents an opportunity for organizations to better serve their employees. they inevitably want to make these services available through different channels. several emerging technologies and standards initiatives allow multichannel access to different back-end systems. interactive voice response units.

Chapter 3 Common Integration Scenarios Company Description Case Study 3-3 Company: Comco.-based Fortune 100 global communications company that provides mobility products and solutions for the home. The company’s 2004 sales exceeded $30 billion. Another challenge was transforming Comco’s existing ERP system into two separate but seamless systems. the touchless system has enabled more than 1. Inc. This would involve a cutover process with minimal interruptions for each company. effectively reducing the number of people required to support the business while maintaining on-time shipments to customers. Bottom-Line Business Benefits The new system has enabled Comco to relocate its manufacturing to Latin America and avoid several million dollars in duties annually.2 million mobile phones to be shipped to Latin American customers with no direct contact by Comco. This is a significant competitive advantage for Comco. an industry standard for B2B commerce. purchase orders. To date. Inc. The company has also reduced its operating costs by close to $1 million each year through the elimination of manual processes. and workplace.S.  . one located at the new contractor’s plant. The system was also enhanced with business activity monitoring (BAM) capabilities that send alerts to business managers when process exceptions occur. Comco implemented a webMethods-based solution that enabled the new system to automatically exchange a variety of information about engineering. and other product information. Comco. plants to Latin America. order tracking and other information is provided in real-time. automobile.S. How Integration Addressed the Challenges Comco had made a formal commitment to RosettaNet. Business Challenges Comco’s mobile device business unit was losing significant margin because of duty fees for phones shipped from its U. In addition. Comco’s major challenge was to define an automated process that would transfer the manufacturing and distribution of mobile phones to the third-party contractor without reducing on-time shipments to Comco’s customers. giving the company added bandwidth to focus on new business opportunities. is a pseudonym for a U. It wanted to build on the RosettaNet work and create an enterprise integration solution that would provide touchless integration from Comco’s back-end Oracle ERP system to the third-party contractor’s back-end SAP system. The company recognized that divesting the manufacturing of these phones to a regional manufacturer would reduce its duty fee costs in the Latin America market by several million dollars each year. allowing them to react quickly and proactively.

Additionally. C#. Composite applications are assembled (or composed) from existing components rather than developed from scratch. including Web servers.Chapter 3 Common Integration Scenarios A midtier server is the least invasive and most flexible way to provide interface services to multiple types of front-end devices. Organizations therefore need to create a strategy and an architecture for multichannel access and integration. and for many it will soon become a requirement. They also incorporate functionality from a variety of underlying systems. Java. Unfortunately. The midtier server renders the interface to fit the display characteristics of each device. and mobile devices. Composite applications are the answer. the back end does not need to change the way it sends the information for each type of device. This approach to building new systems aligns closely with Service Oriented Architecture. the use of XML and style sheets enables information to be displayed in different formats for different devices without requiring changes to the underlying data. data definitions. Composite Applications Organizations seeking to rapidly implement new business solutions soon realize that they cannot pursue a rip-and-replace strategy. Their rules. organizations must leverage the systems they have in place but gain flexibility in how the systems can be repurposed. Few organizations will be able to ignore the demands for multichannel access from multiple constituencies. This way. For accessing back-end systems. To meet business needs. Web services are a good choice because they enable access from a broad range of clients-side systems. Starting fresh with each new system simply costs too much and is too disruptive to the organization. processes. SOA is an architectural approach to creating reusable building-block components  . and user interfaces are embedded in compiled code that requires programming skills and detailed background knowledge to modify. many applications hinder rather than enable business change.

selected functions from other applications.  . however. or entire systems whose business functions have been enabled as Web services (often legacy systems). Composing new applications from existing underlying business services An example of a composite application is a self-service portal for consumers or business partners. Figure 3-7. Here. and testing. programming. A composite application consists of functionality drawn from several sources within an SOA. The components may be individual Web services. this type of composition would require considerable application integration. With an SOA. Without an SOA approach. a number of existing processes supported by IT services might be combined within the portal to provide the end user with one-stop access to a range of the organization’s products and services. the composite application can be assembled easily.Chapter 3 Common Integration Scenarios as services that interoperate through standardized Web services interfaces and that can be combined quickly and easily to address new business process needs.

discover. Whereas a monolithic application typically provides tools to monitor status (for example. Business processes are the lifeblood of an organization. while eliminating manual data entry and automating transactions can yield direct and measurable cost benefits. keeps costs to a minimum. Process-level integration offers the richest business benefits. a tool set is needed to catalog. the scope and complexity of the problems that organizations are looking to solve have increased. Additionally. automating and optimizing business  . organizations are increasingly looking to integration software to streamline entire business processes. Therefore. So. for monitoring and other functions such as policy enforcement. product development. and Web service repositories that enforce governance policies. a composite application offers no such built-in monitors because the status crosses multiple services on potentially different platforms. or exchanging documents electronically with a trading partner. Organizations can increase competitiveness through differentiated business processes. Beyond ensuring data consistency. an organization’s business processes define its ability to compete successfully in the market. This allows IT to respond faster to business demands. Web services.Chapter 3 Common Integration Scenarios regardless of the location or the technology of the different services being drawn together. or service delivery. One caveat must be mentioned here. a run-time deployment infrastructure is needed to connect all the components in a scalable and reliable way. linking a few applications together. As such. and orchestrate services into a new application. integration suites. The technologies that enable composite applications include traditional application development platforms. The services from which composite applications are composed can be created by wrappering existing application functions or by building new ones from scratch. Whether related to customer service. to view the status of an order). and IT can better keep pace with changing business requirements. and powers imaginative new ways to address end users’ needs more effectively. provide security. composite application implementations need an infrastructure that exists outside the services themselves. and enable reuse. Business Process Management As integration technologies and solutions have matured. and to deploy and manage the result.

whereas other parts may involve people performing specific tasks. customer. The term business process management (BPM) generally refers to the set of activities that organizations perform to either manage or optimize their business processes and adapt them to new organizational needs.  . and handle other situations requiring context that is not easily encoded in a system. improving employee. people remain an important part of the equation. Individuals are needed to assess exceptions. BPM also refers to the category of integration software that provides these capabilities. Because these activities are usually aided by software tools. business processes are often complex and disjointed. Some parts of the process may be performed automatically by IT systems. Different organizational entities may be responsible for different parts of the process. Although organizations may want to automate as much of the process as possible. In addition. and increasing business agility Figure 3-8. Coordinating end-to-end business processes across systems and people. typical enterprise business processes run the gamut from fully automated to semi-automated to entirely manual. By nature. business processes often include both automated and human processes. with no single entity responsible for the end-to-end process. In fact.Chapter 3 Common Integration Scenarios processes holds the greatest potential for reducing labor costs and cycle times. and partner satisfaction. apply judgment and experience.

business process management solutions enable process-level optimization—in essence. managers need the ability to assign task to employees with certain skill sets. we only hit what we aim at. and provide management oversight and intervention for manual activities. best practices. Alignment across all organizational levels is required for overall business optimization. Business Performance Optimization Henry David Thoreau said. such as the different roles of managers and employees. BPM solutions should support both types of processes. For example. This focus is necessary for business performance optimization. it can be argued that business optimization should be performed at all levels of the organization. Business performance optimization is a combination of methodology. There are two levels of business optimization. a platform for automating and executing these processes. Business performance management solutions monitor relevant business events.Chapter 3 Common Integration Scenarios Since business processes are typically a combination of automated and manual tasks. and technology infrastructure. from the operational IT level. All require visibility into the KPIs for which they are responsible. balance the workloads when assigning or reassigning tasks. Human processes require a user interface. Both levels of business optimization are required. they can focus not only on what goods and services they are delivering but also on how they are delivering those goods and services. and facilities to monitor them. up to senior executives.“In the long run. and relate them to key performance indicators (KPIs). which may cross multiple processes. Typical human processes involve delegation and organizational considerations. manage their employees’ workloads. Otherwise.” When organizations have tools that deliver real-time visibility into—and performance measurements of—business processes. through line-of-business managers. streamlining the execution of business processes. BPM solutions typically provide graphical tools for modeling business processes. As discussed above. an organization risks optimizing one process while suboptimizing at an enterprise level. They may also provide simulation capabilities to test and optimize business processes. 0 . In fact.

Some BAM solutions include predictive capabilities and can identify emerging patterns that may influence a KPI. In 2002. BAM solutions allow organizations to correlate events from multiple processes and relate them to measures that are meaningful to the enterprise. Gartner coined the term BAM and defined it as “the concept of providing real-time access to critical business performance indicators.Chapter 3 Common Integration Scenarios Business activity monitoring (BAM) is an emerging technology for enabling business performance optimization.” Whereas BPM solutions provide visibility across individual business processes (or process-level management). BAM solutions provide business dashboards that correlate real-time business events with KPIs. Figure 3-9. giving managers the information necessary to proactively handle emerging problems rather than simply react to erupting crises. along with the supporting information to improve the speed and effectiveness of business operations. BAM provides a “big picture” of business performance and helps align business strategy with operational execution. Improving business performance with BAM technology 1 . In doing so.

While such an approach might yield short-term benefits for “low-hanging fruit. Chapter 4 provides more detail on the underlying technologies used to implement the scenarios described in this chapter. software license costs. maintenance.)  . Below are some best practices to increase the odds that an organization’s integration initiatives will succeed in the long run. Create an Integration Competency Center (ICC). an ICC will help it more efficiently build and maintain the complex integrations that its business demands. an enterprise architecture approach to integration will enable longterm business agility. Each of the integration scenarios presented in this chapter could be approached as a tactical implementation or as a necessary technical task within an overall strategic implementation. In contrast.Chapter 3 Common Integration Scenarios BAM is often associated with business intelligence (BI) technology.” in the long run it will increase redundancy. ensure that the technologies are integrated with one another. and management and maintenance costs. As an organization strives to develop core competencies for managing integration. with BAM providing the real-time operational view into business performance and BI the view for historical analytical purposes. skills training. and continual evolution of the organization’s integration platform. The ICC is responsible for the development and deployment standards that will help facilitate reuse and interoperability as well support the implementation. Plan strategically. It is important to understand that midsize and large organizations have a variety of business requirements and therefore require a variety of integration technologies. It will make different integration technologies available to different projects as needed. Some BAM vendors offer the ability to integrate with and feed into existing data warehouses. Consider them complementary sides of a coin. allowing organizations to use their BI tool of preference. Best Practices This chapter illustrates how integration can be applied to a broad range of business requirements. (See Chapter 6 for a description of the ICC’s function and roles. and decrease the time and cost to implement new solutions. implement tactically.

 . Business agility is a journey. not a destination.Chapter 3 Common Integration Scenarios Practice continuous improvement. Technology is just part of the solution. The good news is that integration technology can also help solve this problem through management and operational dashboards. Real-time visibility into processes and KPIs is only useful if someone is paying attention and practicing continuous improvement based on the enhanced information. and processes to optimize the business at all levels. Organizations need to evolve their infrastructures. line-of-business managers. business services. An organization’s success depends largely on what it does with the technology. and that a reward structure is in place to promote the desired outcomes. and business operatives are all aligned along the same business goals. Organizations need to build a culture of continuous improvement and business optimization. This requires that executives.

organizations need to understand how the range of integration solutions available today relate in the overall enterprise architecture. These enable data to be converted to and from the required target representations. Therefore. 4 Data transport and routing capabilities to move information reliably and securely between systems. Application connectivity may also be achieved at the database level using a database adapter or a direct SQL interface. which enable a standard application connectivity approach and eliminate the need to program against proprietary application program interfaces (APIs). 4 Data transformation and mapping capabilities. Application integration solutions typically use a message-based approach to moving data and support a variety of topologies that link systems together in a scalable and flexible manner.4 Guide to Integration Technologies Executive Overview A s mentioned in Chapter 3. organizations will inevitably acquire a collection of technologies to meet their diverse business needs. To maximize agility and reuse. Becoming familiar with the Roadmap is a good way to understand both the integration landscape and the types of business problems addressed by specific technologies. The ebizQ Integration Roadmap can help organizations match their business requirements with the appropriate integration technologies. These services include: 4 Connectivity to the applications. If Web services interfaces are available. application connectivity may be achieved without an adapter. Connectivity is generally achieved using adapters.  . different business scenarios warrant the use of different integration technologies and infrastructure services. Application Integration Services Application integration services offer the basic features and capabilities required to integrate and connect applications reliably and securely.

composite applications. interface integration. integration servers. consequently.Chapter 4 Guide to Integration Technologies 4 Infrastructure services to implement the application integration scenarios described in the previous chapter. Technologies that provide application integration services include message brokers. and business optimization with business activity monitoring (BAM). ebizQ Integration Roadmap The core application integration capabilities often play a role in other integration services. and exception management services. Figure 4-1. which limits agility and reuse. and enterprise service buses (ESBs). such as information integration. Without the underlying application integration services.  . connecting with mainframe systems and integrating the systems of different organizations in a business-to-business (B2B) scenario have special considerations and. these other integration services require point-to-point connectivity with back-end applications. These include support for transactions. In addition. application-level security. process integration. require additional technology solutions.

They can also be used alone for smaller point-to-point integration projects. handling B2B standards like RosettaNet. generally include full B2B capabilities  . also called integration suites. which include additional capabilities and services to address different integration scenarios and provide more direct business value. integration servers. An important function of the message broker is to provide a secure and reliable communications layer—for example. Message brokers usually have a hub-and-spoke architecture. with all processing done at the central broker. with multiple brokers connecting multiple integration servers. including messaging. Integration servers can be used in combination with message brokers to create a scalable integration backbone. enterprise application integration (EAI) vendors have evolved their message brokers. and publish/subscribe for broadcasting information and events to a number of applications. and adapter libraries into integration platforms. layering a Web services interface around an existing application interface). These platforms. InTegraTIOn servers As application integration requirements have become more complex. the basic message broker-centric architecture has evolved to provide more functionality at the integration end points. data translation and transformation. Message broker products support different styles of messaging—such as request/reply for sending a message to an application and receiving a reply. such as supporting more sophisticated application processing rules. message brokers provide the basic connectivity needed for application integration. These additional capabilities are typically combined into an integration server. InTegraTIOn plaTfOrms Since introducing their products in the mid-1990s. and intelligent message parsing and routing. and performing protocol translation (for example.Chapter 4 Guide to Integration Technologies message BrOkers The original application integration technology. to ensure the guaranteed delivery of a message from one application to another. More advanced vendor solutions offer distributed processing options and multiple hubs to improve scalability and availability.

esBs The enterprise service bus is an emerging middleware category that is typically viewed as a key enabling technology for Service Oriented Architecture (SOA). In reality. usually by making updates or changes to the application or its database. SOA relies on ESBs to help bind the services together. Once a message arrives at an application. ESBs can fulfill requirements such as intelligent routing. and data checks for programs that interface with the application. Conceptually. load balancing. ESBs serve as intermediaries between services providers and service consumers to avoid point-to-point connections. and other combinations of the integration technologies. In this role. transaction logic.Chapter 4 Guide to Integration Technologies and some level of business process management (BPM) and workflow capability. and transaction monitors. the lines have blurred as ESB vendors incorporate functionality similar to traditional integration platforms and suites. and security while also coordinating the flow of service calls. ESBs provide much the same functionality in an SOA as an integration platform does in an application integration scenario. Accessing the application via the API is thus preferred over integrating directly at the  . Many commercial applications have APIs. The options for accessing the application include writing a program that uses the API. mainframe integration capabilities. or using a Web services interface. The biggest difference is that ESBs are generally based on Web services standards. it must be processed. Just as previous generations of distributed systems depended on middleware such as RPC. which enforce the necessary business rules. CORBA. a portal. They may also include business activity monitoring features. adapTers In a typical integration scenario. using a prebuilt adapter. transformation. These additional capabilities are discussed later in this chapter. messages are routed between the various applications being integrated. Some vendors originally positioned ESBs as simpler alternatives to full-blown integration platforms.

There are two types of adapters. an adapter exposes a standard interface to the developer. on the other side. Some vendors reduce the development effort further by offering graphical tools for configuring adapters. thereby distributing the processing load and increasing the solution’s overall scalability. Adapters are a desirable alternative to API-level programming. thus requiring a message broker or integration server for translation and transformation.  . Adapters provide a common set of function calls for different back-end applications. they do not perform functions such as data translation and transformation. So vendors frequently provide a development kit or library for building custom.Chapter 4 Guide to Integration Technologies database level. However. which remain an important feature of the overall integration solution. so the developer needs to know how to code to each application’s unique APIs. They simplify the integration effort by eliminating the need for application-specific interface development. On the one side. one-off adapters that interface with the application while providing the niceties of a commercial adapter. Off-the-shelf adapters are not always available for every application that an organization might want to integrate. Intelligent adapters can perform data translation and transformation at the source or target. the adapter deals internally with the task of translating these interface calls to the specific APIs of a supported application. For example. The downside to APIs is that they differ for every application. This increases the time and cost for integrating systems and decreases agility. not the rest of the application integration services. such as graphical configuration or built-in logging and error handling. Standard (non-intelligent) adapters provide only basic connectivity services. The developer needs only learn how to code the standard adapter interface rather than each application’s interface. WeB servICes Web services are an attractive integration alternative because they provide a standard interface protocol that eliminates the need for adapters. Web services only replace adapters.

A major evolution of B2B integration is in the area of EDI. This includes the need to manage data confidentiality and integrity and the identity of the parties involved. For larger partners. Electronic Data Interchange encompasses a set of standard protocols for conducting structured interorganization exchanges. 4 Support for connectivity to proprietary communications networks and over the Internet using protocols such as FTP. The challenge of B2B commerce is to provide a range of access technologies to meet the needs of all partners and suppliers. as well as support for custom document and flat-file formats. 4 More stringent security than what is typical for internal application integration. 4 Management of configuration settings and service levels on a partner-bypartner basis. such as making purchases or initiating loan requests. including integrating with the back-end systems on both sides of the firewall.  . HTTP. Tools that enable organizations to connect faster. B2B integration solutions usually include a variety of connectivity options—sometimes called on-ramps—for different types of partners. such as integration and process templates. These requirements include: 4 Support for a variety of electronic commerce standards and formats. such as partner portals for inputting information or just sending XML messages. EDI transactions are typically conducted through a value-added network (VAN) provider. accelerate the time to value. it might make sense to fully automate the B2B transactions. and Web services. These VANs are essentially private networks for conducting B2B transactions.Chapter 4 Guide to Integration Technologies B2B InTegraTIOn Additional requirements exist when integrating with applications beyond the firewall. including Electronic Data Interchange (EDI) and newer XML-based standards such as RosettaNet. Smaller partners with minimal IT resources require simpler solutions.

Authentication services are required to confirm the user’s or organization’s identity. Some integration solutions on the market today combine EDI and XML standards support in the same framework and tool set. which are based on the transfer protocol used. wrappering legacy code as a service with a standardized interface. integrating at the transaction level through specialized gateways (for example. integrating with the legacy database. Options include screen scraping or report scraping.Chapter 4 Guide to Integration Technologies The explosive use of the Internet has led to the creation of a new standard—EDIINT— that enables EDI transactions to be integrated with Internet transactions using XML. EDIINT has several variations. B2B solutions also demand different—and in many cases greater—levels of security than does internal application integration. which requires some changes to the 0 . while EDIINT AS2 uses HTTP and HTTPS for securely exchanging EDI files over the Internet. Authorization services are required to specify a user group’s levels of access to various applications and services. Nonrepudiated communication sessions may suffice for simple data exchanges. Each partner needs only connect with the exchange rather than manage the details of connecting with every partner. for IBM CICS or IMS transactions).This reduces an organization’s burden when connecting to multiple partners with different technology infrastructures. creating a message-oriented interface to an application). allowing a single solution to be used for all B2B-related integration requirements. but nonrepudiated transactions are generally required when the solution includes more complex B2B transaction processing. And nonrepudiation services are required to ensure the validity and enforceability of electronic contracts. B2B integration exchanges and service providers have become viable alternatives for essentially outsourcing the B2B solution. maInframe InTegraTIOn Mainframe systems usually do not have exposed APIs and therefore present unique application integration challenges. The type of nonrepudiation needed depends on the type of application. such as a Web service. or front-ending (that is. There are several ways to integrate with these legacy systems. EDIINT ASI uses SMTP.

Every integration project requires an understanding of the underlying information in the systems. 1 . The value of the information in an enterprise system is in direct proportion to its quality and consistency with other systems. which is information about data represented independently from the underlying systems. organizations will continue to use traditional data integration approaches—such as data warehouses and extract. that CustName in one system is the same as Company in another system.Chapter 4 Guide to Integration Technologies application. Enterprise metadata repositories that provide real-time data services enable reuse and agility. it provides greater value. This also increases return on investment (ROI) because the effort and cost of discovering the semantic meaning of data within systems can be more readily reused across multiple projects. for example. The next leap forward in data management services and integration productivity will come from semantic integration. It is crucial to know. When data is managed as an important enterprise resource. and metadata repositories are the newer technologies that focus on information integration. Generally. this knowledge is put into proprietary mapping tools. and manage it at an enterprise level to promote data consistency. By capturing it as semantic metadata. Enterprise information integration (EII). and load (ETL) tools—to deliver the full set of information integration services. Specialized legacy integration solutions are often used in combination with other integration software to connect the mainframe systems with applications running on distributed platforms. organizations can maximize reuse and ultimately enable systems to automate the rules for translation and transformation. transform. enterprise content management (ECM). Metadata plays an important role in enabling interoperability among systems because it allows data to be understood and exchanged consistently across the systems. The key to managing distributed information effectively is metadata. Semantic integration captures the meaning and context of data within source systems in a metadata repository. Information Integration Services Information integration services consolidate and integrate structured and unstructured information from multiple sources. Additionally.

such as documents and images. extracting the information from systems on a nightly or weekly basis. transfer. EII solutions can provide the equivalent of a virtual data warehouse or midtier data store for real-time reporting or fast implementation of a portal solution. ECM services are required when the business process is primarily focused on documents being passed around the organization. ETL technology is still used today for applications where it is preferable to physically move the information to create a unified view and where real-time updates are not necessary. EII solutions access the data from the source systems themselves. Graphical views make it easier for programmers and business managers to access the information they need without knowing the underlying data structures of the source systems. cleansing it. Emerging EII solutions are also incorporating semantic integration capabilities. transforming it into the desired target format. eCm Enterprise content management provides integrated access to and management of unstructured data. eII An emerging integration market sector. and load technology was developed for data warehousing implementations. However. enterprise information integration provides an aggregated view of data. and then loading it into the data warehouse or data mart. and caching that enable real-time information access and aggregation. ETL tools operated on large data sets in batch mode.  . ECM solutions can provide workflow and process management as well as integration capabilities.Chapter 4 Guide to Integration Technologies eTl Extract. EII tools have technologies for distributed query optimization. such as for historical analytics. indexing. Whereas data warehousing requires that the information be moved to a central database. the newer information integration technologies discussed below offer the option of leaving the data in the source systems and retrieving it in real-time as needed. or when unstructured content needs to be part of an overall business process integration solution.

There are also management portals for tracking business processes or key performance indicators (KPIs). they aggregate information and functionality from multiple systems. for example. Although portal technology provides a way to develop an integrated user interface. enabling customers to place and track orders. and operational management portals for monitoring system-level performance.  . such as local weather or a customized stock ticker. Portals may also include personalization features that allow users to customize the look and feel of the portal application—for example. This can be done point-to-point or using the application integration technologies discussed above. That is. or allowing employees to enter expenses or manage their 401k’s.Chapter 4 Guide to Integration Technologies Interface Integration Services Interface integration services provide access from a variety of front-end devices to a variety of back-end systems. The technologies that provide interface integration services include portals (for delivering the user interface via a browser) and mobile integration gateways (for enabling the use of mobile devices). Portal technology often includes an environment for developing the portal application. For example. there may be a log-in portlet that manages log-ins to multiple back-end services. Transactional portals provide greater interoperability. Many EAI solutions focus on automating transactions across systems. which provide information on corporate policies or news—are informational only. but the information flows only one way: to the portal. But a portal solution integrates information into a browser for a particular purpose or set of users. This application can be composed of multiple portlets. Various types of portals exist. which are pluggable user interface components. choosing what types of information to display. providing a single touch point for the delivery of application services. pOrTals Portals provide an integrated user interface to multiple back-end systems. back-end integration is still required. Some—such as enterprise information portals.

and only once. It also addresses how security and personalization are handled. who need to work offline and then synchronize with source systems and reconcile the offline transactions across the systems. To support occasional users. and determines when messages are delivered based on the type of network connection. Today. Mobile applications are likely to require backend integration to present information from multiple sources. IT departments are coming under increasing pressure to provide enterprise application services to a variety of mobile devices. In addition. And JSR 168 is a Java API for portlets that enables content sources and application front ends to be aggregated. it controls message delivery by organizing messages into transactional groups. Mobile integration solutions usually include a specialized server that provides the transformation services for each target device. surf the Web. the ability to encrypt data is important.  . PDAs. resolutions. and interface requirements. even in the event of a system failure. To address the insecurity of mobile communications. mOBIle InTegraTIOn gaTeWays Mobile devices—including cell phones. because mobile devices operate on inherently unreliable and insecure networks.Chapter 4 Guide to Integration Technologies Several important standards address portals. management and security need to be part of the integration solution. This server integrates with back-end systems either directly (point-to-point) or through an integration platform. Web Services for Remote Portlets (WSRP) defines a set of standard interfaces for creating portlets that can be accessed remotely using Web services protocols. and conduct other aspects of business. The mobile integration server typically uses message queues and logs to guarantee that messages will be delivered once. This allows applications to consume portlet components without having to write unique code for interacting with each component. And mobile applications often must accommodate occasionally connected users. people commonly use their cell phones to receive email. These mobile devices are proliferating rapidly. along with security. and specialized mobile units such as package tracking systems—all have different screen sizes.

As a result. service orchestration capabilities. Composite applications enable new business solutions to be created by flexibly combining new and existing application functionality. Composite application services are the technical infrastructure services that support the development.Chapter 4 Guide to Integration Technologies Composite Application Services The term “composite application” refers to an application assembled from components that may originate from various underlying systems. The benefits of a composite application approach are maximized when organizations create reusable business services—which then serve as building blocks for a composite application—as part of a Service Oriented Architecture. This allows organizations to deliver systems faster than building them from scratch.  . and management of composite applications. and service registries for governance. reusable high-level business service 4 Design and model the behavior of the composite application. Software vendors are introducing composite application platforms to offer a collection of these technologies and services in an integrated suite. composite applications are typically associated with SOA and Web services. including how information and messages flow across services. deployment. and how services need to be orchestrated in support of a business process 4 Discover services or browse from a registry of available services 4 Manage policies and service level agreements to engender trust and enable the reuse of services 4 Deploy the resulting solution into a managed run-time environment The technologies most commonly associated with composite applications include Web services. These include the ability to: 4 Develop new services (typically based on Web services standards) 4 Create service abstractions by combining several low-level services into a more useful.

and Integration. and standardized message encoding using XML. To reap the benefits of reusability. commercial registry products typically incorporate some level of Web services management capability to make the registry more useful within the enterprise. Doing this in programming code reduces the flexibility to change the application.  . As a result. a standardized communication protocol (Simple Object Access Protocol. making Web services well suited to the role of providing aggregated functionality within a composite application and being the standard for interapplication interfaces. is gaining mind share and market share as the orchestration standard. organizations must manage these services effectively throughout their life cycle. Organizations that embrace SOA will experience a proliferation of services—and versions of services. Web services define a standardized interface (Web Services Description Language. A number of software vendors now support BPEL. but registries are also gaining ground within the enterprise.Chapter 4 Guide to Integration Technologies WeB servICes Web services and their related standards have emerged as the de facto building blocks for composite applications. many organizations today base their application development and integration efforts on Web services. making it possible to share process definitions more easily among tools. a vendor-led initiative. or WSDL). a standardized repository for registering and discovering Web services (Universal Description. OrChesTraTIOn To create a composite application from a set of Web services. These standards enable a Web service to reside anywhere and be accessed from everywhere. This latter approach basically allows developers to modify the way services are linked together without changing the services themselves. Business Process Execution Language (BPEL). UDDI was originally created to manage the cataloging of public Web services. known as UDDI). Beyond basic UDDI support. so an externalized approach to service orchestration is preferable. it is necessary to define the flow of control across services. servICe regIsTrIes fOr gOvernanCe UDDI is the registry standard for dynamically discovering and invoking Web services. Discovery. or SOAP).

Nevertheless. while the application server vendors focus on the development aspects. it will often be necessary to develop new services and to integrate composite applications with other enterprise systems. COmpOsITe applICaTIOn plaTfOrms The development paradigm of composite applications involves assembling existing components. That means policies can be set at an enterprise level. and increase business agility. process orchestration. As composite applications grow in number. Naturally. the composite application platform may be based on an application server that also includes integration and orchestration capabilities.Chapter 4 Guide to Integration Technologies In addition. This approach allows the policies to be enforced across all applications that utilize the service. Because it is an emerging technology segment. integration capabilities. or services. registries and service governance will become critical aspects of the integration infrastructure. with process management capabilities and a registry for managing services. Depending on the vendor. Process Integration Services Process integration services enable organizations to automate and optimize business processes that cross application and organizational boundaries. into a new system. some vendors use the registry as the basis for policy and governance management in their solutions so that policies—such as service level agreements—can be enforced at run-time. Therefore. and user interface functionality. the integration vendors take an integration-centric view. rather than having each application implement the policy independently. This helps organizations reduce operating costs.  . These characteristics include custom development capabilities. or it may have an integration platform as its foundation. However. a common set of composite application platform characteristics is emerging. the offerings are extremely varied. both the integration vendors and the application server vendors are beginning to position their platforms as—and evolve their offerings into—composite application platforms. gain real-time visibility into end-to-end business processes.

Using process integration technology. These capabilities include modeling business processes. Some also offer process simulation and analytics to provide feedback for optimizing processes. manual. and organizations. process integration tools need to foster collaboration on a process model and an iterative approach to its design and implementation. A process-driven approach to integration also improves the alignment between IT and the organization. and groupware or collaboration platforms. automating the processes  . This significantly reduces the chance of misinterpreting the business requirement or getting the process wrong. business units. each type of process has different requirements and therefore calls for different process technologies. The technologies that deliver process-level integration include business process management for handling automated (and sometimes manual) processes. the optimization of business processes can deliver competitive advantage and significant ROI. It starts with the development of business process models that businesspeople can understand and review. Since processes can span departments. or collaborative. Frequently. Dr. in the 1950s. an end-to-end business process includes both automated and manual processes. W. Some vendors offer solutions that combine these capabilities into a single product offering. workflow management for manual processes. Edward Deming proved that improvements in managing business processes lead to improvements in overall business and a significant competitive advantage.Chapter 4 Guide to Integration Technologies While underlying integration technologies and composite application development can help make IT more productive and agile. Therefore. An organization’s business processes may be automated. the models can then be automated. These models depict a shared understanding of the end-to-end business process. From an integration perspective. Bpm The business process management solutions on the market today represent a range of process integration capabilities to support process automation and optimization. no single individual may own or even understand the end-to-end process. As mentioned earlier.

Most have their own modeling notation with BPEL support for process execution. However. As mentioned. which is gaining mind share as an underlying execution language. a number of BPM solutions also include simulation capabilities to facilitate process analysis and optimization. For example.  . WOrkflOW managemenT Workflow management solutions focus on the efficient coordination of manual tasks. Unlike BPEL. That way. no widely supported standards currently exist for process modeling. few vendors currently support BPMN. and optimization. BPM products on the market today generally provide additional capabilities beyond simple process orchestration in the areas of process monitoring. during the busy season. Available solutions have varying levels of support for managing manual processes as part of the overall process and for using simulation capabilities to optimize processes. management. Process designers can perform what-if analysis to test and compare variations of the process for better productivity and efficiency. Another evolving capability is the ability to externalize business rules rather than having them hard-coded into the process integration software. BPM solutions can also be used to create composite applications in the form of business processes that span multiple systems. This makes continuous process improvement possible and is a key to gaining competitive advantage.Chapter 4 Guide to Integration Technologies directly from the model. Some tools also allow performance statistics from running processes to be fed back into the simulation engine to analyze and optimize processes dynamically. businesspeople can directly control their portion of the process and thereby increase business agility. and monitoring running processes. is working its way through the standards process within the Object Management Group (OMG). a manager could raise the dollar amount that would trigger an approval for an order. These solutions have been available for many years and predate most BPM solutions. The BPM tool can model and implement the flow of control across services and applications. Business Process Modeling Notation (BPMN). One candidate.

Robust workflow management involves process modeling as well as the design and development of user interfaces to manage task lists. illness. the interactions in a collaboration scenario are generally free-form and ad hoc. because end-to-end business processes frequently include automated as well as manual processes. In addition. In contrast with workflow. different interfaces should be available for managers and the employees receiving the tasks. balance workloads. This is especially true as project teams increasingly become physically distributed. such as documents and design specifications. and features for managing email threads and discussion groups. accept or change assignments. where the interactions follow established processes with predefined paths and steps. Collaboration is also relevant to B2B. Business Optimization Services Business optimization services enable organizations to align business strategies with coordinated actions. but optimizing indi- 0 . and account for changes in resource availability due to vacations. many integration vendors now offer BPM and workflow management in a single platform. Manual workflows often involve approvals. etc. it is relevant to organizations seeking to implement integration solutions as part of a strategic initiative to optimize business processes. collaboration software facilitates interactions among people.Chapter 4 Guide to Integration Technologies However. and other joint initiatives. where more specialized solutions enable partners to work together on product design or development. version control and security capabilities. Collaboration platforms include repositories for managing team artifacts. Vendors that focus exclusively on workflow management are rapidly decreasing in number. COllaBOraTIOn plaTfOrms Also known as groupware. Although collaboration is not usually considered a mainstream component of business integration. BPM solutions vary in the level of support they provide for manual processes. The goal of BPM is to streamline processes. which need to be logged and audited and require the workflow solution to understand the organizational hierarchy and approval chain.

The most significant of these are dashboards for executives. As a discipline. The key feature requirements for business performance optimization include dashboards. BAM solutions are emerging from BPM. This capability is provided by an emerging technology called business activity monitoring. and advanced correlation and predictive monitoring to help managers identify developing concerns and react before they become full-blown crises. unexpected drop in order volumes—provides real-time analytical capabilities and management dashboards.Chapter 4 Guide to Integration Technologies vidual processes does not necessarily lead to optimization at an organizational level. and systems management vendors. along with the supporting information to improve the speed and effectiveness of business operations. The goal of business optimization is to optimize at the organizational level and to align all parts of the organization to enhance business performance. BAM is the technology that monitors the events. business intelligence.” Advances in integration technology have allowed organizations to instrument applications to deliver real-time information associated with significant business events. and alerts users when exceptions occur. database. and IT operations staff that provide real-time visibility into the organization’s operations and associated KPIs. automated and dynamic performance thresholds. and Lean—best practices. Balanced Scorecard. The well-known analyst firm Gartner coined the term business activity monitoring in 2002 and defines BAM as “the concept of providing real-time access to critical business performance indicators. integration. Several developments have emerged on the technology front to support business performance management. business performance management includes methodology—such as Six Sigma. such as the receipt of an order. 1 . Some BAM solutions include capabilities to predict when exceptions are imminent by comparing current operating metrics with historical norms. Business optimization services include technologies that enable business performance management. and one that is not yet fully defined. correlates them to identify trends—such as a sudden. The market for BAM is a quickly evolving sector. statistical analysis of user-defined KPIs. line-of-business managers. and technology infrastructure.

They are the technologies to use when creating business solutions. Organizations seeking to implement new business solutions that require integration will focus on vendors in this category. Compliance is an example of a business requirement that involves integration across a diverse set of areas and has a strong monitoring and auditing requirement. and other entities. Compliance and Industry Solutions The previous categories all represent integration infrastructure services. these are inherently integrated applications. manufacturing. Basel II. and Health Insurance Portability and Accountability Act (HIPAA) solutions that make it faster and easier for organizations to implement solutions to comply with government regulations. For example.Chapter 4 Guide to Integration Technologies Ultimately. organizational groups. In other words.  . and retail—as well as solutions for supply chain management and customer relationship management. Compliance and industry solutions require an integrated infrastructure—including process monitoring. finance. an emerging category of solutions from software management vendors is business transaction management. The Compliance and Industry Solutions section of the Roadmap encompasses business applications that utilize the integration infrastructure technologies to provide specific business applications. tracking. The difference between these solutions and traditional business applications lies in their process-centric nature and in the fact that the processes typically span multiple underlying systems. business performance optimization must encompass all levels of the organization. This technology enables systems administrators to view the status of end-to-end transactions as they cross systems on a business process level so they can better understand the context of the transaction. integration vendors and consultants are offering solutions for specific industries—including healthcare. and auditing—because none of the individual participating systems has the total end-to-end view. Similarly. Many integration vendors and systems integrators now offer Sarbanes-Oxley. both on the business side and within IT.

and determining problems. the ad hoc creation of Web services across the enterprise can quickly result in chaos. Where standards are lacking. the need for Web services management has become increasingly important. while others offer platforms that incorporate many aspects of the Roadmap in their solutions. Web services management frameworks help users deal with issues such as registering Web services across the enterprise. Solutions that include digital certificates and encryption are especially important for B2B security. as well as related areas such as governance and security. for example. taking a strategic approach to architecture and development is a key element for success in implementing the technologies represented in the Roadmap. With the growing adoption of SOA. Architecture and Development The ebizQ Integration Roadmap helps explain the types of integration technologies available. Some vendors offer only one type of technology. Without management. Security is another extremely important aspect of integration. Organizations that take a tactical approach to developing integration solutions risk suffering the same problems experienced with developing distributed applications: multiple redundant technologies used for different implementations. Whether an organization is partnering with one vendor or several. who is using what service. and which services are secure. Emerging management offerings include third-party solutions for application integration platform management. monitoring those Web services in operation. Organizations that take a strategic approach to developing  . having a complete understanding of what is available. and integrated security solutions are beginning to emerge. Web services management. resulting in increased maintenance costs and a more fragile architecture.Chapter 4 Guide to Integration Technologies Management and Security Every level of the integration infrastructure requires management and security. vendor solutions are filling in the holes. with no one. especially in the area of Web services. and end-to-end transaction management at a business process level.

and increases business agility. leverages IT investments. many organizations are establishing an Integration Competency Center (ICC). If the business services conform to standard interfaces. For example. This prevents the purchase of redundant technologies for different projects.Chapter 4 Guide to Integration Technologies the architecture start with an overall plan and then implement it on a project-by-project basis. However. A key element of this is understanding the role of different integration technologies and their context within the overall integration infrastructure. reduces training and maintenance costs. Organizations that wish to take a strategic approach to developing an integrated infrastructure should establish a centralized group. Organizations that take a strategic approach to building an integration architecture will be better able to avoid integration mistakes. Some organizations may place this group within their Enterprise Architecture Group. A well-defined and integrated technical architecture also increases flexibility in the development phase. Chapter 6 focuses on best practices for creating an ICC. avoids overlaps in integration technologies. lower the cost of ownership. they can be written in a variety of languages. Conclusion Integration is a journey consisting of many tactical implementations. control redundancy.  . they can simply use what is already in place and focus instead on adding business functionality and value. enabling organizations to leverage skill sets within the organization. services written in Java can be part of the same composite application as services written in . increase ROI. The ebizQ Integration Roadmap can guide you in this process. since integration requires specialized skills. and ultimately achieve greater business agility. if they have one. Rather than requiring individual project teams to research and choose their own integration technologies.NET.

An even broader interpretation is that SOA marks the end of monolithic enterprise applications and the beginning of more flexible and adaptable business process-centric applications. the Internet has extended system interfaces to include connections with trading partners. SOA is often associated with Web services. From a software engineering perspective. they do not address the grander architectural and design philosophies of SOA. The past decade has seen unprecedented leaps in computing. there is unanimous agreement on SOA’s value and promise. and electronic connections. In other words.  . Web services address only the low-level interactions between services. The challenge for IT today is to create and support an infrastructure that can harness these diverse assets and deliver on new business requirements more effectively. so it is important to understand the relationship between integration and SOA. however. From a broader IT perspective.5 Integration and SOA Executive Overview S ervice Oriented Architecture (SOA) is transforming IT. SOA also makes it possible for organizations to leverage IT assets so they can rapidly deploy new business solutions as needed. methodology. While definitions of SOA vary. because definitions and interpretations of SOA abound. However. At this level. SOA provides the basis for this infrastructure and offers the promise of tangible business benefits. databases. SOA is simply an approach to software design that involves assembling systems from reusable components (services) that may originate from different sources and underlying technology environments. SOA is about much more than just Web services. Not surprisingly. SOA suggests a fundamental shift in how an organization implements business systems—with attendant changes in technology. More and more aspects of a business use computers. This is not a straightforward task. and organizational structure. this has resulted in an unruly explosion of stovepipe applications. Additionally.

service management. SOA will benefit how organizations approach integration. with SOA making it easier to integrate systems and business processes. This strategy delivers benefits along the way. design concepts. integration and SOA are highly complementary. and best practices of SOA. and the ability to make legacy systems part of the SOA environment.Chapter 5 Integration and SOA With its promise of transforming the way systems are developed. and integration technologies providing the essential technical infrastructure services required for SOA implementations. At a basic level. Unlike previous IT revolutions that were accompanied by massive change. service.  . At the same time. The key to success is to plan strategically at an enterprise level and then implement tactically. the growing adoption of Web services will improve system-to-system interoperability—making it simpler to connect systems—while the componentization of business services will make it easier to create new business solutions from existing applications and retrieve information as the business demands.and process-level monitoring. making it easier to justify SOA investments. Clearly. addressing needs such as service orchestration. This chapter explores the benefits. with the appropriate best practices and governance policies and procedures in place. integration technologies play an important role in the SOA portfolio. SOA enables an evolutionary approach and the opportunity for organizations to move forward incrementally. technologies. and shows how they relate to integration.

000 people in more than 200 locations around the world.000 times. Headquartered in New York. How Integration Addressed the Challenges WRA adopted an SOA strategy to address its requirements. and the webMethods platform performs all necessary mappings and transformations. ordering. Using webMethods as its SOA platform. WRA has gained a competitive edge in the market. and the RoHS Web service is utilized more than 80. and the global components data was located in an Oracle database. The new system provides global enterprise visibility into the company’s products and parts along with RoHS information. WRA needed to make all relevant information about its products accessible to its customers for quoting. WRA was able to link its disparate systems using Web services and to provide a service-oriented interface for consumer applications. WRA employs more than 10. WRA is required to comply with RoHS. In addition to complying with the directive. and purchasing decisions. Bottom-Line Business Benefits WRA is now able to comply with the RoHS mandate. more than 500. Company sales personnel have also been able to help customers make more informed purchasing decisions. quoting and ordering) employed different subsystems running on different platforms and diverse technologies. The company serves as a supply channel partner for more than 500 suppliers and 150. increasing customer satisfaction. contract manufacturers.g. Business Challenges As a player in the global electronic equipment market.. different business processes (e. However.Chapter 5 Integration and SOA Company Description Case Study 5-1 Company: WRA Electronics WRA Electronics is a pseudonym for a Fortune 500 global provider of electronic components and computer products.000 original equipment manufacturers. saving time and errors. Any differences between the platforms are bridged using the Web service interface. By providing its customers with quick and easy access to RoHS-related information. Each month.  . The webMethods solution also provides connectivity to WRA’s mainframe-based systems that currently cannot interoperate with Web services. a European industry directive restricting the use of six hazardous substances in electrical and electronic equipment. The system has enabled the automation of numerous process steps. and commercial customers. WRA has achieved significant cost savings and efficiency improvements with its SOA approach. As part of the directive.000 transactions pass through the company’s internal Web services integration environment. The RoHS Web service provides data from the Oracle database and plugs into the existing business processes for quoting and ordering.

Chapter 5 Integration and SOA

Why SOA?
SOA provides many benefits to both IT and the orga-nization at large. The most important benefit—and the one most often cited in relation to SOA—is reusability. SOA allows organizations to do more with the resources at their disposal. It promotes reusability by enabling services to be shared by different applications running on different platforms. Reusability creates a competitive advantage by providing greater flexibility in the way computer systems can be used to support the business.

Figure 5-1. Flexibility of SOA through reuse

For example, many organizations have stovepipe applications that address specific areas of business functionality, such as order processing, invoice handling, or customer relationship management (CRM). Each application has its own customer data and its own view of the customer. And each has local mechanisms for retrieving information, such as customer details, that were developed in the environment and technology specific to the application platform. In an SOA, these duplicated routines can be re- 

Chapter 5 Integration and SOA

placed by a common reusable service (for example, FindCustomer) that delivers the same functionality. Because there is just one copy of the FindCustomer service, changes need only be made in one place. This reduces application maintenance costs and enables new changes to be accommodated more quickly and easily. Another benefit of SOA is increased agility when addressing business opportunities or changes in business requirements. By reusing common steps and developing only the new steps that are required, organizations can more easily modify or enhance business processes and applications when the need arises. For example, a company that offers home loans may have a loan approval process consisting of numerous SOA services. If the company now wants to offer car loans, it can reuse many components from the existing implementation, such as checking current account details or market interest rates. This increased agility translates directly into increased competitiveness. Thus, organizations that master SOA will gain an advantage over those that do not. A third benefit of SOA is the reduced complexity from a more orderly enterprise architecture and development approach. Modular services enable a less-complex, divideand-conquer approach to software development. This enables SOA to accelerate the deployment of new application functionality, decreases the testing effort by making it easier to test individual components independently, and lowers implementation risks through the reuse of functionality that is already quality-tested and proven. Another benefit of SOA is increased IT adaptability. Packaged application implementations, custom-built solutions, and system changes resulting from mergers and acquisitions can be integrated more quickly and easily. SOA’s inherent platform-independent approach provides organizations with more flexibility to use the software and hardware of their choice. This allows them to engage in a multisource strategy and reduces the threat of vendor lock-in. For solutions developed in-house, it enables programmers to use their existing technical skill sets. 

Chapter 5 Integration and SOA

Figure 5-2. Adding new functionality easily

Best of all, these benefits can be realized quickly. SOA is ideally suited for incremental deployment, with investments made step-by-step for individual projects. Returns can also be realized incrementally. This eliminates the need for costly and risky “big-bang” software implementations, reduces investment risk, and delivers value more quickly. Thus, SOA becomes a far more palatable investment for organizational decision makers.

SOA Explained
Although the term Service Oriented Architecture first emerged in the 1990s, the spirit of SOA existed long before that. The underlying principles of SOA—componentization, encapsulation, separation of interface from implementation, loose coupling, and so on—have been promoted since computing’s infancy. These design principles were used in the days of centralized computing, when the focus was on IT resource optimization. 

0

IT components are formed into services that are relevant to a particular business operation. platforms. The advent of object. On the technology side. It is the logic that resides on computers somewhere on the network and executes 1 . the arrival of distributed computing offered great productivity and price performance. It is basically an electronic description of how to call the service from other programs. An explanation of the term “business services” is helpful here. among other things. the service for FindCustomer might involve code that locates the customer number by searching for the customer’s name in one system.Chapter 5 Integration and SOA Then the landscape changed. then more code that uses the customer number to query the customer’s service subscriptions in another application. The service implementation is the actual code that fulfills the functionality of the service. This is where Web services provide an ideal fit for SOA.oriented computing paved the way for SOA by introducing the concept of having discrete program components with standard inputs and outputs that could be used to assemble applications. or locations. IT was increasingly seen as something that could be used for business advantage rather than just back-office operations. For example. the location of the service and the information required to make a request to the service and get a response. But these changes introduced considerable challenges in linking together different technologies and platforms to utilize the increasingly diverse IT infrastructure to meet new business needs. On the business side. and the emergence of the Internet promised ubiquitous connectivity. SOA is an IT architecture based on the delivery of reusable. A service consists of a well-defined service interface and the service implementation. So what exactly is Service Oriented Architecture? From a software engineering standpoint. The use of Web services ensures a standard protocol for communication between the service consumer and provider and a standard way to define a service’s interface. It specifies. well-defined business services underpinned by IT components in such a way that the providers and the consumers of the business services are loosely coupled and have no knowledge of one another’s technologies. and yet more code to retrieve the organization’s interaction history with the customer from a third database. The service interface describes how to call the service. With SOA.

SOA allows application requirements to be expressed in terms of modular. and any inputs and outputs required. All the calling application needs to know is how to invoke the service. while a course-grained service (such as ProcessInvoice) incorporates more functionality and typically maps more closely to the notion of a business service. Loose coupling also means that one service can be replaced by another—provided it has the same signature (inputs and outputs)—without impacting any of the other connected services or requiring them to be modified. The dynamic nature of SOA is another important distinction from more traditional application development approaches. The only thing that matters is that the service fulfills the behavior specified by its interface. which makes it possible to assemble course-grained services that map to useful business functions. subject to appropriate security constraints. At a technical level. However. This enables the interoperation of different applications and services written in different languages with different development tools and platforms. it does not matter. A fine-grained service (such as VerifyZipCode) performs a very specific and discrete function. Another important term used when defining SOA is “loose coupling. SOA is not concerned with how services are implemented. That means incremental changes to a system can be made without having to redevelop the entire application. This distinction sets SOA apart from previous computing paradigms by providing a common language for IT and the line of business to discuss computing requirements.” Granularity refers to the level of functionality provided by a service. SOA enables services to be made up of other services. C#. An important goal when designing services is to encapsulate business functionality at an appropriate level of granularity so it can be reused across different solutions. Architecturally. reusable business services.  . Rather than talking about low-level bits of programming code. for example.Chapter 5 Integration and SOA when the service is called. whether the FindCustomer service is written in Java. or COBOL. The service implementation is inherently platform-dependent.” Services that are loosely coupled can be invoked by any application on any platform in any location. A final consideration when thinking about SOA is “granularity.

end-user organizations. As a result. Discovery. Propelled by the adoption of the Internet and open standards. The Web services standards define XML-based protocols and interface descriptions that enable services to interact.Chapter 5 Integration and SOA The Role of Web Services As mentioned. and it has the flexibility to handle arbitrary types of data and data structures. and simply implementing Web services does not result in an SOA. normally using HyperText Transfer Protocol (HTTP). Basic Web services standards include Simple Object Access Protocol (SOAP). It has several advantages over previous approaches to describing data: It is readable by humans and machines. or Extensible Markup Language. Web services remove the risk of having to bet on a particular technology (as was the case with CORBA in the days of distributed computing). which are generally filtered out by firewalls.  . XML provides a text-based way to describe and apply a tree-based structure to information. distributed environment. it is platform-independent. Web Services Description Language (WSDL). and standards organizations. SOAP is a lightweight protocol intended for exchanging XML-based messages in a decentralized. SOA is often discussed in conjunction with Web services. is a World Wide Web Consortium (W3C)-recommended standard for defining different types of data. but the two are not synonymous. However. In theory. and Integration (UDDI). Web services and their related standards are driving the widespread adoption of SOA. XML. HTTP was chosen as the primary application-layer protocol for SOAP because it works well with today’s Internet infrastructure and is firewall friendly—unlike other distributed computing protocols. By standardizing essential elements of SOA. XML has emerged as the critical standard for data definition and document markup. Web services are the first standard for service-oriented computing to gain the near-universal support of software vendors. and Universal Description. SOA does not require Web services. the protocol used between Web servers and browsers on the Web.

reliable message delivery. A client program reads a Web service’s WSDL to determine what functions are available on the server. UDDI is an XML-based standard for a Web services registry (i. WSDL is an XML-based format for describing Web services. or supersets of the standards for enhanced functionality. As such. where one program (the client) sends a request message to another program (the server). and the server immediately sends a response message to the client. WSDL is typically used in combination with SOAP. The most common pattern is the remote procedure call (RPC) or request/reply pattern. Originally intended as a public resource.e. and then uses SOAP to actually call one of the functions listed in the WSDL. Vendors may implement subsets of the standards for expediency. WSDL defines the public interface to a Web service. Applications can use SOAP to create more complex interaction patterns. UDDI is often described as a Yellow Pages for Web services. back-and-forth (conversational) exchanges. BPEL. which describes how to communicate with the Web service as well as the operations and messages supported by the Web service.  . directory) that allows service providers to publish information about their services and enables potential service consumers to browse listings of available services (using SOAP messages) to identify the ones most suited to their application requirements. Standards are also being created to define the business process flow across services (for example. UDDI is now more commonly used within organizations as a private registry of their internal Web services. Unfortunately. and transactions. standards alone do not guarantee interoperability. as history has taught us. including request/response and multiple. Additional Web services standards are being developed in areas such as security.Chapter 5 Integration and SOA A SOAP message consists of a one-way transmission from a SOAP sender to a SOAP receiver. which was discussed in Chapter 4).. but these subsets and supersets may actually limit interoperability.

WS-I profile conformance is an important consideration for organizations that want to ensure interoperability of their Web services across different vendor platforms. Attachments Profile. which are implementation guidelines for interoperability. the vendor community has formed the independent Web Services Interoperability Organization (WS-I). Currently available profiles include the Basic Profile.  . and Simple SOAP Binding Profile. How Web services standards interact To guard against interoperability issues and the potential slowdown in Web services adoption that might result. In addition. WS-I creates profiles. work is under way on a Basic Security Profile.Chapter 5 Integration and SOA Figure 5-3. and testing tools to verify conformance with a profile and WS-I guidelines. WS-I’s charter is to promote the interoperability of Web services implementations from different vendors.

Power & Light sends its RTO hourly schedules of where it will be delivering power and where it will be pulling power off.S. it can now meet the Sarbanes-Oxley requirements. RTOs are independent. The company manages a distribution grid that spans 11 states and provides electricity to 5 million customers. U.S. How Integration Addressed the Challenges More than 100 new interfaces were required for the types of data and forms that U. Power & Light is a pseudonym for one of the country’s largest power providers. Using Web services. number of units online. government-regulated organizations focused on energy transmission and distribution for regions of the country. U. U.S. Power & Light needed to meet the RTO’s stringent technical requirements in 10 months. Power & Light become a member of a regional transmission organization (RTO). The company chose webMethods as the anchor of an SOA strategy that enables it to publish data in one place to support the RTO requirement. and outages directly from the company’s power plants.S. all subscribing systems can now instantly retrieve the information they require. Bottom-Line Business Benefits The SOA-based webMethods platform has enabled U. Power & Light met its 10-month schedule and avoided $1 million per day in potential fines. Power & Light needed to improve its operational efficiency to meet SarbanesOxley requirements. Business Challenges Several years ago. the Federal Energy Regulatory Commission mandated that U.S. With all the integration points in its systems connected through a common platform. In addition. which are translated from the source system into U. the company can easily connect disparate systems and reuse code when automating new processes. U. Power & Light to be development platform-agnostic for application interfaces. With the webMethods platform.S. Redundancies in the existing system required employees to enter the same information into 15 different systems for some transactions. Power & Light’s internal canonical forms. At the same time. and other key areas as required for compliance. These requirements included providing real-time and batch information on capacity. errors.S.Chapter 5 Integration and SOA Company Description Case Study 5-2 Company: U. The solution is based on XML schemas. emissions. SOA and Web services were two key ingredients for this success.S. This enables the data to be integrated into multiple systems without point-to-point data transformations. Power & Light would be exchanging with the RTO. Power & Light can easily monitor and document the system and processes.S. U. With the new system. Power & Light U. power produced per hour.S.  . or it risked federal fines of $1 million a day.S.

however. as stated earlier. the non-XML. products. Different data representations—in terms of syntax (the format of the data) and semantics (the meaning of the data)—will exist as long as more than one system is involved. the data format for Web services. Consequently. For example. Web services on their own do not provide the full spectrum of functionality required in an enterprise SOA. if ever. which of course is at the core of integration. before all enterprise data is accessible in XML format.Chapter 5 Integration and SOA Integration’s Role in SOA Although SOA is commonly associated with the creation of new systems. XML-based translation and transformation technologies such as XSLT do not suffice. organizations can increase flexibility and reuse by taking an SOA approach to implementing integration projects. Still. and coordinate Web services into more solutions in real-world business scenarios. and integration technologies enable SOA to achieve greater productivity and utility. Additionally. Different applications—built with different business requirements for different purposes—define business entities (such as customers. or financial securities) their own way. it will be a very long time. So how does integration serve an essential role in SOA? For one thing. Since so much data is represented in non-XML formats. A key driver of SOA is interoperability. and will remain an essential component of SOA. SOA and integration are also closely linked. additional capabilities are required to assemble. Beyond the interoperability enabled by Web services. non-Web-services-based systems that will continue to  . SOA will ease the challenge of integration. In driving a greater degree of system-to-system interoperability through the adoption of Web services standards. forward-migration mechanisms are needed to bring the legacy environment—that is. Therefore. it will not replace the need for higher-level integration capabilities. orchestrate. In essence. integration technologies— with their robust support for complex legacy data formats—are needed to provide SOA with a wider range of translation and transformation capabilities. SOA makes integration easier. translation and transformation of data into the native formats of each application is an essential component of integration today. For reasons explained below.

which crosses application and organizational boundaries. Finally. SOA simplifies integration by driving interoperability between systems via Web services. human workflow management. On the one hand. and process simulation. integration technologies and solutions provide a migration path to SOA from the existing installed base of pre-SOA systems. and legacy applications. however. The relationship between SOA and integration can be summed up as follows: 4 SOA and integration are highly complementary. since starting with a clean slate is not a viable option. Using application adapters for tactical solutions while taking an evolutionary approach to creating Web services from existing applications will allow an organization to realize an early return on investment (ROI) from SOA implementations. For some implementations. getting a new solution up and running quickly is the prime requirement. but BPM goes further with real-time monitoring management. On the other hand. it may be faster and easier to use existing application adapters provided by an integration vendor than to recreate access via Web services interfaces.  .Chapter 5 Integration and SOA constitute the majority of enterprise systems for a long time—into the SOA fold. When connecting to enterprise resource planning. making systems more accessible to integration. Some integration vendors enable a more seamless transition by making operations exposed through an adapter automatically available as Web services. Creating Web services interfaces for existing systems requires considerable development and investment. Process optimization requires managing the end-to-end business process. and process optimization through process automation. This forward migration is often expressed in terms of putting Web services wrappers around legacy interfaces to make them available to the SOA world. it may not be desirable from a financial or risk-management perspective. packaged. Though jumping from an existing legacy interface to one based on Web services may be technically feasible. Web services do not provide the requisite infrastructure services for organizations focused on using SOA to support a strategic initiative to optimize business processes. the investment will pay off over time. This is what business process management (BPM) systems do. and by componentizing application functionality into business services. If the service will be reused often. Web services orchestration defines the flow of control for a composite application.

process orchestration. In short. but true business integration demands higherlevel capabilities such as process orchestration. governance on the use of organizational IT assets. monitoring.Chapter 5 Integration and SOA 4 Interoperability and integration are not equivalent. the expertise and competencies developed for integration give organizations a head start with SOA. but also provides important infrastructure capabilities for delivering the reliability. and exception management. These same principles apply in practice to SOA. Web services promote cross-platform interoperability. process monitoring. and manageability that global organizations expect. meet required service levels. without which sustainable reusability would not be possible. data reconciliation. 4 The disciplines behind integration and SOA-based development are similar. the expertise and processes developed to manage integration interfaces—to ensure that they are well specified. A coherent SOA infrastructure will also help manage the complexity of the environment.  . Organizations that take a strategic approach to integration understand that success lies in appropriate methodologies. auditing. These capabilities are necessary elements of SOA-based business solutions. simply making Web services available on the network does not create an SOA. and the other integration capabilities described in the previous chapter—as well as capabilities for security. The latter capabilities are especially important for enabling SOA to address mission-critical business requirements. management. and ensuring predictable quality of service. Enterprise-Class SOA As stated earlier. versioning. scalability. and so on—can be applied directly to a registry of business services. For example. It is useful therefore to think of an enterprise-class SOA as one that not only incorporates the business services underlying the applications delivered via the SOA. and dedicated organizational structures and roles. numerous infrastructure capabilities are needed—such as data translation and transformation. For an SOA to be effective in the real world. and best practices developed for integration can be applied to SOA. Integration technologies and solutions exist to fill these needs. but they are not features provided by Web services.

Chapter 5 Integration and SOA Company Description Case Study 5-3 Company: State Bank State Bank is a pseudonym for a European bank based in Germany. making them reusable and easier to maintain while eliminating the single point of failure. This requirement made State Bank’s previous interface model obsolete because it was time-consuming and led to a single point of failure. Business Challenges State Bank was introducing SAP’s Bank Analyzer. The bank implemented webMethods Fabric as the SOA-based integration layer for connecting its legacy systems and databases to the FDB. Bottom-Line Business Benefits The SOA-based integration solution has replaced State Bank’s obsolete interface architecture with a flexible. As a result. obtain foreign exchange prices in real-time. the new platform is already processing 170. which includes a financial database (FDB). State Bank can more easily synchronize customer information with the customer master data. and all enterprise applications needed to be connected to it. all of the bank’s systems can now communicate with the FDB for the SAP solution. The FDB was to be the bank’s new core database.700 employees and over €70 billion in revenues. As a result of its approach. reusable system. 0 . State Bank has more than 1. and update new securities holdings after every transaction. With a focus on midsize enterprises. How Integration Addressed the Challenges State Bank decided that adopting an SOA would shorten the development time of the interfaces. the bank offers its domestic and international customers the financing services of an internationally oriented credit institution and the investments and services of an investment bank. and used XML to map all the data entering and leaving the integration layer.000 transactions each month. After only a three-month implementation phase.

Chapter 5 Integration and SOA sOa InfrasTruCTure Deploying an SOA infrastructure as part of the overall SOA implementation strategy is vital if organizations are to realize the benefits promised by SOA. and problem diagnosis 1 . session-related information. Web services requests and responses. dynamic systems from Web services 4 Provides a central point for managing and monitoring deployed services 4 Delivers the performance and robustness required for business-critical systems through features such as load balancing and automatic failover between services 4 Provides mechanisms whereby services can easily locate one another. such as transformation and filtering 4 Allows services to be provisioned dynamically as business needs fluctuate 4 Provides security across services. exception handling. long-term ROI requires a proactive approach to creating the SOA infrastructure. 4 An enterprise-class SOA infrastructure: 4 Promotes architectural consistency by ensuring that Web services are deployed in a manner consistent with a service-oriented model 4 Facilitates the assembly of large-scale. and promotes loose coupling of business-oriented services through location transparency and dynamic binding 4 Provides metadata management and registry services to support the reusability of services 4 Provides a framework for creating and deploying infrastructure services—services that function between business-level services. regardless of where they are deployed in the network. addressing needs such as client authorization 4 Offers various levels of logging—for example. and security events—to support trend analysis. Although a tactical approach might be possible in the early stages of an SOA implementation.

Figure 5-4. different Web services containers may have different capabilities (including no native Web services capabilities at all). In addition. support for a specific WS-* standard). even when different containers have similar capabilities (for example. service monitoring. calls are made through an interception layer that provides a consistent set of infrastructure capabilities across service providers.Chapter 5 Integration and SOA An important requirement of an enterprise-class SOA infrastructure is that it exist independent of the business services and applications it supports. In addition to enabling consistency. this layer enables the SOA infrastructure to do things like load-balance across different services and provide Web services capabilities to systems that are not natively built on Web services. As the figure on the next page illustrates. this means that instead of consumers calling Web services providers directly. Conceptually. and quality of service is to externalize the SOA infrastructure from the service providers. the actual implementations may vary or be incompatible because of version differences. As a result. transactions. the only way to achieve the appropriate consistency across different service containers for functions such as security. Enterprise SOA infrastructure  .

It should also be capable of operating natively within supported platforms. governance becomes an essential part of the SOA strategy. a developer can make an informed decision about whether to use the service for a specific business requirement. One important area of SOA governance is managing reuse. overlap. and enforcement. As Web services begin to proliferate in organizations and services are reused in different applications and different areas of the organization. Although the appropriate technologies are important. by profiling the performance characteristics of a given Web service (such as average response time). With this flexibility. Although these issues are primarily ones of methodology and process. A company may need to implement a new business policy that requires customer credit card information to be transmitted  . it leads to duplication. and as an intermediary for nonsupported platforms. Consider the FindCustomer service. such as an application server.Chapter 5 Integration and SOA As explained. an SOA infrastructure can bring all the services in an organization’s environment under a single management scope. Managing reuse means architecting services so they can be reused. technologies. corresponding policies and procedures are just as crucial to realizing the benefits of SOA. For example. As experience with objectoriented programming has shown. and an overall loss of control that erodes the reusability benefits of SOA. So does the need to enforce policies and procedures throughout the development and maintenance life cycle of a Web service. Reuse does not happen by accident. some vendors provide capabilities to help manage reuse. An additional governance issue relates to policy definition. if ROI is to be realized. and procedures. sOa gOvernanCe Providing the capabilities of an enterprise-class SOA infrastructure requires a combination of standards. and providing appropriate incentives for reuse. implementation. monitoring how services are reused. such as a packaged application or a service hosted by a third party). When the proliferation of Web services is unmanaged. the discipline of reuse must be designed into the process. the SOA infrastructure should be non-intrusive to the business services themselves. policies.

unencrypted version of that service. it is also a strategic asset in the SOA infrastructure from a governance perspective. and security. A final issue relates to the operational monitoring and management of Web services from a service-level perspective. A Web service may be used by applications across the enterprise that have different quality-of-service expectations regarding availability. the more progressive software vendors offer not only technology solutions but also a broader portfolio of capabilities that includes best practices and methodologies for helping organizations realize their SOA strategies effectively.  . including users outside the organization. The developer of the service can easily ensure that the service receives and delivers only encrypted information.Chapter 5 Integration and SOA in an encrypted format to protect customer confidentiality. It may be possible to monitor the performance of the service in isolation. the benefits of SOA rapidly decline. and enforcement of service policies. The challenge is communicating and enforcing the policy change with individuals who are using the earlier. A Web service may be created for use within a single department but then adopted by other users. and tactical Webservices-based solutions begin to proliferate without effective governance. An ad hoc approach to Web services management provides no mechanism for implementing a security policy and enforcing compliance across the organization. performance. Because of this. When issues such as these are not addressed at an enterprise level. Putting policy rules in a centralized governance repository makes them much easier to enforce across applications and platforms. Enforcing security policies across services is another important governance issue. implementation. and run-time policies can be enforced when a service is invoked. development policies can be enforced when a service is registered. A centralized registry helps support the definition. Security must be maintained across users. For example. but the more important requirement is whether the service satisfies the service-level expectations of a particular user. As such. the UDDI registry is increasingly viewed as more than just a place to catalog services.

These can be avoided with strong enterprise controls and best practices. Control redundancy. Among other things. organizations must implement an enterpriselevel approach. Programmers need to understand how to design and build systems that follow and take advantage of the new paradigm. and providing active administration. as described in Chapter 6. it is not enough to provide programmers with an integrated development environment for creating Web services. if one exists. having a registry of the services available for reuse. To reap the rewards of SOA. Policies and incentives for reuse are also important. Duplication of technical and business services drastically increases maintenance costs and complexity.  . Best practices include the following: Create an enterprise SOA group. The enterprise SOA group is similar in function and purpose to the Integration Competency Center (ICC). and managing the SOA’s incremental implementation by working with project groups across the organization. and may evolve as an outgrowth of the ICC or the Enterprise Architecture Group. developers should understand how to design services at the right level of granularity to maximize reuse. Manage reuse. managing reuse requires planning and designing for reuse. For long-term success. organizations need to invest in developing a core competency in SOA design and implementation. As mentioned earlier. Redundancy usually results from poor governance and a temptation to take shortcuts. Loosely coupled Service Oriented Architectures provide maximum business agility. For example. maintaining appropriate documentation and metadata about reusable assets. not a technology or a tactical solution. However. reuse must be managed if any benefits are to be realized.Chapter 5 Integration and SOA Best Practices in SOA By now it should be clear that SOA is an enterprise strategy. This group is responsible for defining the enterprise SOA architecture and roadmap. Develop SOA competency within the organization.

and its approach to investment in applications and software infrastructure. managed approach that has implications for an organization’s development practices. a strategic approach offers a much higher ROI than what might be experienced with quick and easy tactical solutions. but the benefits accrue as the number of deployments grows. Although there are many important technology considerations—including the implementation of the integration technologies represented in the ebizQ Integration Roadmap (Chapter 4)— SOA is about more than technology. Over the long run. The broad. Taking the proper strategic approach requires an initial upfront investment.Chapter 5 Integration and SOA Conclusion SOA is the undisputed path for creating an agile IT infrastructure. its organizational structures.  . successful adoption of SOA requires a proactive.

integration needs to become a core competency within the organization so that new solutions can be easily implemented on an ongoing basis. and lacking processes and procedures for implementing integrated solutions.6 Best Practices for Integration Executive Overview A major theme of this book is that integration and Service Oriented Architecture (SOA) are not shrink-wrapped solutions. and in short supply. defines roles and responsibilities. This chapter addresses some common integration pitfalls. and identifies the critical requirements for successful integration projects.  . organizations can avoid the pitfalls that jeopardize an integration project’s success. If an organization wants to reap all the rewards that integration has to offer. One of the more important best practices to emerge is the notion of an Integration Competency Center (ICC) within the IT organization. An ICC provides an environment in which critical skills can be honed. integration processes can be developed and tested. Risks include ignoring data quality issues. avoid common pitfalls. Integration skills are specialized. However. it needs to follow best practices. nor are they simply about technology. lacking clear requirements or well-defined project plans. Common Pitfalls By taking a few precautions. explains the importance of taking a strategic approach to integration. and governance can be applied to reusable integration assets and business services. examines various models for establishing an ICC in an organization. knowledge and experience from previous projects can be consolidated and reused. and understand how to structure the organization for success. Most organizations will spend considerable sums on implementation and consulting services. in demand. not having a cohesive organizational structure.

it is a good idea to meet key trading partner contacts in person. and operations management to ensure a smooth process. and testing processes that factor in the complications of dealing with external entities. As the adage goes: “garbage in. Integration is pointless if the underlying data quality is poor.  . but integration projects are particularly vulnerable because of the number of related parties involved. garbage out. Some level of data cleansing in the source systems is often necessary to avoid downstream exceptions that reduce the efficiency of an automated integration process. Another aspect of poor planning is failing to coordinate and establish realistic deadlines when integrating with trading partners. milestones. Lack of clear requirements. Organizations frequently fail to plan for the long lead times often associated with hardware and software procurement or firewall and other security changes. Consequently. meaning that even minor changes to the requirements can entail significant rework of the design and solution. This pitfall can be avoided by working closely with procurement.” Before integrating systems. When kicking off a project. the devil is in the details. security. This is a potential issue on all projects. Before beginning an integration project. the integration requirements become a moving target that is much harder to hit. Organizations should work closely with their trading partners to establish realistic project timelines. Organizations should identify the key stakeholders and decision makers for all the interested parties. They should also communicate frequently to ensure that all parties are on track with the schedule and milestones.Chapter 6 Best Practices for Integration Data quality issues. and establish a formal process to obtain sign-off. Inadequate communication between IT personnel and key stakeholders in the business opens the door for future scope changes. Poor coordination between IT and business stakeholders. This often reduces the communication barriers during the project. With integration. organizations should first assess the quality of the original data to determine whether it meets the usage needs elsewhere. Poor project planning. When an integration project is started while an application is still being developed or customized. dependency-related delays and scheduling issues are common. organizations should first ensure that the source systems and interface requirements have solidified.

Organizations today need individuals with the skills required by more sophisticated software and integration platforms that offer built-in support for integration-specific requirements such as transactions. Treating each integration requirement on a project-by-project basis does not afford the opportunity for reuse from one application to the next. integration with customers and suppliers). few organizations have invested in the resources required to support these new technologies.Chapter 6 Best Practices for Integration Lack of development standards and guiding principles. It also increases maintenance costs. and code promotion standards are critical for agility and reuse. The increasing complexity of integration requirements is also fueling the push toward a strategic approach. However. Organizations need to establish standards and guiding principles early to ensure a consistent and orderly solution. they can no longer treat integration as a series  . and process automation. configuration management standards. auditing. Although application developers have been doing integration for many years.specific programming interfaces. application. This is quickly changing because of the inefficiency and cost of taking a piecemeal approach to integration. most of this has been done in a point-to-point. Naming standards. integration today seeks to automate process flows both within the organization and across organizations (for example. business process modeling. Taking a Strategic Approach to Integration Historically. Rather than focusing simply on linking two or more applications together. integration skills tend to be specialized and scattered throughout an organization’s IT community rather than centralized in one group. most integration projects have been undertaken as tactical projects. or as tasks within a larger implementation project. since there is no opportunity for reuse and no way to gain economies of scale. As a result. application-to-application manner using custom code. or flat-file data transfers. If organizations want to more efficiently build and maintain the complex integration processes that their businesses demand. A tactical approach also inhibits the development of integration skills in an organization.

An integration team can range from a few resources to several dozen and can vary in the level of services offered—from simply developing guidelines and best practices to serving as a full-scale shared services provider that offers hosting and administration for all of the organization’s integration needs. Regardless of the team’s size or the services offered. IT groups operate fairly autonomously. The actual composition and services of the ICC will depend in large part on the IT organization’s infrastructure. greater skills development. it can provide a set of integration guidelines for the independent IT organizations. or a hybrid of the two. and increased business agility. distributed. The benefit of this model is that knowledge is shared and some procedural reusability is achieved. This increases the likelihood that individual integration projects are architected properly. In this model. project-driven tasks. product lines. Organizing for Success When it comes to creating an organizational structure to support integration. That is.Chapter 6 Best Practices for Integration of heterogeneous. business units. there is no “one-size-fits-all” approach. or geographical units. though ultimately the project teams implement the technology and thus retain a fair degree of autonomy. the ICC model can be centralized. 100 . Organi-zations that take a strategic approach to integration by creating a separate organizational entity to focus on integration issues and promote best practices throughout the organization will benefit through lower maintenance costs. The ICC may also make technology recommendations. best practice recommends that an organization create an Integration Competency Center. and there is little coordination among them. an organization’s IT resources are divided along functional areas. Although the ICC may not be able to enforce policies and standards in a distributed IT organization. In the distributed model. the ICC establishes best practices that standardize processes (such as development methodologies and naming conventions) across project teams.

Chapter 6 Best Practices for Integration In a variation of the distributed model. the ICC acts as a full-service provider. As before. the ICC is responsible for building and maintaining the integration backbone. the ICC establishes the methodology and software standards. The ICC builds and maintains a library of standard interfaces into enterprise systems that application developers can reuse in their projects. It also assists the IT operations staff with production support and monitoring once the integration project is in production. A full-service ICC typically performs the following functions: 4 Running the integration platform as a public utility. 4 Providing data center operations for integration projects. the ICC needs to offer services such as capacity planning. Just as a telephone can be plugged into a wall jack and receive phone service. and deployment assistance to the external integration teams. and operations support services. but it also delivers architecture. This scenario provides a greater degree of consistency than the previous one. the ICC defines not only the standard procedures but also the standard integration technologies that project teams should use. and production environments for integration projects. disaster recovery planning. 101 . Because the integration platform is not under the ICC’s control. however. in this model it also contributes to the development of shared integration assets. test. A centrally run and administered integration backbone has been likened to a public utility. In a highly centralized IT organization. design. developers can plug systems into the integration backbone and tap into the stream of data flowing through the integration platform. some of the integration implementation workload shifts to a centralized group of development resources. In the centralized model. As before. Not only does it set the standards. development. 4 Offering standard interfaces to organizational systems. The ICC basically acts as an internal consulting group to the distributed IT organizations. All or most of an organization’s application development and IT infrastructure is consolidated into one organizational IT group. the actual implementations are done by the project teams. In the hybrid model. with resulting benefits to reusability. The ICC maintains the shared integration infrastructure and allocates development. However.

The downside of this model. it might be applied to a select subset of projects but seldom across-theboard. This role is critical to maintain data consistency and a high level of reusability. 4 Maintaining the enterprise information model. the ICC must develop and maintain a business object model (also referred to as a canonical data model) of all the documents and data to be exchanged through the integration platform. and standards are easily set and administered. design. providing architecture. is the increased effort required to effectively manage and respond to the project teams’ commitments and priorities. and reuse. procedures. and development services either directly through its own staff or by contracting with preferred systems integration partners and off-shore development companies. Allowing project teams to innovate without constraints may have some benefits. But this scenario runs counter to the SOA philosophies of interoperability. establishing their own development practices and making one-off technology choices. A centralized model provides several benefits. governance. Additionally. as with any centralized service organization. Policies. In an environment where multiple integration processes publish and subscribe to data.Chapter 6 Best Practices for Integration 4 Providing integration development services. The ICC acts as an internal consulting group to project teams. project teams operate entirely independently. having all the services under one roof and point of control maximizes the opportunity for reuse and consistency. In the absence of a formal ICC. 10 . As such.

integrated systems process 300 million transactions per week across all projects. whose charter was to create a strategy and platform for integrating business processes and applications.Chapter 6 Best Practices for Integration Company Description Case Study 6-1 Company: MK Healthcare Products MK Healthcare Products is a pseudonym for a Fortune 50 manufacturer of healthcare products. templates. The methodology uses standards. By moving toward a centralized approach to enterprise integration. combine education and training initiatives. resulting in a cost avoidance savings of $9. tools.1 million. and it significantly limited the organization’s ability to maximize knowledge sharing. consumer products. the operating companies developed new processes and services individually. Each operating company had its own integration strategies. share code and components. A combination of organic growth. or business processes. consolidate hardware. MK Healthcare Products was able to standardize on an integration platform. applications. and data across the organization’s multiple operating companies. acquisitions.000 people in 57 countries and sells products throughout the world. common technology. and standards. from project initiation to deployment. How Integration Addressed the Challenges MK Healthcare Products created an Integration Services Group. In the past. and a global market has increased the need for collaboration of business processes. MK Healthcare Products employs approximately 109. To date. processes. 10 . This often resulted in systems that could not integrate easily with one another. without a common platform. The company was founded over a century ago and currently has more than 200 operating companies grouped into segments focusing on pharmaceuticals. a common design approach. and improve business processes. architecture. and leverage best practices across the operating companies. It adopted a total business integration methodology based on an enterprise application integration solution to connect the operating companies with one another and with external business partners. more than 120 projects have been implemented using the total business integration methodology and architecture. which equates to $30 billion in transactions annually—approximately half of MK Healthcare Products’ annual revenues. The new. Bottom-Line Business Benefits The centralized Integration Services Group has made tremendous gains for MK Healthcare Products. and medical devices and diagnostics. and a philosophy of reuse and covers the entire integration life cycle. reduce redundancies. Business Challenges MK Healthcare Products is well known for its highly decentralized management structure.

an integration team should incorporate the following roles and responsibilities: Business analyst. the integration specialist is the middleware expert. Process-oriented developer. Services may be created from scratch (for example. The application specialist builds the individual components and Web services from which higher-level integration applications and processes can be assembled. and provides some operational support. the application and integration specialist roles are filled by the same individuals. conducts capacity planning and other system-level tests.Chapter 6 Best Practices for Integration Integration Roles Regardless of the model chosen. In some organizations. define the related business processes and integration requirements. writing a new customer lookup service). or by wrappering existing functionality with Web services interfaces. turning 10 . The integration specialist also handles the detailed technical engineering of the implementation. Typically. The integration specialist develops custom adapters and interfaces. With integration requirements increasingly being driven by business process automation. the application specialist develops business-level services that are associated with one or more applications. and perform the associated data-level analysis. so he or she needs to have some level of application domain expertise. the integration team or ICC should include resources with strong business domain expertise and analysis skills. and other utility integration services to enable the implementation of more complex business process and integration solutions. The process-oriented developer bridges the gap between the business analyst and the application and integration specialists. While the application specialist focuses on developing reusable business services. The integration specialist’s expertise is generally centralized within the ICC. The primary responsibilities of the business analyst are to understand the business needs. Application specialist. by recombining existing services into new services. reusable infrastructure services to facilitate different integration scenarios. Integration specialist.

and mentors the rest of the IT organization to ensure that the integration software is appropriately leveraged and properly implemented and that integration solutions are architected to take best advantage of the integration infrastructure. The relationship manager acts as the central contact between the organization and the vendor. The operational support tasks that the ICC takes on can vary widely. In addition to driving standard integration practices and methodologies. then project managers need to be on board. 10 . the integration architect is often responsible for educating the IT community on the merits of the integration platform and evangelizing the adoption of the software throughout the organization. performs design reviews of integration implementations.Chapter 6 Best Practices for Integration business process models into executable business processes that link together the services developed by the application and integration specialists. The staffing of project managers within the ICC also depends on its charter. Project manager. He or she keeps the vendor informed of issues that arise within the organization and notifies the organization about the vendor’s current and future product and service offerings. the ICC should include resources that can provide integration platform deployment guidance to project teams and data center personnel. At a minimum. Integration architect. the number and composition of the operational support team will vary accordingly. If the ICC is responsible for providing hands-on consulting to the IT community. This individual defines the overall integration strategy and architecture. The process-oriented developer needs both a business-level understanding of the business process and the technical expertise to assemble and orchestrate the underlying services into a process. depending on its scope. Operational support specialist. Relationship manager. Therefore. A strong vendor relationship is critical to the successful adoption of new technologies. The integration architect plays a critical role within the ICC.

processes for managing and controlling the use of enterprise assets—is essential for ensuring reusability. batch integration processes. and instructions for building security into integration processes 4 Creating interoperability techniques to standardize the integration of heterogeneous platforms—for example. harvesting. This section describes the major areas of responsibility for most ICCs and lists possible objectives for each area. Governance responsibilities of the ICC include: 4 Establishing policies for how an organization’s services are built and maintained. standards for documenting an integration. flat-file integration processes. develOpmenT sTandards and praCTICes An important function performed by an ICC is developing standards to facilitate reuse and interoperability.Chapter 6 Best Practices for Integration Responsibilities of the ICC The individual responsibilities that an ICC takes on will depend on the organization’s stated goals for the ICC as well as the nature of the organization’s IT group. and productivity. is in the area of governance. real-time data lookups. procedures for promoting code to production. integrating between the webMethods Fabric platform and IBM MQ Series-based applications 4 Developing architectural patterns for common integration scenarios—for example. naming standards and conventions. and maintaining reusable integration components gOvernanCe Another important function of an ICC. and transactional business processes 4 Developing and maintaining an organizational business object model to allow disparate integration processes to interoperate easily 4 Developing. Governance—essentially. Responsibilities in this area include: 4 Creating and maintaining integration best practices—for example. especially in relation to SOA. 10 . long-term maintainability.

the ICC should act as the main contact between the organization and the vendor.Chapter 6 Best Practices for Integration 4 Defining and managing SOA domains. providing expertise on the use of integration tools and techniques. and ensuring that development practices incorporate the appropriate use of the repository. and negotiating with vendors 10 . To facilitate the relationship. 4 Managing the organization’s business services repository and the associated metadata. Responsibilities include: 4 Working with the vendor to help market the benefits of both the products and the ICC’s services to the organization 4 Prioritizing the organization’s enhancement and feature requests. 4 Establishing service level agreements for business services. The domain owner brokers the relationship between line-of-business users and the owners of the applications on which the domain services are built.and second-tier support for integration tools 4 Assigning individual developers to project teams to build integration processes CusTOmer advOCaCy and vendOr managemenT As mentioned earlier. continued success with a chosen technology is often predicated on a strong relationship with that technology’s vendor. COnsulTIng and TraInIng The ICC can act as an internal consulting group to an organization’s IT community. which are sets of services related by a common business context. Services include: 4 Providing architectural guidance and review for individual projects 4 Developing and delivering custom training materials 4 Providing in-house first.

which would deploy the integration processes. At one end of the spectrum. create the physical infrastructure on which the enterprise integration backbone is deployed.Chapter 6 Best Practices for Integration 4 Participating in vendor early-release programs and planning for the rollout and migration of new and upgraded projects TeChnOlOgy leadershIp The ICC should constantly look for advances in integration technology that can benefit its organization. It would. the ICC may own and administer the hardware on which the integration platform is deployed. in essence. ICC services in this scenario might include: 10 . Project teams would request development and production space on the shared platform and then turn over their integration processes to the ICC to deploy and monitor in production. In this scenario. the ICC could simply provide integration platform deployment guidance to the data center team. ICC services include: 4 Procuring and maintaining the physical integration platform infrastructure 4 Providing production support for integration processes built by project teams 4 Allocating development and testing environments for project teams At the other end of the spectrum. Services in support of technology leadership include: 4 Writing white papers on emerging technologies and trends 4 Hosting internal information-sharing sessions 4 Staying up-to-date on new vendor features and capabilities 4 Developing prototype applications using new technologies OperaTIOnal suppOrT The area of operational support is where the ICC’s responsibilities are most likely to differ from one organization to the next. The rest of the organization should view the ICC as an expert body for advice on implementing and deploying integration technologies.

integration developers would support the integration logic they have developed. and infrastructures that were established for the pilot project. Or it can be established in concert with an organization’s first integration project and then evolved. We have found that it is better to take an evolutionary approach and allow the ICC to develop from the pilot integration project. As a result.Chapter 6 Best Practices for Integration 4 Planning capacity for integration platform components 4 Planning for disaster-recovery scenarios 4 Developing fail-over and load-balancing environments 4 Providing production support for integration platform software—for example. 4 Members of the IT community are more willing to accept the ICC’s guidance because the staff has real-world experience. The following example illustrates how to build a full-service ICC using an evolutionary approach. data center personnel would be responsible for hardware and operations system support. and the ICC team would be responsible for supporting the integration platform software Creating an ICC There are two general approaches to forming an Integration Competency Center. This increases the opportunities for success. the ICC can be transitioned to an independent body and can continue refining the methods. reusable architectures. 10 . and reusable frameworks produced by the ICC tends to be higher because they reflect the team’s actual experience working with the integration tools in a project setting. the ICC’s return on investment can be achieved quickly. Once the original project has been completed. tools. practices. 4 Management is more likely to approve funding for the ICC because it is contributing immediately and directly to project deliverables. for several reasons: 4 The quality of the standards. An ICC can be created independent of any one integration project.

the ICC: 4 Assesses best practices and standards developed in the pilot project. OngOIng suppOrT As the ICC becomes well established and its standards and practices are deployed. 4 Begins building the reuse library by harvesting integration components from the pilot project and making them more generic. TransITIOn TO The IndependenT OrganIzaTIOnal unIT Once the pilot project has been completed.Chapter 6 Best Practices for Integration pIlOT prOjeCT In this phase. MQ Series. and CICS mainframe applications). The ICC staff should include a professional services resource from the integration platform vendor to facilitate the knowledge transfer of best practices and architectural techniques to the ICC team. the ICC moves to an independent organization. development and deployment techniques. gaining resources by transitioning some of the pilot project team members into the ICC. Web services. the team continues to provide support to the organization and: 110 . 4 Builds architectural frameworks around common patterns of integration. organizational systems that are widely integrated to. and publishes them for use on future integration projects. This will allow future integration projects to be more productive as well as ease maintenance and support. The ICC’s aim is to help develop standard architectures. In this phase. it is important to formalize the ICC’s charter and clearly spell out the services that the team plans to offer the IT community. and the beginning of an enterprise information model. As this phase begins. since all integration processes will be based on a common framework. for reuse on future projects. makes any changes necessary to render them generic. and common technologies (for example. if necessary. the ICC team may consist of only one or two resources working with the pilot project team.

building out the enterprise information model to define a single view of customer data. and so on. the following factors are critical for success: Link the ICC mission and charter to business objectives. which is essentially a cost center to the organization. 111 . For example. reduce implementation costs. The ICC’s charter and deliverables must be aligned with the organization’s overall objectives. Even when based on sound technology and executed flawlessly. and so on. If the organization is focused on improving customer service. if the orga-nization is focused on cost savings. the ICC should strive to make integration processes more efficient.Chapter 6 Best Practices for Integration 4 Stays current on integration industry trends and practices 4 Participates in vendor early-release programs to plan for upgrades and to advise project teams on the use of new features 4 Continues educating the IT community on integration technologies and best practices 4 Continues building the reuse library by gathering components from project teams as well as assessing the need for common integration processes and developing them from scratch to offer on future projects 4 Maintains and grows the enterprise information model to ensure that the integration backbone provides a coherent canonical model for all the applications deployed on it 4 Maintains the business services repository and expands the inventory of services available for reuse Critical Success Factors Integration can be a complex subject because multiple technologies may be applied to different business scenarios. the ICC should concentrate on creating standard integration processes across all organizational systems that store customer data. many a project has been seen as a failure because it did not deliver on the business objectives that the organization was striving to meet. Linking mission to business objectives is even more critical when forming an ICC. Regardless of the type or scope of the integration project.

Obtain executive support. valuable expertise could leave the organization altogether as the systems integration partners complete their assignments and move on. Integration is inherently complex. However. Then the successes should be tracked and communicated. Taking on too much too quickly can lead to failure. It is better to underpromise and overdeliver. For each project. or improving customer service—should be defined. Instead. integration experience could become scattered throughout the organization and be difficult to leverage across projects. In the early stages. Management support is critical for helping to break down any barriers to acceptance that may exist within the organization. Automating. the ICC should be supplemented with experienced professional services resources from the vendor or a systems integrator to satisfy key project team needs. Integration projects routinely cross organizational boundaries. optimizing. This will build confidence in the ICC and encourage further organizational investment in integration infrastructure projects.Chapter 6 Best Practices for Integration Identify and communicate integration and ICC successes. it should start small and add integration processes incrementally. Many benefits of SOA and integration come from reusing existing organizational assets. specific key performance indicators—such as reducing cost or implementation time. Policies and procedures should be defined. This is a critical success factor for managing reuse. Start small and build on successes. Define governance policies and procedures. Worse. and managing end-to-end business processes or providing unified access to distributed information can mean stepping on toes. Systems integrators and consultants on an integration project must transfer their knowledge to ICC resources. Build a core competency in integration within the organization. It’s critical to invest in training to continue building skill sets and knowledge. An organization should not begin by trying to boil the ocean. ICC resources should be involved regularly in integration projects so they can continue to increase their skills and add to the collective knowledge of the organization. enabling others to utilize existing functionality for new solutions requires proactive governance of the reusable assets. as should service level agreements. If the ICC does not actively focus on these tasks. 11 .

Finally.Chapter 6 Best Practices for Integration Conclusion As organizations invest more strategically in integration and SOA. reusability. Establishing and evolving an ICC is a proven approach to promoting best practices. making efficient use of resources and skills. and agility that enable organizations to reap the rewards of integration. and improving the consistency. To successfully navigate these obstacles. 11 . organizations that have made a strategic investment in an ICC will find themselves poised for success with SOA. many decisions and obstacles are sure to present themselves. it is critical to develop and maintain integration competency within the organization. because of the synergy between SOA and integration and the commonality of best practices.

Beth holds an M. Office of the CTO. Gary So VP. in Computer Information Systems from Bentley College. webMethods. ebizQ Beth Gold-Bernstein is a recognized expert in integration technologies and SOA. Gary So is Vice President. Formerly VP of Strategic Marketing. he was in charge of driving webMethods’ agenda through media and analyst relations.. before joining webMethods in 2000 and spear-heading the development of the company’s integration methodology. architects.” was published by Prentice Hall in August 1997. She has over 20 years experience working with financial institutions. and implementers. Gary has over 10 years experience in the integration field.Beth Gold-Bernstein VP. Beth is the author of “Enterprise Integration: the Essential Guide to Integrated Solutions. “Designing Enterprise Client/Server Systems. Inc. Office of the Chief Technology Officer. serving previously as a system architect in corporate IT and as a director of professional services at Active Software.” published in July 2004 by Addison Wesley. Gary has a Masters degree in Computer Engineering from the University of Toronto. Her first book. at webMethods. She has developed training programs for business analysts. 11 . Inc.S. where he is responsible for advancing the company’s status as a recognized industry thought leader. and speaks nationally and internationally at conferences and seminars. designing and implementing large scale distributed systems and enterprise architectures. retail chains and manufacturers on planning. Inc.

Sign up to vote on this title
UsefulNot useful