Professional Documents
Culture Documents
Vision
Mission
ii
Course Code: IT 4129
The course will explore tools and techniques for systems integration as well as proven
management practices for integration projects.
Course Requirements:
Assessment Tasks - 60%
Major Exams - 20%
Periodical Exam - 20%
_________
Periodic Grade 100%
iii
MODULE 1
OVERVIEW OF SYSTEMS INTEGRATION:
CHALLENGES AND DRIVERS
Introduction
According to Motiwalla & Thompson (2012), today, perhaps more than ever before, it
is essential that companies be efficient and effective with their products and services.
There are many drivers in organizations for needing integrated systems. The ability to respond
quickly to market conditions is a key part of protecting your customer base against the
incursions of a global set of hungry competitors. It is also the key to growing or retaining that
customer base. In other words, the inability to meet the market demand effectively can have
unfortunate consequences. Having too much or not enough inventory, or having the inventory
at the wrong place and the wrong time, can have a disastrous impact on a company’s
profitability—and even survivability. Integrated systems allow companies to accomplish
something that has alluded most to date: the linking of demand- and supply-side functions in
a way that enables a quick and flexible response to changes in demand. Developing
processes to support integrated systems is not an easy task, but it can be done, as evidenced
by industry leaders like Dell, Amazon, and others that have already put integrated systems in
place.
Learning Outcomes
4
2. Know what systems integration is and why it is important for organizations;
and
3. Understand the role of enterprise resource planning (ERP) systems in
systems integration.
5
Lesson 2. Steps in Integrating Systems
In conjunction with systems integration, management has to work with the
information technology group to come up with an approach for the seamless
integration of data and services to support the new organizational structure and
business processes (Motiwalla & Thompson, 2012, p. 47).
As mentioned before, organizations tend to add functionality to meet
organizational demands. At times, application systems are added to the
environment (Motiwalla & Thompson, 2012, p. 47).
These applications, while not encouraged, are developed on different platforms.
Information system organizations often have to be able to support a variety of
systems with multiple platforms and vendors (Motiwalla & Thompson, 2012, p. 47).
This could mean supporting multiple operating systems, databases, or
development environments. Most IT organizations today support a Windows and
flavor of UNIX. A database can be Oracle or MS SQL and even MySQL (Motiwalla
& Thompson, 2012, p. 47).
Most important is the support of a development environment. This area continues
to grow. At one point in time, C or C++ with SQL was the key development tool.
That has somewhat given way to Java and SOAP with SQL. Integrating and
supporting multiple platforms requires planning (Motiwalla & Thompson, 2012, p.
47).
6
Table 1.1 System Integration Steps (Motiwalla & Thompson, 2012, p. 47)
Step 1 Resource categorization Take an inventory of the various hardware and
software resources focusing on vendors, operating
systems platform, IS architecture used in these
resources.
Step 2 Compliance and standards Check whether the database and other
technologies used in various applications are such
supporting standards as JDBC/ODBC compliance
for databases.
Step 3 Legacy systems support Develop a policy in support of older legacy
applications.
Step 4 Middleware tools Think of middleware tools because most
organizations will not dispose of their old system
right away for systems integration. Middleware tools
are essential for integration in the short term—if
existing applications must be used by the
organization.
Step 5 Authentication and Develop a single sign-on policy for application and
authorization policies data access because all employees and external
partners will need access to an integrated system
from anywhere, anytime.
Step 6 Centralized IT services and Instituting IT support for an integrated systems
help desk support environment is necessary to avoid support and
maintenance problems with the integrated system.
Centralization does not mean that they are all
physically in one location. The IT staff can be all over
the organization, but they need to be able to support
all applications and platforms with a centralized IT
help desk support.
Step 7 Backup, recovery, and security Planning data and disaster recovery for
policies organization’s data in an integrated system IT is
crucial for building the trust and confidence for the
new system. A good backup and recovery system is
essential if there is a system failure or a major
disaster
Step 8 Hardware and software and Develop organization standards and policy on
standardization policies acquisition of new hardware and software which are
aligned with organization IT strategy.
7
Lesson 3. Benefits of System Integration
According to Motiwalla & Thompson (2012), if done right, systems integration can produce
tremendous benefits as shown in Table 2.2. Some of the key benefits of systems integration
are as follows:
1. Increased Revenue and Growth - In general, one of the biggest benefits is reduction
in inventory and personnel costs due to integrated systems. For example, Uvex Sports,
Inc., a sports gear company, saw sales grow from $1.2 million to $5.2 million without
additional costs and with the addition of only two extra employees.
2. Leveling the Competitive Environment - Systems integration can make a small
company behave like a big player because, with the help of integrated business-
to-business (B2B)software, many of them can now compete with big companies to
get orders from giant retailers like Walmart, Target, and others because they can
provide the same level of service with enterprise systems.
3. Enhanced Information Visibility – The increased availability of information enables
managers and employees to make informed decisions in a timely manner. For
example, customer service representatives of American Express can now make credit
approval decisions on the spot while talking with their customers due to better access
to customer credit profiles.
4. Increased Standardization - A side benefit of integration is that it forces organizations
to standardize on their hardware, software, and IT policy. This may initially cost some
money, but in the long run companies easily recoup those costs (Motiwalla &
Thompson, 2012, p. 48).
According to Motiwalla & Thompson (2012), systems integration does have its drawbacks, as
shown in Table 2. These are as follows:
1. High Initial Setup Costs - The initial implementation of integrated systems is high in
terms of both hardware and software costs and human costs due to the re-engineering
of business processes. Although these cannot be avoided, their negative influence on
the implementation can be minimized by a long-term resource allocation plan and
commitment from top management.
8
2. Power and Interdepartmental Conflicts - Systems integration often involves sharing of
information across department and interdepartmental teams. This often creates power
conflicts among the functional departments if they have not bought into the integration.
Educating employees with a good change management strategy that communicates
the long-term benefits from systems integration can minimize these conflicts.
Table 1.2 Benefits and Implementation of System Integration (Motiwalla & Thompson,
2012, p. 47)
Benefits of System Integration Limitations of Systems Integration
Increased revenue and growth High initial setup costs
Leveling the competitive environment Power and interdepartmental conflicts
Enhanced information visibility Long-term and intangible ROIs
Increased standardization Creativity limitations
As you can see, the benefits generally outweigh the drawbacks when
implementing systems integration projects, particularly in the long run. In industries
with competitive markets, systems integration is very necessary regardless of the
cost. For example, suppliers of automobile manufacturers (e.g., Ford or GM) or
retailers (e.g., Walmart) do not have a choice for systems integration. If they
skipped systems integration, they would not be able to do business with these
large companies, which moved to electronic data interchange (EDI) systems in
early 2000 with electronic commerce (Motiwalla & Thompson, 2012, p. 48).
Enterprise resource planning (ERP) systems are integrated, multi module application software
packages designed to serve and support several business functions across an organization.
An ERP system is a strategic tool that helps the organization improve its operations and
management by integrating business processes and helping to optimize the allocation of
available resources (Motiwalla & Thompson, 2012, p. 49).
9
b. At the logical level, ERP systems require organizations to focus on business
process rather than on functions (Motiwalla & Thompson, 2012, p. 49).
c. ERP systems come with built-in processes for a wide variety of common
business functions (Motiwalla & Thompson, 2012, p. 49).
d. An ERP system implements the best practices via specific built-in steps for
processing a customer order in terms of how the order information is entered
in other system, how it will be routed through various departments for actions
or decisions, and how the output from system is communicated to the various
parties, including the external customer and suppliers (Motiwalla & Thompson,
2012, p. 49).
e. While ERP systems can address data integration, if business processes do not
change, an organization will not be able to take full advantage of the ERP
capabilities. The term is business process reengineering (BPR) (Motiwalla &
Thompson, 2012, p. 49).
f. With the implementation of ERP business processes, organizational structures
and even roles and responsibilities within an organization will change.
(Motiwalla & Thompson, 2012, p. 49)
10
Lesson 6. Implication Management
According to Robert Tucker (nd), the author of the book Driving Growth through
Innovation, one of the reasons that innovation has not become embedded as a key
driver of growth and profitability in many organizations is that it has been limited
by functional and divisional “silos” within companies
In other words, the responsibility for innovation has been limited to the R&D
department, a special innovation SWAT team, or a senior-level strategic planning
group (Motiwalla & Thompson, 2012, p. 51).
Silos do not work. Most organizations lose out in the long term when information
is not shared in real time across the functional boundaries within the company
(Motiwalla & Thompson, 2012, p. 51).
In today’s globally competitive environment, organizations have to compete both
on lower cost and by providing better customer service, through alliances and
partnerships with competition, and from taking other agile strategies to survive
(Motiwalla & Thompson, 2012, p. 51).
Silos will prevent organizations to take advantage of supply chain management
and B2B e-commerce activity to introduce efficiencies in production and
procurement (Motiwalla & Thompson, 2012, p. 51).
Integrated systems are a critical and basic foundation for such other information
systems as customer resource management or sales force automation systems
(Motiwalla & Thompson, 2012, p. 51).
A long those lines information that is not accessible to customer service
representatives when they are interacting with customers can spoil the
relationships with the clients and have a negative impact on future sales.
Integrated systems are a critical and basic foundation for such other information
systems as customer resource management or sales force automation systems
(Motiwalla & Thompson, 2012, p. 51).
System integration has many hidden benefits
System integration has many challenges
Systems integration raises many new ethical issues
11
Assessment Task
I. True or False
1. Silos will prevent organizations to take advantage of supply chain management and
B2B e-commerce activity to introduce inefficiencies in production and procurement.
2. The increased availability of information enables managers and employees to make
informed decisions in a timely manner.
3. At the physical or technical level, systems integration means providing irregular
connectivity between heterogeneous application systems.
4. Leveling the competitive environment was a limitation of System Integration.
5. At the logical level, ERP systems require organizations to disperse on business
process rather than on functions.
6. In general, one of the biggest benefits is deduction in inventory and personnel costs
due to integrated systems.
7. Most IT organizations today support a Windows and flavor of UNIX.
II. Arrange the following statement then number it 1-8 to correct the arrangement of the steps
in system integration.
____________1. Check if they used various applications are such supporting standards as
JDBC/ODBC compliance for databases.
____________2. Centralization does not mean that they are all physically in one location.
____________3. Develop a policy in support of older legacy applications.
____________4. Middleware tools are essential for integration in the short term—if existing
applications must be used by the organization.
____________5. Develop organization standards and policy on acquisition of new
hardware and software which are aligned with organization IT strategy.
____________6. Take an inventory of the various hardware and software resources
focusing on vendors, operating systems platform.
____________7. A good backup and recovery system is essential if there is a system failure
or a major disaster.
____________8. Create a single sign-on policy for application and data access because all
employees and external partners will need access to an integrated system from anywhere,
anytime.
12
Summary
There are many drivers in organizations for needing integrated systems. The ability to
respond quickly to market conditions is a key part of protecting your customer base against
the incursions of a global set of hungry competitors. At the logical or human level, systems
integration means developing information systems that allow organizations to share data
with all of its stakeholders based on their need and authorization. On the other hand, at the
physical or technical level, systems integration means providing seamless connectivity
between heterogeneous application systems. Other parts of this module was: steps in
integrating system, benefits of system integration, limitation of system integration, ERP and
System Integration, ERP’s Role in both logical & physical Integration, and lastly implication
for management. (Motiwalla & Thompson, page 52, 2012)
Reference
Motiwalla L.F. & Thompson J. : “Enterprise System for Management Second Edition”:
Pearson Education, Upper Saddle River, New Jersey: 2012
13
MODULE 2
TYPES OF SYSTEMS INTEGRATION
Introduction
In this module we will focus on the notion of moving information between two
or more systems. This is the primary function of application integration. Although this
seems to be a straightforward theory, you may want to watch out for some new
concepts here including transformation, routing, and data analysis.
Learning Outcomes
14
Lesson 1. Information-Oriented Application Integration
This is a key concept for the future of the technology and also controls how we
deal with information movement and the invocation of local and remote application
services. They all roll up into a BPIOAI model that controls the application integration
domain, inside and outside the company (Linthium, 2003).
15
According to Linthium D. S. (2003) it is a complimentary form of application
integration to both IOAPI and Service-Oriented Application Integration (SOAI)—(even
Portal-Oriented Application Integration in some instances). BPIOAI provides a control
mechanism of sorts that defines and executes the movement of information and the
invocation of processes that span many systems. The goal is to abstract both the
encapsulated application services and application information into a single controlling
business process model (see Figure 3.1).
Figure 2.1 - BPIOAI introduces the notion of a common business process model that controls the
movement of information and the invocation of application services across many different systems,
both inter- and intra-company
Source: Linthium (2003)
16
BPIOAI Technology: A Deeper Dive
o Graphic modeling tool, where the business model is created and
behavior defined (Linthium, 2003).
o Business process engine that controls the execution of the
multistep business process and maintains state and the interactions
with the middleware, which in turn, interacts with any number of
source or target systems (Linthium, 2003).
o Business process monitoring interface that allows end users to
monitor and control the execution of a business process in real time
and optimize where needed (Linthium, 2003).
o Business process engine interface that allows other applications to
access the business process engine.
o Integration technology or application integration middleware (such
as an integration server or application server) that connects the
source and target systems to the BPIOIA technology (see Figure
3.2).
17
Figure 2.2 – The components of a typical business process integration solution.
Source: Linthium (2003)
18
Figure 2.3 - A composite application is the aggregation of many remote applications into a single
application.
Source: Linthium (2003)
19
Lesson 4. Service-Oriented Application Integration
20
Portals have become so common and so much has been written about them
that we will cover just the basic concepts here (Linthium, 2003).
The important point to remember in the context of application integration is that
portals have become the primary mechanism by which we accomplish
application integration. Whether that is good, bad, or indifferent doesn't really
matter. It is simply the way it is. Trading partners have extended the reach of
internal enterprise systems by utilizing the familiar Web browser interface.
(Linthium, 2003)
Assessment Task
Summary
The discussion was about the different types of application integration. Which
consist of Information-Oriented Application Integration, Business Process Integration-
21
Oriented Application Integration, Service-Oriented Application Integration, and Portal-
Oriented Application Integration. The function, benefits, features, and the advantages
of this integration have been discussed.
References
22
MODULE 3
ENTERPRISE RESOURCE PLANNING SYSTEMS
AND BUSINESS PROCESS MODELS
Introduction
According to Monk Ellen F. & Wagner Bret J (2013), in today’s competitive business
environment, companies try to provide customers with goods and services faster and less
expensively than their competition. How do they do that? Often, the key is an efficient,
integrated information system. Increasing the efficiency of information systems results in more
efficient management of business processes. When companies have efficient business
processes, they can be more competitive in the market place. An Enterprise Resource
Planning (ERP) system can help a company integrate its operations by serving as a company-
wide computing environment that includes a shared database—delivering consistent data
across all business functions in real time. (The term real time refers to data and processes
that are always current)
Learning Outcomes
1. Identify the factors that led to the development of Enterprise Resource Planning
(ERP) systems
2. Describe the distinguishing modular characteristics of ERP software
3. Summarize ongoing developments in ERP
23
Lesson 1. The Evolution of Information Systems
According to Monk & Wagner (2013) author of the book Concept in Enterprise Resource
Planning fourth edition, until recently, most companies had un-integrated information systems
that supported only the activities of individual business functional areas. Thus, a company
would have a marketing information system, a production information system, and so on—
each with its own hardware, software, and methods of processing data and information.
The hard economic times of the late 1980s and early 1990s caused many
companies to downsize and reorganize. These company overhauls were one
stimulus for ERP development (Monk & Wagner, 2013, p. 20).
In 1972, five former IBM systems analysts in Mannheim, Germany—Dietmar Hopp, Claus
Wellenreuther, Hasso Plattner, Klaus Tschira, and Hans-Werner Hector—formed
Systemanalyse und Programmentwicklung (Systems Analysis and Program Development),
or SAP—pronounced “S-A-P”. Later, the acronym was changed to Systeme, Anwendungen
und Produkte in der Datenverarbeitung (Systems, Applications and Products in Data
Processing.) (Monk & Wagner, 2013, p. 25-26).
24
SAP Begins Developing Software Modules
o Software modules are individual programs that can be purchased, installed,
and run separately, but all of the modules extract data from a common
database (Monk & Wagner, 2013, p. 26).
o To keep up with the ongoing development of mainframe computer
technology, in 1978 SAP began developing a more integrated version of its
software products, called the R/2system. In 1982, after four years of
development, SAP released its R/2 mainframe ERP software package
(Monk & Wagner, 2013, p. 26).
o Sales grew rapidly in the 1980s, and SAP extended its software’s
capabilities and expanded into international markets.
o By 1988, SAP had established subsidiaries in numerous foreign countries,
launched a joint venture with consulting company Arthur Andersen, and
sold its 1,000th system. SAP also became SAP AG, a publicly traded
company (Monk & Wagner, 2013, p. 26).
SAP R/3
o In 1988, SAP realized the potential of client-server hardware architecture
and began development of its R/3 system to take advantage of client-server
technology (Monk & Wagner, 2013, p. 26).
o The first version of SAP R/3 was released in 1992. Each subsequent
release of the SAP R/3 software contained new features and capabilities
(Monk & Wagner, 2013, p. 26).
o The client-server architecture used by SAP allowed R/3 to run on a variety
of computer platforms, including UNIX and Windows NT.
o The SAP R/3 system was also designed using an open architecture
approach (Monk & Wagner, 2013, p. 26).
o In open architecture, third-party software companies are encouraged to
develop add-on software products that can be integrated with existing
software (Monk & Wagner, 2013, p. 26-27).
New Directions in ERP
25
o In the late 1990s, the year 2000, or Y2K, problem motivated many
companies to move to ERP systems. As it became clear that the date
turnover from December 31, 1999, to January 1, 2000, could wreak havoc
on some information systems, companies searched for ways to consolidate
data, and ERP systems provided one solution. By 2000, SAP AG had
22,000 employees in 50 countries and 10 million users at 30,000
installations around the world. By that time, SAP also had competition in
the ERP market, namely from Oracle and PeopleSoft (Monk & Wagner,
2013, p. 27).
Oracle is now SAP’s biggest competitor. Oracle began in 1977 as
Software Development Laboratories (SDL).
SAP ERP software (previous versions were known as R/3, and
later, mySAP ERP) has changed over the years, owing to product
evolution—and for marketing purposes.
The latest versions of ERP systems by SAP and other
companies allow all business areas to access the same
database, as shown in Figure 4.1, eliminating redundant
data and communications lags.
26
SAP ERP Software Implementation
o A truly integrated information system requires integrating all functional
areas, but for various reasons, not all companies that adopt SAP software
use all of the SAP ERP modules (Monk & Wagner, 2013, p. 31).
o After a company chooses its major modules, it must make an incredible
number of decisions on how to configure the system. These configuration
options allow the company to customize the modules it has chosen to fit its
needs. For example, in the Financial Accounting (FI) module, a business
might need to define limits on the dollar value of business transactions that
certain employees can process (Monk & Wagner, 2013, p. 31).
o This is an important consideration in minimizing the risk of fraud and abuse,
and is just one example of the many decisions facing a company at the
start of a major ERP implementation (Monk & Wagner, 2013, p. 31).
27
they must still change some of the ways they work to fit the software (Monk
& Wagner, 2013, p. 33).
By 1998, most of the Fortune 500 companies had already installed ERP systems, so ERP
vendors refocused their marketing efforts on midsized companies (those with fewer than1,000
employees). Small and midsized companies represented a ripe, profitable market. To address
the needs of smaller companies, such as those with less than 500employees, SAP purchased
the ERP system of Israeli-based Top Manage Financial Systems in2002, and renamed this
package SAP Business One.
SAP and Oracle have significant competition for small-business customers from a number of
smaller ERP software companies such as Sage Business Solutions, Exact, Infor, and Epicor
(Monk & Wagner, 2013, p. 34).
Responses of the Software to the Changing Market
In the mid-1990s, many companies complained about the difficulty of implementing
the SAP R/3 system.
SAP continues to extend the capabilities of SAP ERP with additional, separate
products that run on separate hardware and that extract data from the SAP ERP
system.
The tools used to analyze data in the BW system are known as business
intelligence (BI).
The SAP ERP system provides some tools to manage customer interactions and
analyze the success of promotional campaigns, but SAP also sells a separate
application called Customer Relationship Management (CRM).
SAP addressed the issue of Internet-based data exchange with its Net Weaver
integration platform, which provides a unified means to connect SAP systems to
other systems and to the Internet.
Like all technology, ERP software and related products are constantly changing.
Thus, the challenge for a company is not only to evaluate an ERP vendor’s current
product offerings, but also to assess its development strategies and product plans.
(Monk & Wagner, 2013, p. 34-35).
28
Lesson 4. The Significance and Benefits of ERP software and
Systems
According to Monk & Wagner (2013), the significance of ERP lies in its many benefits. Recall
that integrated information systems can lead to more efficient business processes that cost
less than those in un-integrated systems. In addition, ERP systems offer the following benefits:
ERP allows easier global integration. Barriers of currency exchange rates, language,
and culture can be bridged automatically, so data can be integrated across
international borders (Monk & Wagner, page 36, 2013).
ERP integrates people and data while eliminating the need to update and repair many
separate computer systems. For example, at one point, Boeing had 450 data systems
that fed data into its production process; the company now has a single system for
recording production data (Monk & Wagner, page 36, 2013).
ERP allows management to actually manage operations, not just monitor them. For
example, without ERP, getting an answer to “How are we doing?” requires getting data
from each business unit and then analyzing that data for a comprehensive, integrated
picture. The ERP system already has all the data, allowing the manager to focus on
improving processes. This focus enhances management of the company as a whole,
and makes the organization more adaptable when change is required (Monk &
Wagner, page 36, 2013).
Assessment Task
I. Case Study
1. Assume you own and run a small local coffee shop. You do all your ordering
of ingredients for your coffee shop by hand—using pencil, paper, mail, and
telephone. All your sales are recorded by hand in a book, and transcribed for
filing of small-business taxes. How could a small ERP system help your
business become more efficient? What would an ERP system allow you to do?
29
II. True or False
1. Oracle is now SAP’s biggest competitor. Oracle began in 1975 as Software
Development Laboratories (SDL).
2. In the mid-1990s, many companies complained about the difficulty of
implementing the SAP R/2 system.
3. A truly integrated information system requires integrating all non-functional
areas.
4. The ERP system already has all the data, allowing the manager to focus on
improving processes.
5. The first practical business computers were the mainframe computers of the
1960s.
6. The tools used to analyze data in the BW system are known as business
intelligence (BI).
7. In 1972, five former IBM systems analysts in Manheim, Germany—Dietmer
Hopp, Claus Wellenreuther, Haso Plattner, Klaus Tschira, and Hans-Werner
Hector—formed Systemanalyse und Programmentwicklung (Systems Analysis
and Program Development), or SAP—pronounced “S-A-P”.
8. The original SAP ERP system targeted very large companies.
9. These company overhauls were one stimulus for ERP development.
10. The first version of SAP R/3 was released in 1990.
Summary
30
Increasingly sophisticated software facilitated integration, especially in two
areas: Accounting and Finance and material requirements planning.
As businesses grew in size, and the business environment became more
complex and competitive, business managers began to demand more efficient
and competitive information systems.
SAP AG produced a complex, modular ERP program called R/3. The software
could integrate a company’s entire business by using a common database that
linked all operations, allowing real-time data sharing and streamlined
operations.
SAP R/3, now called SAP ERP, is modular software offering modules for Sales
and Distribution, Materials Management, Production Planning, Quality
Management, and other areas.
ERP software is expensive to purchase and time consuming to implement, and
it requires significant employee training—but the payoffs can be spectacular.
For some companies, however, the ROI may not be immediate or even
calculable.
Experts anticipate that ERP’s future focus will be on applications for mobile
devices and providing instant access to large volumes of data.
References
Monk Ellen F. & Wagner Bret J: “Concepts in Enterprise Resource Planning Fourth
Edition”: United States of America: 2013 page 19
31
MODULE 4
DESIGNING SYSTEMS INTEGRATION
SOLUTIONS AND ENTERPRISE INTEGRATION
PATTERNS
Introduction
This module will illustrate how the patterns in this book can be used to solve a
variety of integration problems. In order to do so, we examine common integration scenarios
and present a comprehensive integration example. As we design the solution to this example,
we will express the solution using the patterns contained in this module.
Learning Outcomes
32
Lesson 1. The Need for Integration
According to Hohpe & Woolf (2003), enterprises are typically comprised of hundreds if not
thousands of applications that are custom-built, acquired from a third-party, part of a legacy
system, or a combination thereof, operating in multiple tiers of different operating system
platforms. It is not uncommon to find an enterprise that has 30 different Websites, three
instances of SAP and countless departmental solutions (pg. 31).
We may be tempted to ask: How do businesses allow themselves to get into such a mess?
Shouldn’t any CIO of such an enterprise spaghetti architecture be fired? Well, like in most
cases things happen for a reason (Hohpe & Woolf, 2003, pg. 31).
First of all, writing business applications is hard. Creating a single, big application to run a
complete business is next to impossible. The ERP vendors have had some success at
creating larger-than-ever business applications. The reality, though, is that even the
heavyweights like SAP, Oracle, Peoplesoft and the like only perform a fraction of the business
functions required in a typical enterprise. We can see this easily by the fact that ERP systems
are one of the most popular integration points in today’s enterprises (Hohpe & Woolf, 2003,
pg. 31).
Second, spreading business functions across multiple applications provides the business with
the flexibility to select the “best” accounting package, the “best” customer relationship
management or the order processing system that best suits the business’ needs. One-stop-
shopping for enterprise applications is usually not what IT organizations are interested in, nor
is possible given the number individual business requirements. Vendors have learned to cater
to this preference and offer focused applications around a specific core function (Hohpe &
Woolf, 2003, pg. 31).
However, the ever-present urge to add new functionality to existing software packages has
caused some functionality spillover amongst packaged business applications. For example,
many billing systems started to incorporate customer care and accounting functionality.
Likewise, the customer care software maker takes a stab at implementing simple billing
functions such as disputes or adjustments. Defining a clear functional separation between
33
systems is hard: is a customer disputing a bill considered a customer care or a billing function?
(Hohpe & Woolf, 2003, pg. 31).
Users such as customers, business partners and internal users do generally not think about
system boundaries when they interact with a business. They execute business functions,
regardless of the how many internal systems the business function cuts across. For example,
a customer may call to change his or her address and see whether the last payment was
received. In many enterprises, this simple request can span across the customer care and
billing systems. Likewise, a customer placing a new order may require the coordination of
many systems. The business needs to validate the customer ID, verify the customer’s good
standing, check inventory, fulfill the order, get a shipping quote, compute sales tax, send a
bill, etc. This process can easily span across five or six different systems. From the customer’s
perspective, it is a single business transaction (Hohpe & Woolf, 2003, pg. 32).
Anyone who has been through an EAI deployment can attest to the fact that EAI solutions are
a critical component of today’s enterprise strategies, but make IT life harder, not easier. It’s a
long way between the high-level vision of the integrated enterprise (defined by terms such as
“Straight-Through-Processing”, “T+1”, “Agile Enterprise”) and the nuts-and-bolts
implementations (what parameters did System.Messaging.XmlMessageFormatter take
again?) (Hohpe & Woolf, 2003, pg. 33).
35
Lesson 3. How Integration Patterns Can Help
There are no simple answers for enterprise integration. In our opinion, anyone who claims
that integration is easy must be either incredibly smart (or at least a good bit smarter than the
rest of us), incredibly ignorant (OK, let’s say optimistic), or they have a financial interest in
making you believe that integration is easy (Hohpe & Woolf, 2003, pg. 33).
Even though integration is a broad and difficult topic, we can always observer some people
who are much better at it than others. What do these people know that others don’t? Since
there is no such thing as “Teach Yourself Integration in 21 Days” (this book sure ain't!) it is
unlikely that these people know all the answers to integration. However, these people have
usually solved enough integration problems that they can compare new problems to prior
problems they have solved. They know the “patterns” of problems and associated solutions.
They learned these patterns over time by trial-and-error or from other experienced integration
architects (Hohpe & Woolf, 2003, pg. 34).
The “patterns” are not copy-paste code samples or shrink-wrap components, but rather
nuggets of advice that describe solutions to frequently recurring problems. Used properly, the
integration patterns can help fill the wide gap between the high-level vision of integration and
the actual system implementation (Hohpe & Woolf, 2003, pg. 34).
According to Hohpe & Woolf (2003), We intentionally left the definition of “integration” very
broad. To us it means connecting computer systems, companies or people. While this broad
definition gives us the convenience of sticking whatever we find interesting into this book, it is
helpful to have a closer look at some of the most common integration scenarios. Helping
clients design and implement integration solutions, we repeatedly came across the following
six types of integration projects, which can be found on the next page (pg. 34):
36
Information Portals
Data Replication
Shared Business Functions
Service-Oriented Architectures
Distributed Business Processes
Business-to-Business Integration
This list is by no means a complete taxonomy of all things integration but it does help to
illustrate the kind of solutions that integration architects build. Many integration projects
consist of a combination of multiple types of integration. For example, reference data
replication is often required in order to tie applications into a single distributed business
process (Hohpe & Woolf, 2003, pg. 34).
Information Portal
Many business users have to access more than one system to answer a specific
question or to perform a single business function. For example, to verify the status of
an order, a customer service representative may have to access the order
management system on the mainframe plus log on to the system that manages orders
placed over the Web. Information portals aggregate information from multiple sources
into a single display to avoid having the user access multiple systems for information.
37
Simple information portals divide the screen into multiple zones, each of which
displays information from a different system. More sophisticated systems provide
limited interaction between zones, for example when a user selects an item from a list
in zone A, zone B refreshes with detailed information about the selected item. Other
portals provide even more sophisticated user interaction and blur the line between a
portal and an integrated application (Hohpe & Woolf, 2003, pg. 34).
Data Replication
According to Hohpe & Woolf (2003), Many business systems require access to the
same data. For example, a customer’s address may be used in the customer care
system (when the customer calls to change it), the accounting system (to compute
sales tax), the shipping system (to label the shipment) and the billing system (to send
an invoice). Many of these systems are going to have their own data stores to store
customer related information. When a customer calls to change his or her address all
these systems need to change their copy of the customer’s address. This can be
accomplished by implementing an integration strategy based on data replication (pg.
35).
38
There are many different ways to implement data replication. For example, some
database vendors build replication functions into the database, we can export data into
files and re-import them into the other system, or we can use message-oriented
middleware to transport data records inside messages (Hohpe & Woolf, 2003, pg. 35).
In the same way that many business applications store redundant data, they also tend
to implement redundant functionality. Multiple systems may need to check whether a
social-security number is valid, whether the address matches the specified postal code
or whether a particular item is in stock. It makes business sense to expose these
functions as a shared business function that is implemented once and available as a
service to other systems (Hohpe & Woolf, 2003, pg. 36).
A shared business function can address some of the same needs as data replication.
For example, we could implement a business function called ‘Get Customer Address’
that could allow other systems to request the customer’s address when it is needed
rather than always storing a redundant copy. The decision between these two
39
approaches is driven by a number of criteria, such as the amount of control we have
over the systems (calling a shared function is usually more intrusive than loading data
into the database) or the rate of change (an address may be needed frequently but
change very infrequently) (Hohpe & Woolf, 2003, pg. 36).
Service-Oriented Architecture
40
integration between the two applications. On the other hand a service-oriented
architecture usually provides tools that make calling an external service almost as
simple as calling a local method (performance considerations aside). Because all
services are available in a consistent manner, SOAs are sometimes referred to as
“service bus architectures” (Hohpe & Woolf, 2003, pg. 36).
One of the key drivers of integration is the fact that a single business transaction is
often spread across many different systems. A previous example showed us that a
simple business function such as “place order” can easily touch six or seven systems.
In most cases, all relevant functions are incorporated inside existing applications. What
is missing is the coordination between the applications. Therefore, we can add a
business process management component that manages the execution of a business
function across multiple existing systems (Hohpe & Woolf, 2003, pg. 37).
41
Business-to-Business Integration
So far we have mainly considered the interaction between applications and business
functions inside an enterprise. In many cases, business functions may be available
from outside suppliers or business partners. For example, the shipping company may
provide a service for customers to compute shipping cost or track shipments. Or a
business may use an outside provider to compute sales tax rates. Likewise, integration
frequently occurs between business partners. A customer may contact a retailer to
inquire on the price and the availability of an item. In response, the retailer may ask
the supplier for the status of an expected shipment that contains the out-of-stock item
(Hohpe & Woolf, 2003, pg. 37).
One of the biggest buzz words in enterprise architecture and integration is the notion of loose
coupling. It is in fact such a popular term that Doug Kaye wrote a whole book titled after this
ubiquitous concept [Kaye]. The benefits of loose coupling have been know for quite some time
now, but they have taken center stage more recently due to the surging popularity of Web
services architectures (Hohpe & Woolf, 2003, pg. 37-38).
The core principle behind loose coupling is to reduce the assumptions two parties
(components, applications, services, programs, users) make about each other when they
exchange information. The more assumptions two parties make about each other and the
42
common protocol, the more efficient the communication can be, but the less tolerant the
solution is of interruptions or changes because the parties are tightly coupled to each other
(Hohpe & Woolf, 2003, pg. 38).
A great example of tight coupling is a local method invocation. Invoking a local method inside
an application is based on a lot of assumptions between the called and the calling routine.
Both methods have to run in the same process (e.g. a virtual machine) and be written in the
same language (or at least use a common intermediate language or byte code). The calling
method has to pass the exact number of expected parameters, each using the correct type.
The call is immediate, i.e. the called method starts processing immediately after the calling
method makes the call. Meanwhile, the calling method will only resume processing when the
called method completes (meaning the invocation is synchronous). Processing will
automatically resume in the calling method with the next statement after the method call. The
communication between the methods is immediate and instantaneous, so neither the caller
nor the called method have to worry about security in the form of eavesdropping 3rd parties.
All these assumptions make it very easy to write well structured applications that break
functionality into individual methods to be called by other methods. A large number of small
method allow for flexibility and reuse (Hohpe & Woolf, 2003, pg. 38).
43
The challenge that all these approaches face lies in the fact that remote communication
invalidates many of the assumptions that a local method call is based on. As a result,
abstracting the remote communication into the simple semantics of a method call can be
confusing and misleading. Waldo et al. reminded us back in 1994 that "objects that interact in
a distributed system need to be dealt with in ways that are intrinsically different from objects
that interact in a single address space" [Waldo]. For example, if we call a remote service to
perform a function for us, do we really want to restrict ourselves to only those services that
were built using the same programming language as we do? A call across the network also
tends to be multiple orders of magnitude slower than a local call. Should the calling method
really wait until the called method completes? What if the network is interrupted and the called
method is temporarily unreachable? How long should we wait? How can we be sure we
communicate with the intended party and not a 3rd party “spoofer”? How can we protect
against eavesdropping? What if the method signature (the list of expected parameters) of the
called method changes? If the remote method is maintained by a third party or a business
partner we no longer have control over such changes. Should we have our method invocation
fail or should we attempt to find the best possible mapping between the parameters and still
make the call? It becomes quickly apparent that remote integration brings up a lot of issues
that a local method call never had to deal with (Hohpe & Woolf, 2003, pg. 38-39).
To show the effects of tightly coupled dependencies and how to resolve them, let’s look at
different options of connecting two systems. Let’s assume we are building an on-line banking
system that allows customers to deposit money into their account from another bank. To
perform this function, the front-end Web application has to be integrated with the back-end
financial system that manages fund transfers (Hohpe & Woolf, 2003, pg. 39).
The easiest way to connect the two systems is through the TCP/IP protocol. Every self-
respecting operating system or programming library created in the last 15 years is certain to
include a TCP/IP stack. TCP/IP is the ubiquitous communications protocol that transports data
44
between the millions of computers connected to the Internet and local networks. Why not use
the most ubiquitous of all network protocols to communicate between two applications?
(Hohpe & Woolf, 2003, pg. 39).
Let’s assume that the remote function that deposits money into a person’s account takes only
the person’s name and the Dollar amount as arguments. The following few lines of code then
suffice to call such a function over TCP/IP (we chose C#, but this code would look virtually
identical in C or Java) (Hohpe & Woolf, 2003, pg. 39).
socket.Close();
This code opens a socket connection to the address www.eaipatterns.com and sends two
data items (the amount and the customer’s name) across the network. No expensive
middleware is required, no EAI tools, RPC toolkits, just 10 lines of code. When we run this
code it tells us: “7 bytes sent”. Voila! How can integration be so difficult? (Hohpe & Woolf,
2003, pg. 40).
There are a couple of major problems with this integration attempt. One of the strengths of
the TCP/IP protocol is its wide support so that we can connect to pretty much any computer
connected to the network regardless of the operating system or programming language it
uses. However, the platform independence works only for very simple messages: byte
streams. In order to convert our data into a byte stream we used the BitConverter class. This
45
class converts any data type into a byte array, using the internal memory representation of
the data type. The catch is that the internal representation of an integer number varies with
computer systems. For example, .NET uses a 32 bit integer while other systems may use a
64 bit representation. Our example transfers 4 bytes across the network to represent a 32 bit
integer number. A system using 64 bits would be inclined to read 8 bytes off the network and
would end up interpreting the whole message (including the customer name) as a single
number (Hohpe & Woolf, 2003, pg. 40).
Also, some computer systems store their numbers in big-endian format while others store
them in little-endian format. A big-endian format stores numbers starting with the highest byte
first while little-endian systems store the lowest byte first. PCs operate on a little-endian
scheme so that the code passes the following 4 bytes across the network (Hohpe & Woolf,
2003, pg. 40):
232 3 0 0
232 + 3 * 2^8 equals 1000. A system that uses big-endian numbers would consider this
message to mean 232* 2^24 + 3 * 2^16 = 3,892,510,720. Joe will be a very rich man! So this
approach works only under the assumption that all connected computers represent numbers
in the same internal format (Hohpe & Woolf, 2003, pg. 40).
The second problem with this simple approach is that we specify the location of the remote
machine (in our case www.eaipatterns.com). The Dynamic Naming Service (DNS) gives us
one level of indirection between the domain name and the IP address, but what if we want to
move the function to a different computer on a different domain? What if the machine fails and
we have to setup another machine? What if we want to send the information to more than one
machine? For each scenario we would have to change the code. If we use a lot of remote
functions this could become very tedious. So we should find a way to make our communication
independent from a specific machine on the network (Hohpe & Woolf, 2003, pg. 40).
Our simple TCP/IP example also establishes temporal dependencies between the two
machines. TCP/IP is a connection-oriented protocol. Before any data can be transferred, a
connection has to be established first. Establishing a TCP connection involves IP packets
traveling back and forth between sender and receiver. This requires that both machines and
46
the network are all available at the same time. If any of the three pieces is malfunctioning or
not available due to high load, the data cannot be sent (Hohpe & Woolf, 2003, pg. 40-41).
Lastly, the simple communication also relies on a very strict data format. We are sending 4
bytes of amount data and then a sequence of characters that define the customer’s account.
If we want to insert a third parameter, e.g. the name of the currency, we would have to modify
both sender and receiver to use the new data format (Hohpe & Woolf, 2003, pg. 41).
In summary, our minimalist integration solution is fast and cheap, but it results in a very brittle
solution because the two participating parties make the following assumptions about each
other (Hohpe & Woolf, 2003, pg. 41):
47
Lesson 7. A loosely Coupled Integration Solution
In order to connect two systems via an integration solution, a number of things have to happen.
These things make up what we call middleware – the things that sit between applications
(Hohpe & Woolf, 2003, pg. 42).
Invariably, some data has to be transported from one application to the next. This data could
be an address record that needs to be replicated, a call to a remote service or a snippet of
HTML headed for a portal display. Regardless of the payload, this piece of data needs to be
understood by both ends and needs to be transported, usually across a network. Two
elements provide this basic function. We need a communications channel that can move
information from one application to the other. This channel could be a series of TCP/IP
connections, a shared file, a shared database or a floppy disk being carried from one computer
to the next (the infamous ‘sneakernet’). Inside this channel we place a message – a snippet
of data that has an agreed-upon meaning to both applications that are to be integrated. This
piece of data can be very small, such as the phone number of a single customer that has
changed, or very large, such as the complete list of all customers and their associated
addresses. We call this piece of data a message (Hohpe & Woolf, 2003, pg. 42).
48
For example, one data format may store the customer name in two fields, called FIRST_NAME
and LAST_NAME, while the other system may use a single field called Customer_Name.
Likewise, one system may support multiple customer addresses while the other system only
supports a single address. Because the internal data format of an application can often not
be changed the middleware needs to provide some mechanism to convert one application’s
data format in the other’s. We call this step translation (Hohpe & Woolf, 2003, pg. 42).
So far we can send data from one system to another and accommodate differences in data
formats. What happens if we integrate more than two systems? Where does the data have to
be moved? We could expect each application to specify the target system(s) for the data it is
sending over the channel. For example, if the customer address changes in the customer care
system we could make that system responsible for sending the data to all other systems that
store copies of the customer address. As the number of systems increases this becomes very
tedious and requires the sending system to have knowledge about all other systems. Every
time a new system is added, the customer care system would have to be adjusted to the new
environment. Things would be a lot easier of the middleware could take care of sending
messages to the correct places. This is the role of a routing component such as a message
broker (Hohpe & Woolf, 2003, pg. 43).
Integration solutions can quickly become complex because they deal with multiple
applications, data formats, channels, routing and transformation. All these elements may be
spread across multiple operating platforms and geographic locations. In order to have any
idea what is going on inside the system we need a systems management function. This
subsystem monitors the flow of data, makes sure that all applications and components are
available and reports error conditions to a central location (Hohpe & Woolf, 2003, pg. 43).
Our integration solution is now almost complete. We can move data from one system from
another, accommodate differences in the data format, route the data to the required systems
and monitor the performance of the solution. So far we assumed that an application sends
data as a message to the channel. However, most packaged and legacy applications and
many custom applications are not prepared to participate in an integration solution. We need
a message endpoint to connect the system explicitly to the integration solution. The endpoint
49
can be a special piece of code or a Channel Adapter provided by an integration software
vendor (Hohpe & Woolf, 2003, pg. 43-44).
For this example, we assume that the solution needs to support the following requirements.
Naturally, we simplified the requirements a bit for sake of brevity, but nevertheless these
types of requirements occur frequently in real businesses (Hohpe & Woolf, 2003, pg. 44).
Take Orders: Customers can place orders via Web, phone or fax
Process Orders: Processing an order involves multiple steps, including verifying
inventory, shipping the goods and invoicing the customer
50
Check Status: Customers can check the order status
Change Address: Customers can use a Web front-end to change their billing and
shipping address
New Catalog: The suppliers update their catalog periodically. WGRUS needs to
update its pricing and availability based in the new catalogs.
Announcements: Customers can subscribe to selective announcements from
WGRUS.
Testing and Monitoring: The operations staff needs to be able to monitor all individual
components and the message flow between them.
Assessment Task
51
3. A failing or misbehaving integration solution can cost a business __________of
Dollars in lost orders, misrouted payments and disgruntled customers.
a. Billion
b. Hundreds
c. Millions
d. Thousands
4. One important constraint of developing integration solutions is the limited amount
of control the integration developers typically have over the participating
applications. In most cases, the applications are “_________” systems or
packaged applications that cannot be changed just to be connected to an
integration solution.
a. Inheritance
b. Integration
c. Legality
d. Legacy
5. Developing an ______ solution is challenging in itself, operating and maintaining
such a solution can be even more daunting.
a. EAI
b. CRM
c. SAP
d. Oracle
6. In Integration Pattern, the “_________” are not copy-paste code samples or
shrink-wrap components, but rather nuggets of advice that describe solutions to
frequently recurring problems.
a. Sequence
b. Flow
c. Patterns
d. Trial
7. One of the biggest buzz words in enterprise architecture and integration is the
notion of ________.
a. integration coupling
b. cable coupling
c. loose coupling
52
d. enterprise coupling
Summary
In summary, in order to support common business processes and data sharing across
applications, these applications need to be integrated. Application integration needs to provide
efficient, reliable and secure data exchange between multiple enterprise applications (Hohpe
& Woolf, 2003, pg. 32).
The Wide World of Integration many of the above considerations apply equally to
business-to-business integration. However, communicating across the Internet or some other
53
network usually raises new issues related to transport protocols and security. Also, since
many business partners may collaborate in an electronic “conversation” standardized data
formats are critically important (Hohpe & Woolf, 2003, pg. 37).
References
Hohpe, Gregor & Woolf, Bobby: “Enterprise Integration Patterns: Designing, Building,
and Deploying Messaging Solutions”: A Martin Fowler Signature Book: 2003 page 31
54