12 Steps to Managing EAI Projects

virtualtechpartners
rmayers@virtualtechpartners.net
 Overview
Many technology services organizations have now implemented Enterprise Application
Integration methodologies or are just now scratching the surface. In many cases
organizations find out the hard way that the success of an EAI project has inherited risks
that are unique to EAI projects. Therefore normal project management methodologies
(while they can and should be applied) are not complete if one does not perform the
proper planning activities to ensure the success of the EAI project.

This presentation will cover some of the most important steps to ensure the success of your
EAI project. These are not the only steps one should consider but they are the most
practical steps in approaching EAI.

The 12 step strategy is taken from best practices and follows the “Keep it Simple” rule, as
the more complex one makes the EAI domain the more difficult it will be to manage and
deliver your project.

Familiar concepts and terms such as metadata, schemas, object oriented modeling, analysis
and design (OOA & OOD), patterns and requirement gathering techniques are all key to
the success of your project. Assigning a project manager with good EAI technical and/or
architecture knowledge is also a good thing to have for the success of your project from
a risk mitigation perspective (Engage an EAI Project Manager).
Step 1-Understanding your problem domain
Clearly define the problem domain from an end-to-end business perspective;
• Identify and partner with your Business Owner/s (BO) and Subject Matter Expert/s (SME)
• Document how they do business; what information systems do they use, business
processes, information content, interfaces and business requirements for the new
system.
• Identify and partner with your Technology Services and Technology Support SME’s
• Document existing integration patterns of the existing system
• Does the Enterprise have Architecture Standards and/or Products in place?
• Utilization of the Unified Modeling Language (UML) to document your requirements
is highly recommended

This step is a basic requirements gathering process. It demands you to engage with the three P’s
(below) to identify and document the information that will define the EAI problem domain
correctly so it can then be analyzed, modeled and re-engineered
1) People
2) Paper
3) Processes
Step 2-Make sense of the Data
Since all EAI projects exist merely for movement of data, this reality alone justifies your
quest to understand what exists in the databases which may also be scattered
throughout the enterprise.

Ultimately, the implementation of any data-level EAI project comes down to understanding
where the data exists, gathering information about the data (i.e. Database Schema
and/or Data Dictionary) and applying business principles and/or rules to determine
which and what data flows where, why and when?

You must document & verify the enterprise metadata required for the success of your EAI
project.
Step 3-Make sense of the Processes
Once you understand the enterprise metadata, you have created a baseline for developing
your project level metadata model, however you must also decide how to approach the
enterprise business model for your EAI project.

This decision will depend of several things. First, there is your (potentially limited) view of
the enterprise at the process or method level. Your success here will depend on how
well you understand and document the business processes and the way they relate to
each other. Verification with your business partners is highly recommended.

Traditional process modeling techniques such as UML are the best way to document
business processes in a way that everyone can understand (your Business & Technical
partners). Work from existing and/or known business process in an iterative fashion to
better understand the end-to-end business processes; what they do, the sequence of
events, are they technology or people processes or a combination of both?

Verify assumptions, the only bad question is the one you do not ask…
Step 4-Identify Business Events
When something happens (a business event), then there is a resulting reaction. For example
when someone applies for a credit application online, this represents an event. This
event will trigger something else to happen such as executing a real-time credit check
with a consumer credit agency for his/her credit rating, The returned credit rating will
invoke other events such as approval or rejection processes to occur. These events are
usually asynchronous in nature, however they can also be synchronous as well.

In attempting to understand your EAI problem domain your must strive to capture the
business events that occur within the domain. It is important to know what invoked a
business event, what takes place during the event and any other up or down stream
events that your system may be dependant upon. If a business event is dependant on
manual intervention – remember a true EAI solution should look to eliminate manual
intervention entirely (where applicable). Therefore you must understand the business
events within your EAI problem domain to analyze where and how an EAI strategy
will help.
Step 5-Identify all application interfaces
Application interfaces can be challenging as they will differ from system to system. What
may be called an interface in one system may not e deemed an interface by another,
what’s more complicating is what may be referred to as an application interface by an
application support team may not truly be an application interface at all. So the time
you spend validating assumptions about application interfaces will be well worth it.

The best way to begin with interfaces is to catalog all application interfaces into a directory
for the project. As with any other repository of data you will want to store high level
data elements about your interfaces; Source System & target System, XML schema or
file formats used, communication protocols, is publish or subscribe messaging utilized,
messaging middleware, XML over HTTP, Web Services, etc.

This Interface Directory along with the Enterprise Metadata captured from step 2 along with
the Business Process Model defined in Step 3 will help you to understand the points of
integration within all systems that ultimately comprise of the EAI problem domain for
your project.
Step 6-Identify data movement
As part of your interface directory, you will want to map the movement of data from system
to system. Mapping what data element or elements are moving from what source
system to what target system where it will ultimately be stored, further processed or
both.

You will want to document where the data is physically located, what security may be
present, what enabling technology exists and how the information is extracted from the
source system and method used to push or pull the data to the target system. Is this an
event driven process or if no event is required what condition causes the movement of
the data? Is it a batch or real-time process or simply when the state of the data changes
(i.e. data replication).

Are the systems loosely or physically coupled? Physically couples means both systems must
know about each other and if the data formats passed is changed on either side this will
trigger change on the other side. Loosely coupled means the exact opposite, the
systems do not need to know about each other and there is some type of data
transformation logic or product (i.e. middleware) component managing the exchange
of data between the two systems.
Step 7-Identify data transformation scenarios
This leads us to Data Transformation scenarios, once you know the data movement you will
want to also document in your interface directory the rules for data transformation.

This is especially important for several reasons which was touched on in Step 5.
1) The data in one system may not make sense to another system until the data
schema and format are reformatted to be compatible with the target system.
2) The EAI strategy and solution must take into account the transformation
requirements as well as data formats required by the source and target systems.
Step 8-Apply Technology
There are a wide range of choices of technology toolkits to use within the EAI space
including but not limited to application servers, distributed objects, message brokers,
business process management engines, etc.

The choice of technology will likely be a mix of products and vendors or the Enterprise
Architecture team may already have standards on which vendors and or technologies
are acceptable for utilization.

Technology selection requires a great deal of time and effort, however the requirements you
have now compiled in the previous steps will help you to determine the right mix of
products and vendors to meet your EAI problem domain.

The time it takes to select your technology could take as long as the development of the EAI
solution itself, however consider the consequences of choosing the wrong technology
that is not in line with the Enterprise Strategy for EAI solutions. This wrong decision
will ultimately result in the failure of the EAI project itself.
Step 9-Test, Test, Test
Testing is expensive, time consuming and usually a thankless job. Still if an EAI solution is
not tested properly , then disaster looms large. For many EAI toolkits available today,
the EAI solution tends to be a black box where all the EAI processing occur with no
real sense of tracking once the black box is invoked. Data comes in from the source
system and goes out to the target system and life is good.

Well this is where negative testing is very critical, the described scenario above is typically
called positive testing, it’s what happens when all in the world is good and everything
executes as expected. Negative test cases are developed to see what happens when the
data received and/or the systems behavior is not as expected.

Develop a test plan that incorporates all positive and negative test case scenarios. Most EAI
solutions are mission critical and therefore cannot be taken offline. As result the better
your test plan is to thoroughly perform all positive and negative testing, your new EAI
solution will be resilient when handling error conditions and reliable when processing
valid transactions as expected.
Step 10-Consider performance
The EAI solution must run efficiently with better than average response time in performing
end-to-end transactions. If the EAI solution runs less efficiently than the old system it
replaced - it will be deemed a failure.

While much of the system performance is tied to the Infrastructure the EAI solutions runs on.
The choice of EAI enabling product needs to be equally matched with infrastructure
components with the processing power capable of handling the expected load of the
system.

Therefore, Capacity Planning is a very important planning activity that must be done with the
right involvement from the right people within the enterprise. As well, load balancing
techniques which provide high availability and/or higher system throughput must also
be considered as part of the Capacity Planning activities.
Step 11-Define the value
OK – the difficult part is over. You must now try to determine the business value of
integration the systems in question. The 2 things that must be considered are the hard
dollars cost save and the soft dollar savings.

The hard dollar savings should be easy to identify, such as the value of having eliminated
manual processes, labor cost reduction, reducing error rates, processing transaction
more efficiently, etc.

However, the soft dollar savings may be harder to define. These would potentially be
increased productivity by the Business Unit, ease of use of the system which fosters
customer satisfaction and/or the ability of the two disparate Business Units to
implement change within their respective systems without effecting their Business
Partner up or down stream.
Step 12-Monitoring and Maintenance Procedures

Lastly, you will want to define how system health monitoring will be accomplished. Many
EAI products lack having good monitoring tools as part of their EAI toolkit. You may
need to look at what monitoring tools the company already has as enterprise monitoring
tools first to see if the ones already in usage will help monitor your new EAI solution.
Otherwise you may need to look at other monitoring tools and source one from another
vendor.

Maintenance procedures must also be defined, document all of the maintenance procedures
that need to occur. For instance, who will administer the message broker server,
middleware servers, application servers and database servers, etc. Who will manage
security, who will monitor system performance, what will your problem notification
call list look like, who is the primary and secondary contacts, etc.

What are your disaster recovery requirements for the applications in question? This may
incur additional hardware being procured…
 12 Steps to Managing EAI Projects
In Summary
1) Understand the Problem Domain
2) Make sense of the Data
3) Make sense of the Processes
4) Identify Business Events
5) Identify all Application Interfaces
6) Identify Data Movement
7) Identify Data Transformation scenarios
8) Apply Technology
9) Test, Test, Test
10) Consider Performance
11) Define the Value
12) Monitoring and Maintenance procedures
Some top 10 reasons why EAI projects fail
1) The EAI implementation is more complex and expensive then
expected
2) The Business Unit does not communicate and/or cooperate in
drafting/documenting the end-to-end Business Processes
3) There are no standardized methodologies in place and/or a lack in
competencies when designing EAI blue prints
4) Entitlements are often not considered for initial implementations
5) EAI implementations differ greatly from application implementations
6) Un-Normalized Data Models and Schema inconsistencies between
applications complicate communication and/or data mapping
7) Transactional and Data integrity issues not identified in testing
8) Poor performance of the EAI solution post production
implementation
9) Poor end-to-end system management and monitoring
10) Performance testing tools for load testing and profiling tools do not
exist