An Approach to Identify and Extract SOA Services from Legacy iSeries Applications

***Rama Krishna Thommandra, **Hari Prasad Chinta, *Nakul Sharma, *Sateesh Ranjan ***Senior Project Manager, **Senior Software Engineer, *Software Engineers (HCL TECHNOLOGIES Ltd)-Email {ramakrishnaTM, hariprasad.chinta, nakul_sharma, sateesh.ranjan}@hcl.in

Abstract
Making legacy applications work in SOA architecture is a definite boon to the enterprise. However before embarking on SOA-enablement, a very important question that needs to be answered is: Where should we look - within the legacy applications - when searching for services that can be re-used? Though there is a lot of information available on what SOA is and how legacy can benefit from SOA, there has not been a generic approach specified on EXACTLY WHERE in the legacy applications one can look when attempting to extract services. The paper is a listing of all the legacy application artifacts that need to be considered while extracting services from monolithic legacy applications.

enterprise. This value of the legacy systems arises because these systems encapsulate substantial knowledge of the business and significant business functionality in the language and tools that current staff understands. However, these legacy systems are also the cause of pain points as listed in Figure 1.

Figure 1: Pain Points of Legacy systems
• Application Deficiencies o o o o o Undocumented Application Tightly-coupled Modules Monolithic non-modular architecture Lack of a consistent underlying architecture (Redundant code, functionality and data) Closed and outdated technology that is difficult to integrate and interface with new, open technologies and modern distributed architectures. UI limitations (Inefficient re-entry, navigation, etc.)

Goal and Scope of the Paper
This paper attempts to list some of the signposts in existing legacy applications that can be used to identify and extract services from such systems so as to render the monolithic legacy application into multiple useful Business Services. This paper addresses this goal solely from the perspective of functions typically found in a business application.

o

• Side-Effects of Operating Legacy Applications o o o High cost of ownership Low Agility Knowledge of these systems is usually restricted to a core set of people who are difficult to replace Limited and difficult integration of functionality Limited workflow support Difficult to exploit new technologies Single-platform limitations and risk

1. Introduction
Overview of Legacy Systems
Most modern analysis tends to classify applications as "Legacy" depending on their age – say, typically, twentyodd years – and/or usage of a relatively old platform, language or technology. Typically, these are applications and data that have been inherited from languages, platforms, and techniques earlier than current technology. However, even today these legacy investments are running critical business processes. In fact, Legacy systems are often considered to be a "cash cow" for the

o o o o

• Changing Realities o o Shrinking talent pool of developers skilled in legacy systems and decreasing vendor support. Difficultly training new end users and developers

Of all the points mentioned above, the biggest difficulty is making Legacy systems interface with new, open technologies and modern distributed architectures.

Overview of Service Oriented Architecture
In today’s world, this new, open and distributed architecture is best exemplified by Service Oriented Architecture (SOA). Undoubtedly it is possible to re-use legacy assets in a more agile alternative without using services or without using SOA, but our proposition is that by making them as services on a SOA architecture we can create more flexibility for utilization of such services from unexpected quarters in the future, in effect creating a more agile application which, in turn, would facilitate the creation of an agile enterprise. A detailed exposition on the various facets of SOA is beyond the scope of this article. However, we feel a recap of the same serves the purpose of setting the stage for what to keep in mind when exploring legacy applications for probable services. With that intention we provide below a very brief outline about SOA. SOA separates functions into distinct units (services), which can be distributed over a network and can be combined and reused to create business applications. OASIS (organization) defines service as "a mechanism to enable access to one or more capabilities, where the access is provided using a prescribed interface and is exercised consistent with constraints and policies as specified by the service description." The commonly agreed aspects of the definition of a service in SOA are: • • Services are defined by explicit, implementationindependent interfaces Services are loosely bound and invoked through communication protocols that stress location transparency and interoperability Services encapsulate reusable business function. Services may exist on different levels and provide different granularity.

o Providing interoperability with more modern Web-based systems (This would extend the practical life of legacy routines and processes) o Lowering the overall IT cost structure by sharing business services across multiple applications o Decreasing complexity by greatly reducing redundancy in the enterprise • SOA benefits to the Business o Improving productivity, agility, and speed o Closer alignment of IT with business allows for greater agility in modifying business processes SOA characteristics that enable or facilitate the above mentioned advantages are listed in Figure 3.

Figure 3: Chief Characteristics of SOA
• Abstract: Hiding the business logic from the service consumers. • Autonomous: If a service is autonomous it has the basic data, or information to completely govern the business logic that it encapsulates. • Composable: Services should be defined in such a manner that it should be possible to facilitate the assembly of composite services resulting in highergrained services. • Contractual: Contract-driven agreement on service descriptions serve to formalize obligations to be met by the publisher as well as the consumer of the service. • Discoverable: It should be possible to ‘discover’ (or search for) services across the network. • Loosely coupled: This minimizes dependencies between services. • Reusable: Define the business logic as reusable services. • Stateless: Minimize retained information specific to an activity so that any invocation of the service does not need to ‘remember’ data from earlier invocations. Web Services is just one method by which SOA architecture can be implemented. Web services are based on invocation using SOAP messages which are described using WSDL over a standard protocol such as HTTP.

• •

SOA strategy brings enormous benefits to the enterprise such as listed in Figure 2.

Figure 2: Benefits of SOA
• SOA benefits to IT o Improving productivity, agility, and speed o Composing new applications from existing software assets o Align IT more closely with business

Linking Legacy and the new, open technologies and modern distributed architectures.
If we need to build more agility into Legacy applications deploying services in SOA architecture is a good approach to adopt. SOA accelerates time to value in legacy applications by:

• • •

Decreasing complexity by providing a uniform way of linking services and a uniform framework for integrating them. Replacing static linkage of services with dynamic linkage and reducing the resistance to change, allowing IT to track changes in business processes. Providing an environment for reuse and encouraging consistency. Simplifying operations by providing a uniform way to monitor and manage services. Simplifying service implementations by handling resources and other management tasks in service containers.

not intended to be a comprehensive exposition on SOA and does not attempt to address the myriad of topics that SOA-enablement involves, e.g. Governance, Performance, Security, Testing, etc.

2. Extracting Services from Legacy Applications
Note: Instead of simply listing down what legacy artifacts can be reviewed for mining of services, the approach adopted by this paper is to provide a brief explanation of the preparatory work that must necessarily happen BEFORE service mining can actually begin. This introductory material is covered – though briefly – in the sub-section A below. Subsections B through D concentrate on the IT artifacts that can be reviewed for mining of services.

However, based on the application deficiencies mentioned earlier in Figure 1, it is not really easy to make the unchanged legacy applications effectively participate in SOA architecture. The critical pre-requisite that would enable this participation would be the identification and extraction of possible candidates of services that can be made available to a SOA-enabled ecology (i.e. enterprise’s business processes, enterprise’s IT infrastructure, vendors, customers, etc). The fundamental problem that we are attempting to resolve in this paper is to define a generic approach for identifying and extracting services from legacy application keeping in mind the following criteria – 1. The identified service should make use of code from legacy application – either unchanged or modified to eliminate the user-interactions The process of identification of services should encompass not only the legacy application but also the manual processes that might accompany the legacy work-flow or process-flow Depending on the nature of the service, it should be re-usable either within the organization or from outside the organization

A. CREATE A BLUEPRINT FOR A SOAENABLED ENTERPRISE
Before we embark on a journey to identify and extract services from legacy applications, we need to have a clear definition of the business landscape (TO-BE) in which the SOA-based IT assets and business processes would function. This might –just, MIGHT – be the same as the current business landscape (AS-IS), but still that decision needs to come out of conscious deliberations. As an example, SOA can start in a small manner – say with web services – before growing into a full-blown enterprise-wide architecture. In either case, it helps to have the AS-IS and TO-BE clearly defined before we embark on identifying and extracting services. 1. Identify business domains within the enterprise business structure Enterprises are typically composed of many business function (for example, Sales, Marketing, Account Management, Customer Services, and so on) or implementations (such as Mainframes and Wintel servers). Making sense of the big-picture in such an enterprise is facilitated by dividing the each facet of the enterprise – be it business function or implementation – into domains. Essentially, a domain would contain applications and functions that are connected tighter, and would therefore have greater dependencies to each other than to the rest of the enterprise. This domainidentification process would facilitate later processes because of the advantages brought about by separation of concerns. Use the following to identify business domains –

2.

3.

With the above-mentioned guiding principles, • In Part II of this paper we attempt to lay down guidelines on how to go about looking for and extracting services from Legacy applications. Part II provides a listing of some signposts in existing legacy applications that can be used to identify and extract services from such systems. This topic forms the main thrust of this paper. • In Part III of this paper we also document a very simplistic approach to deploying a service extracted from legacy into a SOA architecture using Web Services. However, please note that this last part is

• AS-IS enterprise organization structures • TO-BE enterprise organization structures (if any change is being envisaged) • AS-IS Menu Structure • AS-IS Business Functions • Those portions of Vendor’s systems that the AS-IS and/or TO-BE environments would interface with • Those portions of Customer’s systems that the ASIS and/or TO-BE environments would interface with Initial separation into domains may also be purely conceptual, allowing for identification of application functions that need to be exposed as services to the rest of the enterprise. Once services are identified, domains need to be separated from each other by establishing clear boundaries enforced through use of gateways and firewalls. Such separation allows for better control over application interactions and further flexibility for making changes to applications without dramatically affecting the rest of the enterprise. 2. Create a high-level Architectural Layout of how the domains interface with each other Use the following to define the interfaces between business domains identified in earlier step • • AS-IS information flow between domains TO-BE interactions between the various domains – functional and/or technical

• • • • •

Identify the major and minor processes executed by TO-BE department in the organization structure. The domain should be validated for : Sufficiency of processes for executing the business Have adequate “inflow” of data / information required to execute the processes Provide appropriate “outflow” of data / information expected from the domain

B. TOP-DOWN SERVICE MODIFICATION
Figure 3 elaborates SOA characteristics. It would be advantageous to bear those characteristics in mind while mining for candidate services (i.e. functions that can be used to create services) from within the legacy application. • The service should present the functionality to the user / consumer at the level of granularity that the user / consumer needs or understands. There are different kinds of re-usability ex. code re-use or functionality service. The key criteria to be kept in mind when mining for services within the legacy application is that one should mine for re-usable services rather than only re-usable code snippets. SOA Services are typically published so as to be discoverable by others – from within the organization or even from outside the organization. When analyzing legacy functions that could possibly be converted into services, one should keep in mind the users or callers of that service. That in turn should determine the kind of visibility (i.e. publishing) that needs to be provided for the service. Levels of Abstraction that can be achieved within a legacy function should be kept in mind. The legacy function that is being evaluated for its candidature as a service needs to be analyzed if it can be abstracted, generalized or extensible. If not, the legacy function is, quite possible, very specialized or specific – and hence may not be an appropriate candidate as a SOA service. Another related factor that can be analyzed is whether it would be possible to build a formal contract around that service wherein the publisher as well as the consumer has certain obligations.

3. For each domain identified above, also identify business processes that make up the business domain. Processes can be seen as the blueprint for the behavior of people within an enterprise. All important activities will be covered by a process, requiring the definition of ownership, relation with other processes et cetera. From an IT perspective, when thinking about business processes related to a domain, you must also ask • • • • What functions are contained within them What data is used Where the data resides Who's using these processes (people or applications)

This can be a difficult task based on the design of your in-house applications. Use the following steps to identify business processes that are a part of the identified domain

When identifying candidate services, one should also bear in mind that the service should be selfcontained (i.e. autonomous, loosely coupled, composable and stateless). In other words the legacy functionality that is being converted to a service should be o o Capable of having access to all the rule sets and data sets that can govern the service’s behavior Definable in terms of data that it has access to – without having to remember data pertinent to earlier runs. Usable by the consumer without requiring detailed knowledge of the service before invoking it Definable at the right level of granularity (When mining for services, one needs to bear in mind that there can be different levels of granularity of services. Rather than making one big (very highgrained service) it would be best to identify candidate services at a lower-level that can be composed into higher-grained service. In this attempt one should bear in mind that whatever be the level of granularity at which a particular service exists, the candidate service should represent one complete function to its consumer.)

which of those are data that can be re-used by different user communities – some within the enterprise and, maybe, some outside the enterprise. • Transaction Processes – Transaction processes in legacy applications – ex. ‘Place Order’, ‘Make Reservation’, etc – are the backbone of the business. One can review the transactions in the AS-IS and TO-BE designs to define which of those are re-usable enough – either today or probably in the future – to qualify being abstracted into a service that is callable either from within or outside the enterprise. Audit Log Process – Changes made to sensitive data is many times auditable. When reviewing the transactions we need to keep note of those that record such changes in an audit log. It is possible that we might be required to derive services for query and presentation of audit logs. Processes that send data to systems other than the legacy platform – The interfaces between the enterprise, its’ customers and vendors are good candidates for extracting services because these services can then be made available on the internet. (ex. EDI) Archival of information – Periodicity of Archives can have a constraint on the data that can be available to services. It is definitely advisable to keep in mind the archival cycles when visualizing services.

o

o

With the above considerations in mind, the rest of the paper identifies some possible sources from where candidate services could be extracted from legacy application. For each Business Process within the Domain perform the following steps. i. Map the business process with supporting processes in the existing system o Identify “Information Generator” processes such as : • Application Set-up Processes - An example where this information can be useful is the User Profiles that get set-up for applications. Based on the profiles, a broad categorization of user community can be done which in turn will have usage in defining the kind of services that each user class would be interested in. Master Maintenance processes - Typically master maintenance processes on a legacy system would define some of the major data repositories in the system. By reviewing the Master Maintenance options available in Legacy systems we can define

o Identify “Information Publisher” processes Processes which are responsible for supplying users with the latest information are also candidates for services because once these are made available as services and brought out of the constraints of legacy technology, we can immediately consider many of such services for propagating critical information to various devices such as handhelds, tablet computers, internet, charting tools, etc thereby at least facilitating the creation of a more agile enterprise responsive to changing business dynamics. Some of the processes that can be reviewed from this angle are: • • • Reports and Query Screens of critical data, alerts, KPIs, etc Views Processes that receive data from systems other than the legacy platform e.g. EDI

o Identify “Data Collation” processes Very often it has been observed that critical decision-making logic is embedded in data collation processes such as: • • • • • Day-End Processes Month-End Processes Year-End Processes Triggers Stored Procedures

others – either within the organization or outside the organization. • Data Archival Processes (What to backup, With what frequency and Where to store the archives) Disaster Recovery Processes (How to recover from a Disaster) Business Continuity Processes (How to continue business in the face of a disaster)

• •

It has also been observed that more often than not this decision-making logic – which typically reflects the way the business operates – is embedded deep in code. By evaluating such legacy programs we might identify logic that is better packaged as a service so that it is easier to change without fear of accidentally disrupting a whole lot other areas of these processes. Packaging logic into services can help not only from the perspective of re-usability but also from easy maintainability. o Identify “Application Support” Processes Many legacy systems lack the functionality to efficiently interact with support processes such as Work-Flow processes. This is typically because transactions that can and should get triggered based on a task’s progress in the workflow system are too heavily enmeshed within the existing code. By thinking of the AS-IS and TOBE from the Work-Flow perspective – irrespective of whether one has or does not have a work flow application – one can identify those pieces of the legacy application that can be extracted into services which can be used to build a more robust work flow process. o Identify “Database Support” Processes Most of the legacy systems are characterized by tens or hundreds of programs writing data to critical tables. This is a definite problem when a change is to be implemented in the way data is currently being written. This problem could be alleviated by identifying the scope and utility of wrapper services that act as the ONLY way to send or retrieve data from database tables. o Identify “Technology Support” Processes Typically technology support processes such as those mentioned below would not yield any reusable services that one would want to expose to

However, consciously looking through such processes and consciously deciding that none of it is going to be useful as a service is an extremely important step when you do not want to miss any possible candidate services. ii. Map the business process with supporting manual processes In many businesses, in spite of wide-spread computerization, manual processes have not disappeared totally. The services that are being envisaged need to be reviewed with respect to the manual processes to determine: o o Can the services reduce manual processes? If not, how can the services at least facilitate or optimize the manual processes?

It might so happen that these deliberations itself can give rise to new or modified processes – and hence services. iii. Identify the functions that are the constituent parts of the identified business process in the system o Decompose the identified processes to finergrain functions that happen within those processes

iv.

For each identified business function in the system: o Analyze function for re-usability. If neither the function nor the business processes is re-usable probably there is not much candidacy for creating a service. If function is not re-usable but business process is re-usable, analyze if the function would be impacted if the business process is made into

service. If yes then, probably there is not much candidacy for creating a service. o Identify where similar functions are repeated in the Legacy application - If multiple occurrences are found, analyze each occurrence for similarities and differences. Analyze if the similarities are significant enough and the differences are NOT significant enough to conceptualize a generic function that can address both requirements with minor (if any) changes, ex. adding a process type parameter. Apart from the main programs itself, possible candidates could be found in: • Copy Books • Service Programs • Modules o o Envision a new module of proper granularity and generality This new module should execute only one function related to any one of the following needs - Business Needs, Technical Needs, Application Needs, Database Needs Based on the specific needs, identify the interface for the module. Create the module implementing various SOA requirements Package the module(s) into re-usable services

3. Creating Services derived from Legacy Applications
Note: As noted earlier in Part II, this section does not attempt to provide a comprehensive coverage of SOArelated topics such as Governance, Performance, Security, Testing, etc. Instead it is just an example of implementing a service mined from legacy iSeries application and brought out as a web service. We have chosen web service as a representative method of implementing SOA. There can be other implementations of SOA. In the example that we have attempted, we extracted the Player (i.e. Customer) Rating logic from an iSeries-based Casino Management System and made it available as a web service. In this section, we highlight some of the considerations that went into this process. Once functionality has been identified as a suitable candidate for converting into service, it is best to determine how the business logic can be disconnected from the original user interface and made into a self contained subprogram wherein the original input arguments will be represented as input parameters and the original output arguments would be represented as output parameters. One approach for achieving this could be to create a separate version of the function that is devoid of all user-interface (UI) related items (ex. Messages, Prompts, input and output fields on the screen, etc). This can be attempted for simple functions. However, if it is perceived that this process is not trivial and might introduce risks, then an alternative approach might be to utilize products such as look software that convert interactive programs to run in batch. This would enable the calling routines to use the existing program without worrying about possible problems due to the presence of UI components. In either case, the end result will be a subroutine with a call interface. The next step is to wrap this sub-routine into a Web Services Description Language (WSDL) interface, providing a proxy to it on the application server and, finally, making it discoverable by making it available on Universal Description, Discovery, and Integration (UDDI) or equivalent tools that allow callers to search for available services. Web Sphere Studio Client for iSeries (WDSCi) was the IDE tool used for this purpose. Attached below is a screen snapshot of WDSCi IDE depicting various facets of the end product viz. Sources generated, input parameters and output XML.

o

o

o o o v.

Sufficiency of Services o Ensure all Services and identified business process are together sufficient for conducting the business of the domain

C. CROSS-DOMAIN SERVICE VERIFICATION
For each domain perform the following verificationrelated steps. • • • Identify which business processes interface with other domains Analyze nature of cross-domain interface Analyze the data being interchanged across the interface for the following factors: Consistency, Correctness, Adequacy, Recovery, Message flows.

Example Screenshot of Deployment of Service Using WDSCi

4. Conclusion
In this paper we have attempted to define some of the critical activities and decision points that are integral to identification and extraction of services from Legacy applications. Brief overviews of sign-posts identified in this article are: (a) Functions that handle critical business services (b) Functions critical for ensuring data integrity and consistency.

5. Future Scope
As an extension to this work, we would like to develop a similar guideline document especially from the perspective of how the presence of Legacy Application can influence SOA features such as Governance, Quality, Security, and Testing.

6. References

[1] Adapting legacy systems for SOA - Finding new business value in older applications - Calvin Lawrence, IBM http://www.ibm.com/developerworks/webservices/library/wssoa-adaptleg/ [2] Towards Legacy Enablement Using SOA and Web Services http://webservices.sys-con.com/read/164558.htm [3] Service Oriented Architecture http://en.wikipedia.org/wiki/Service-oriented_architecture [4] OASIS http://en.wikipedia.org/wiki/OASIS_SOA_Reference_Model [5] Service-Oriented Architecture (SOA): Concepts, Technology, and Design (The Prentice Hall Service-Oriented Computing Series from Thomas Erl) http://www.amazon.com/ServiceOriented-Architecture-SOA-TechnologyComputing/dp/0131858580 [6] Unlocking the Mainframe www.bea.com/content/news_events/white_papers/BEA_Unlocki ng_Mainframe_wp.pdf [7] Web Sphere Development Studio – Basics http://search400.techtarget.com/sDefinition/0,,sid3_gci1164479, 00.html