Ab Initio in Enterprise Application Integration

The recent pace of technological and environment changes have led to massive structural as well as operational changes as to how the organization is managed and how it interacts with customers in a highly competitive market economy. The main factors contributing to change are Mergers & Acquisitions, business expansion, globalization, and a growing and diverse customer base. To meet these changes companies need to implement increasingly complex software architecture, with much enhanced capability than the legacy systems, with the help of Enterprise Application Integration (EAI) solutions.

Ab Initio in Enterprise Application Integration



About the Authors Bikram Das Bikram is a senior consultant with TCS. He has been associated with BFSI. Arnab Das Arnab is a consultant with TCS BI practice. He currently heads TCS’ Ab Initio Center of Excellence. He has been involved in the execution of various BI projects and has earned kudos from customers for his value adding suggestions and creative solutions. Telecom and Healthcare industry verticals. He is currently a consultant for Ab Initio based solutions. Ab Initio in Enterprise Application Integration 1 TATA CONSULTANCY SERVICES . performance tuning and in providing solutions for Data Warehouse ETL environments. He has rich experience in database administration.

Ab Initio Capabilities 4. Introduction 2. Application Integration Implementation Using Ab Initio 5 3 3 3 4 Ab Initio in Enterprise Application Integration 2 TATA CONSULTANCY SERVICES . Application Integration Challenges 3.Table of Contents 1. Ab Initio Product Components 5.

It essentially performs the following functions: ● Reduces the complexity of an organization’s IT framework ● Provides a step improvement in its efficacy. complex. has gradually emerged as a strong player in the application integration spectrum with its wide range of enterprise scale. ● Establish an architectural framework that rapidly develops integration and reusability of applications. and a diversified and expanding customer base. It can run heterogeneously with parallel execution over distributed networks. acquisitions. ● Remove the redundancy of applications resulting from mergers and acquisitions. flexibility and scalability ● Supports new organizational initiatives. Ab Initio in Enterprise Application Integration 3 TATA CONSULTANCY SERVICES . ● Achieve faster return on investment in integration of business-to-business and business-toconsumer systems. To meet these changes. Transformation and Loading) and EAI (Enterprise Application Integration) tasks within a single. The primary factors contributing to the change are mergers.Introduction Information Technology systems have grown exponentially over the past few decades. It can integrate diverse. Application Integration Challenges The major challenges in implementing a suitable IT solution in this varying world of business operations are that the system must be able to do the following: ● Provide cost-effective integrated systems across the enterprise along with reduced time cycle for implementation. companies need to implement increasingly complex software architecture with much enhanced capability than the legacy systems. providing both ETL (Extraction. The best-of-breed software architecture for integrating disparate and distributed systems is Enterprise Application Integration (EAI). and continuous data streams ranging in size from several gigabytes to tens of terabyte. globalization. ● Adapt to the complexity of changing business models and customer demand. classically an ETL vendor. business expansion. The revolutions in technology and variation in environment have led to immense structural and operational changes in the way organizations are managed and the way they interact with the customers in the highly competitive market economy. and highly secure integration of systems and applications. The shift of focus toward tangible result-oriented customer relationship management projects is further highlighting the need for enterprise information integration capabilities as a key factor in reducing the time to market and the cost of ownership for customer-centric applications. fast. mission-critical applications that include: ● data warehousing (ETL) ● real-time analytics ● customer relationship management (CRM) ● enterprise application integration (EAI) Ab Initio provides a robust architecture that would allow simple. consistent framework. Ab Initio Capabilities Ab Initio.

Ab Initio Product Components The Ab Initio product portfolio consists of three core building blocks: ● The CO>Operating System – This system delivers distributed and parallel execution and platformindependent data transportation. connecting the components into executable flowcharts. which runs for an indefinite period of times. retail. an adaptation of the FIFO queue concept. health care. ● The Component Library – This is a set of reusable software modules that can perform the basic ETL functions as load/unload from different sources. to Windows NT or 2000. Subscriber is a component that writes data from various sources into a continuous flow process. multi stage aggregation. departmental organizations. The power and flexibility that make Ab Initio so effective at the enterprise level also empower smaller.Organizations of all sizes – in industries as diverse as telecommunications. e-commerce. including complicated tasks like checksum computation. The various components in Ab Initio that cater for the real time data mechanics are: ● ● Ab Initio Queues. It can distribute data load into heterogeneous systems and can execute a transformation on the entire volume of data simultaneously. Ab Initio in Enterprise Application Integration 4 TATA CONSULTANCY SERVICES . data sorting. to the different flavors of UNIX. etc. ● Ab Initio queues provide a method for storing records in an ordered sequence of files that can be fed to the ETL process. and consumes computepoints and checkpoints. continually taking in new input and producing new. ● A major capability of Ab Initio is its ability of parallelism. The Graphical Development Environment – This is a development environment to create Ab Initio applications with drag & drop facility. Publisher is a component that writes data to various destinations. insurance. data transformation etc. Ab Initio also provides a user friendly and effective mechanism for real time or near real time data processing. are a reliable method for storing continuous flow data. and shipping – use Ab Initio software to manage their most complex data. usable output. finance. and originates computepoints (the marking of blocks of records that are to be processed as groups) and checkpoints (intermediate states –wherein data processing can be restored from this point). XML reformation. It can run across a huge variety of operating systems and hardware platforms. from OS/390 on mainframes.

and loads data into target systems can also be provided. Redundancy – Multiple truth of source data due to data duplication. aggregations/ derivations for Business intelligence. The diagram below describes the general design architecture followed to integrate data from disparate sources into an Enterprise Data Warehouse and loaded into target CRM application. generic entities. A cost-effective solution can be provided using the Ab Initio batch or real time (continuous flow) execution mechanism. Figure 1: Implementation Architecture The major challenges in such an implementation are: ● ● ● Multiple Sources . Ab Initio in Enterprise Application Integration 5 TATA CONSULTANCY SERVICES . creates Data Warehouse. Complex Business Logic – Achieving common data format aligned with the target systems. A scalable solution that extracts data from distributed and disparate systems. operational data stores. transforms multiple format data into common format.Application Integration Implementation Using Ab Initio The following section discusses an application integration process facilitated by Ab Initio. with different data load frequencies.Data from disparate sources like Mainframe /Oracle table using different technologies. data formats. and data cleansing requirements.

The DRA can handle multiple source data in different data format. The data is then transported to the data staging area where the data is converted to common format data. Figure 2 : A Design Option for Ab Initio in Application Integration In this design.The following schematic diagram explains the solution to implement the above system architecture. The generic loader or extractor process is Ab Initio based applications that can perform miscellaneous functions: ● ● ● ● ● ● Load data into operational data stores Use metadata driven rules engine to generate code Provide PAI facility to perform Query Interface with database for data extraction and data load Provide delta and before-after image of data Feeds target system/ reporting tools or message queue Ab Initio in Enterprise Application Integration 6 TATA CONSULTANCY SERVICES . data from different sources are loaded into the data reception area (DRA) in periodic batch execution as well as near real time data flow using MQ series/AI queue.

posted or distributed in any form without prior written permission from TCS. Manufacturing and Retail domains. transmitted. Copyright © 2004-05 Tata Consultancy Services Limited Ab Initio in Enterprise Application Integration 7 TATA CONSULTANCY SERVICES . Insurance.tcs. and could result in criminal or civil penalties. The BI Practice has developed solutions for Banking. MIDC Andheri (East) Mumbai 400096 India Phone: +91-22-5550 6868 Fax: +91-22-5660 6855 Email: ranjita. No material from here may be copied. It has delivered 200+ projects in the last 3 years and has a consultant base of 2200 (as of March 2005). services and business process outsourcing organizations. reproduced. republished.com Website : www. trademark and other applicable laws. modified. TCS focuses on delivering technology led business solutions to its international customers across varied industries. Telecom.About Business Intelligence Practice The Business Intelligence (BI) Practice delivers solutions spanning across industry verticals and technologies to a majority of Global Fortune 500 companies under the umbrella of its proprietary BI Solutions BIDS™ (Business Intelligence for Decision Support). BFSI. The content / information contained here is correct at the time of publishing. About Tata Consultancy Services For more information contact Ranjita Saran Tata Consultancy Services Ltd Akruti Business Port Road No 13. uploaded. It has mature Technology Centers of Excellence and Alliances with all major BI tool vendors. Unauthorized use of the content / information appearing here may violate copyright. Pioneer of the flexible global delivery model for IT services that enables organizations to operate more efficiently and produce more value. Tata Consultancy Services (TCS) is among the leading global information technology consulting.saran@tcs.com/datawarehouse All content / information present here is the exclusive property of Tata Consultancy Services Limited (TCS).

Sign up to vote on this title
UsefulNot useful