You are on page 1of 7

Cognizant 20-20 Insights

A Standardized Approach to SSIS Migration


Within every migration project, interface issues emerge that create maintenance complexities and unnecessary system down time. By embracing common practices and tools, IT organizations can more quickly and effectively move SSIS software to new and more modern platforms.
Executive Summary
SQL server integration services (SSIS) is the enterprise class extract-transform-load (ETL) tool that accompanies Microsoft SQL2005/2008. The data transformation services (DTS) that existed with SQL 2000 was rewritten for SSIS in subsequent versions of the tool. To deliver an improved feature set, which primarily consists of enhanced SQL 2008 database performance, existing DTS packages were migrated to SSIS 2008. Though there exist options/tools for migration of DTS to SSIS, the best-in-class approach to leverage SSISs improved functionality and avoid the complex logic of DTS (e.g., ActiveX scripts) is to use the complete rewrite of DTS functionality in SSIS. Standardized processes must be followed whenever such migrations occur to increase the maintainability of SSIS jobs after migration. Without a standardization, a single SSIS may have multiple configuration files maintained in multiple or single locations making the change to the configuration complex. Also changes (minor or major) to the configuration have a dependency on the DBA; there is no way for the DBA to do impact analysis of changes made to a particular configuration of server/location without much manual work. This white paper explains one such standardization that can be followed for the generic actions of most SSIS jobs:

Configuration. Error Handling. Monitoring.

Configuration Standardization
Most of the connection strings used in the SSIS components as well as other change-prone properties of any task in SSIS must be configured to enable an SSIS job that is transferable and maintainable. Challenges in Configuration Management

During SDLC of any SSIS, the tool must be moved to work in different environments (DEV, TEST and PROD predominantly). This results in changing SSIS/Config files during every move, which is always a time-consuming process.

cognizant 20-20 insights | november 2012

Post-production, any expiring user ID passwords, server migrations, database migrations and resource movements necessitate a configuration change of a related SSIS package. Most servers/job rights would be held under a DBA resulting in frequent requests to update an SSIS configuration which leads to an actual delay of the change and also appears as overhead for IT and the support function.

failed. If the IT/support has access to the server, the job history can be viewed for the actual error message. If not, then there is a dependency on the DBA for the error message to proceed with the actual resolution of the error. The package should be designed in such a way that it has tasks embedded to notify the target audience of the failure of the job with the error message. This will improve failure resolution efficiency. If required per the business, notifications should also be configured in the package to send success of the job process to the desired audience. This can also be done for monitoring highly critical jobs. The package should have logging enabled and appropriate log type chosen such as log file or event viewer.

If required per the business, notifications should also be configured in the package to send success of the job process to the desired audience. This can also be done for monitoring highly critical jobs.

To overcome the above challenges, rather than hard coding a package configuration or having the configuration in a file, we suggest holding all the configurations in database tables and just use one single configuration to connect to that database in a configuration file that will be a common configuration file across all SSIS packages for an application or customers using the software.

Proposed Design Solution


A new database must be created to hold the SSISs configuration tables. It is a best practice to have this database exclusively for configuration purposes only to avoid security restrictions. This database configuration will be created in an XML configuration file and used across the SSIS packages as a generic configuration file. Further properties/connections in the SSIS will be configured using an SQL server configuration type of SSIS in the new database table (see Figure 1). Configuration Standardization One table should be designed to hold only the configurations such as connection strings, variables, properties, log paths, etc. (see below).

Error Handling And Monitoring


If under any circumstance the package fails, then the target team should be informed of the failure. Challenges During Job Failures

If no error handling is embedded in the job, the failure may exist unnoticed, resulting in critical business impact for the customer. The SQL server agent can be configured to send an alert on failure, but it is just an alert with the name of the job and the particular step that

Configuring SSIS

All Other Connections & Parameters

SSIS 1XML Config file AppCong DB Connection

APPCong DB

6 Tables 1 SP

Figure 1

cognizant 20-20 insights

The ConfiguredValue column of the below table can be stored encrypted if the customer security policies require encryption of the sensitive data in the configuration. See section Securing Configurations for a detailed description of this approach. Table Name Column Name
ConfigurationFilter ConfiguredValue PackagePath ConfiguredValueType

SSIS_CONFIGURATIONS Data Type


nvarchar nvarchar nvarchar nvarchar

Remarks
Holds the value that links the table and the package. Value of the parameter like DB name, password, etc. Configuration path details. Data type.

This table will be used by the SSIS at runtime to configure its various properties. With any changes due to server migrations, database migrations and/or password changes, an update of that particular row of this table ends the change and activates the package for the new configuration. Error Handling and Monitoring For every successful job, there will be a set of people within different roles who should receive these e-mails; there also may or may not be a different set of people for the failure e-mail. This also varies across job areas and with applications. Role group, application group, people group, status group, success content and failure content, in order, should be configured in interlinked database tables that can be dynamically picked by the job using a custom stored procedure per job needs. To incorporate the aforementioned requirements, five additional database tables and a stored procedure need to be created (as shown below). The new database AppConfig will have the below components: 1. APPLICATION_HEADER Table 2. EMAIL_CONTENT_DETAIL Table 3. EMAIL_CONTENT_HEADER Table 4. EMAIL_SERVICE_DETAIL Table 5. EMAIL_SERVICE_HEADER Table 6. SSIS_CONFIGURATIONS Table (See above Configuration Standardization section for details) 7. SP_EMAIL_CONFIGURATIONS Stored Procedure

Any changes due to resource movements, additions and role changes can be handled by updating the above table records. 1. APPLICATION_HEADER: This table holds the key and the name of each application. Column Name
App_Key App_Description Mod_Date Mod_User

Data Type
numeric varchar datetime varchar

Remarks
Holds the key for each application. Application name. Date field. Modified user details.

2. EMAIL_CONTENT_DETAIL: This table holds the e-mail content for success and failure run of the package. Column Name
EMAIL_KEY EMAIL_CODE DESCRIPTION MOD_DATE MOD_USER

Data Type
numeric varchar varchar datetime varchar E-mail key.

Remarks
E-mail code to differentiate the email sending option. Description of e-mail receiver. Modified user details. Modified date.

cognizant 20-20 insights

3. EMAIL_CONTENT_HEADER: This table holds the header details of the email such as subject and footer notes. Column Name
EMAIL_KEY JOB_CODE EMAIL_SUCCESS_SUBJECT EMAIL_FAIL_SUBJECT EMAIL_FOOTER MOD_USER MOD_DATE

Data Type
numeric varchar varchar varchar varchar varchar datetime E-mail key.

Remarks
Job name which must be the same as package name. Holds e-mail subject for success run. Holds e-mail subject for failure run. Holds e-mail footer. Modified user details. Modified date.

4. EMAIL_SERVICE_DETAIL: This table holds the e-mail IDs of the personnel for whom the e-mail notifications are to be sent. Column Name
LOGIN EMAILID FULLNAME APP_KEY EMAIL_KEY MOD_DATE MOD_USER

Data Type
varchar varchar varchar numeric numeric datetime varchar

Remarks
Holds UPIN ID. E-mail address of the person. Name of the associate. Application key. E-mail key. Modified date. Modified user details.

5. EMAIL_SERVICE_HEADER: This table holds the different roles of the users who will be receiving e-mail notifications on job success/failure.. Column Name
EMAIL_KEY EMAIL_CODE DESCRIPTION MOD_DATE MOD_USER

Data Type
numeric varchar varchar datetime varchar E-mail key.

Remarks
E-mail code to differentiate the email sending option. Description of e-mail receiver. Modified user details. Modified date.

7. SP_EMAIL_CONFIGURATIONS: Retrieves the e-mail ID based on the input parameters passed from the SSIS package. The input parameter to the SP will be set based upon the e-mail notification requirement of the job such as success/failure, customer/IT, etc.

Note: Multiple email IDs can be connected into a row using ;.


Securing Configurations Whenever customer sensitive package configuration data (e.g., passwords, usernames, server details) is stored in plain text in the configuration table, it appears as a security concern per most customer policies. This should be addressed with the following design addition and changes to the aforementioned standardization design.

Create a view SSISConfigurations that has all the fields from the table SSIS_Configurations with the ConfiguredValue column decrypted. Create three triggers on the view for insert, delete and update. The triggers should facilitate the corresponding action in the configuration table SSIS_Configurations with the plain value in the view encrypted by the triggers and stored in the table.

Change column ConfiguredValue in the table SSIS_Configurations to VarBinary: datatype and add an identity ID column to the table. cognizant 20-20 insights

The encryption and decryption is by a password. Also, the view should have restricted access for reading its DDL. All SSIS packages should use the created view for SQL server configurations. To implement the security concept alone for the standardized package, just renaming the configuration table with encryption column and

creating a view (with decryption) by the name of the old configuration table and creation of triggers would suffice, requiring no changes at the SSIS packages front. Hence, for SSIS the view will retrieve decrypted plain text configurations and at the database end it will appear encrypted (see screen capture, below).

Attached below is a text document containing sample create scripts for the table, view and triggers.

Increased ease of maintenance. Any changes to the intended users (business or IT) can be made easily application-wise or in any grouping applied. Removing any inactive person from the notification list is a one-place activity rather than searching all the job configuration files (see screen capture, below).

Benefits
Key advantages of this approach include:

One time change incorporated with the migration.

cognizant 20-20 insights

Simplified configurations lookup for a specific job (see screen capture, below).

* The values are hidden as it is sensitive data.

Simplified lookup for a specific configuration across jobs. This helps find the usage of any server/DB/ file location, which is very useful during server migrations and DB changes. Even lookup by username is possible, which will make username and password changes much easier (see screen captures, below).

cognizant 20-20 insights

Well-monitored jobs increase database availability. Decreased down times. Decreased change implementation time. Decreased dependency on DBA/other teams for support.

Increased resolution efficiency due to data (error cases) availability for the desired people. Secured configurations. Security can be incorporated without change to the existing packages.

References

http://msdn.microsoft.com/en-us/library/cc768544(v=sql.100).aspx http://blog.scalabilityexperts.com/2008/02/14/the-key-sql-server-2008-bi-migration-dts-to-ssis2008-part-1-of-2/ http://www.jasonstrate.com/2011/01/31-days-of-ssis-sql-server-configuration-1831/ http://curionorg.blogspot.com/2007/05/encrypted-sql-server-ssis.html

About the Author


Surya P. is a Programmer Analyst within Cognizants Enterprise Information Management Practice. She has over three and a half years of IT within the business intelligence projects, development and support spaces. Surya has a bachelor of engineering degree in electronics and communication from Government College of Technology, Coimbatore (Anna University). She can be reached at Surya.Prakash3@cognizant.com.

About Cognizant
Cognizant (NASDAQ: CTSH) is a leading provider of information technology, consulting, and business process outsourcing services, dedicated to helping the worlds leading companies build stronger businesses. Headquartered in Teaneck, New Jersey (U.S.), Cognizant combines a passion for client satisfaction, technology innovation, deep industry and business process expertise, and a global, collaborative workforce that embodies the future of work. With over 50 delivery centers worldwide and approximately 145,200 employees as of June 30, 2012, Cognizant is a member of the NASDAQ-100, the S&P 500, the Forbes Global 2000, and the Fortune 500 and is ranked among the top performing and fastest growing companies in the world. Visit us online at www.cognizant.com or follow us on Twitter: Cognizant.

World Headquarters
500 Frank W. Burr Blvd. Teaneck, NJ 07666 USA Phone: +1 201 801 0233 Fax: +1 201 801 0243 Toll Free: +1 888 937 3277 Email: inquiry@cognizant.com

European Headquarters
1 Kingdom Street Paddington Central London W2 6BD Phone: +44 (0) 20 7297 7600 Fax: +44 (0) 20 7121 0102 Email: infouk@cognizant.com

India Operations Headquarters


#5/535, Old Mahabalipuram Road Okkiyam Pettai, Thoraipakkam Chennai, 600 096 India Phone: +91 (0) 44 4209 6000 Fax: +91 (0) 44 4209 6060 Email: inquiryindia@cognizant.com

Copyright 2012, Cognizant. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Cognizant. The information contained herein is subject to change without notice. All other trademarks mentioned herein are the property of their respective owners.