Server Integration Services (The Power of .
NET for ETL for SQL Server 2019 and
lding-custom-tasks-for-sql-server-integration-services-the-power-of-net-for-etl-for-s
Building Custom
Tasks for SQL Server
Integration Services
The Power of .NET for ETL
for SQL Server 2019 and Beyond
—
Second Edition
—
Andy Leonard
This Book is Available on [Link]
Server Integration Services (The Power of .NET for ETL for SQL Server 2019 and
lding-custom-tasks-for-sql-server-integration-services-the-power-of-net-for-etl-for-s
Building Custom
Tasks for SQL Server
Integration Services
The Power of .NET for ETL for
SQL Server 2019 and Beyond
Second Edition
Andy Leonard
This Book is Available on [Link]
Server Integration Services (The Power of .NET for ETL for SQL Server 2019 and
Building Custom Tasks for SQL Server Integration Services: The Power of .NET for
ETL for SQL Server 2019 and Beyond
Andy Leonard
lding-custom-tasks-for-sql-server-integration-services-the-power-of-net-for-etl-for-s
Farmville, VA, USA
ISBN-13 (pbk): 978-1-4842-6481-2 ISBN-13 (electronic): 978-1-4842-6482-9
[Link]
Copyright © 2021 by Andy Leonard
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with
every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an
editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the
trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not
identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to
proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date of publication,
neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or
omissions that may be made. The publisher makes no warranty, express or implied, with respect to the
material contained herein.
Managing Director, Apress Media LLC: Welmoed Spahr
Acquisitions Editor: Jonathan Gennick
Development Editor: Laura Berendson
Coordinating Editor: Jill Balzano
Cover image designed by Freepik ([Link])
Distributed to the book trade worldwide by Springer Science+Business Media LLC, 1 New York Plaza, Suite
4600, New York, NY 10004. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.
com, or visit [Link]. Apress Media, LLC is a California LLC and the sole member (owner)
is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware
corporation.
For information on translations, please e-mail booktranslations@[Link]; for reprint,
paperback, or audio rights, please e-mail bookpermissions@[Link].
Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and
licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales
web page at [Link]
Any source code or other supplementary material referenced by the author in this book is available to
readers on GitHub via the book’s product page, located at [Link]/9781484264812. For more
detailed information, please visit [Link]
Printed on acid-free paper
This Book is Available on [Link]
Server Integration Services (The Power of .NET for ETL for SQL Server 2019 and
Table of Contents
lding-custom-tasks-for-sql-server-integration-services-the-power-of-net-for-etl-for-s
About the Author ��������������������������������������������������������������������������������������������������� xiii
Acknowledgments ���������������������������������������������������������������������������������������������������xv
Foreword ���������������������������������������������������������������������������������������������������������������xvii
Author’s First Edition Thoughts ������������������������������������������������������������������������������xxi
Chapter 1: The Story of This Book ���������������������������������������������������������������������������� 1
Tribal Knowledge 1
A Starting Point 2
The Problem We Are Trying to Solve 3
Updates to the 2019 Edition 3
Chapter 2: Preparing the Environment ��������������������������������������������������������������������� 5
Creating an Azure DevOps Account 5
Creating an Azure DevOps Project 6
Configuring Visual Studio for Source Control 8
Conclusion 13
Chapter 3: Creating the Assembly Project �������������������������������������������������������������� 15
Opening Visual Studio IDE 15
Adding a Reference 20
Conclusion 24
Chapter 4: Check in the Project Code ��������������������������������������������������������������������� 25
Initialize Source Control in Azure DevOps 25
Conclusion 31
iii
This Book is Available on [Link]
Server Integration Services (The Power of .NET for ETL for SQL Server 2019 and
Table of ConTenTs
Chapter 5: Signing the Assembly ��������������������������������������������������������������������������� 33
Preparing to Add a Key 33
Creating the Key 38
lding-custom-tasks-for-sql-server-integration-services-the-power-of-net-for-etl-for-s
Applying the Key 42
Checking In Changes 45
Conclusion 48
Chapter 6: Preparing to Build ��������������������������������������������������������������������������������� 49
Using Build Events to Register/Unregister 49
Finding gacutilexe 50
Adding Build Events 50
Setting the Output Path 54
Conclusion 55
Chapter 7: Coding the Task ������������������������������������������������������������������������������������� 57
Using a Reference 57
Decorating the Class 58
Inheriting from MicrosoftSqlServerDtsRuntimeTask 60
Adding a Property 61
Investigating Task Methods 63
Overriding the Validate Method 67
Overriding the Execute Method 68
Conclusion 70
Chapter 8: Coding a Simple Task Editor ����������������������������������������������������������������� 71
Adding a Task Editor 71
Adding References 73
Some Implementation 75
Adding the User Interface Form 80
Coding the Form 86
Coding the Click Event and the Form 88
Conclusion 92
iv
This Book is Available on [Link]
Server Integration Services (The Power of .NET for ETL for SQL Server 2019 and
Table of ConTenTs
Chapter 9: Signing and Binding ������������������������������������������������������������������������������ 95
Creating a New Public Key Token Value 95
Signing the Task Editor Project 99
lding-custom-tasks-for-sql-server-integration-services-the-power-of-net-for-etl-for-s
Binding the Task Editor to the Task 102
Coding the Task Functionality 104
Add an Icon 108
Building the Task 113
Testing the Task 115
Conclusion 118
Chapter 10: Expanding Editor Functionality ��������������������������������������������������������� 121
Refactoring 121
Adding and Updating Task Properties 122
Updating the Execute Method 126
Adding a New Editor 127
Complex UI Overview 128
Adding the Editor Project 131
Adding References 136
Using Referenced Assemblies 138
Conclusion 142
Chapter 11: Minimal Coding for the Complex Editor �������������������������������������������� 143
Updating DtsTask Interface Members 144
Adding Internal Variables 145
Coding Interface Member Methods 145
Creating the Editor Form 147
Adding References 152
Coding the Form Constructor 153
Calling the Views 155
Coding the GeneralView Class 157
Coding the SettingsView 166
v
This Book is Available on [Link]
Server Integration Services (The Power of .NET for ETL for SQL Server 2019 and
Table of ConTenTs
Commenting Out Exceptions 175
Conclusion 175
lding-custom-tasks-for-sql-server-integration-services-the-power-of-net-for-etl-for-s
Chapter 12: Editor Integration ������������������������������������������������������������������������������ 177
Signing the Assembly 177
Creating the Key 178
Sign the ExecuteCatalogPackageTaskComplexUI Assembly 180
Configure the Build Output Path 181
Configure Build Events 182
Editing the ExecuteCatalogPackageTask 183
Build the Solution 184
Test the Custom SSIS Task 185
Conclusion 189
Chapter 13: Implement Views and Properties ������������������������������������������������������ 191
Implementing the GeneralView IDTSTaskUIView Interface 191
Implementing GeneralView OnInitialize 193
Implementing GeneralView OnCommit 198
Coding the GeneralNode 199
Coding the GeneralNode Properties 200
Testing GeneralView 206
Implementing the SettingsView IDTSTaskUIView Interface 206
Implementing SettingsView OnInitialize 208
Adding ExecuteCatalogPackageTask Properties 210
Implementing SettingsView OnCommit for FolderName, ProjectName, and
PackageName Properties 211
Coding SettingsNode for FolderName, ProjectName, and PackageName Properties 212
Coding the SettingsNode FolderName, ProjectName, and PackageName Properties 214
Testing SettingsView FolderName, ProjectName, and PackageName Properties 217
Coding SettingsNode for Connection-Related Members 218
Regarding SSIS Package Connections 219
Isolating SSIS Package ADONet Connections 220
vi
This Book is Available on [Link]
Server Integration Services (The Power of .NET for ETL for SQL Server 2019 and
Table of ConTenTs
Building the ADONetConnections TypeConverter 221
Surface the SourceConnection Property 227
Testing the SourceConnection Property 228
lding-custom-tasks-for-sql-server-integration-services-the-power-of-net-for-etl-for-s
Using the SourceConnection Property 229
Extracting the ServerName from the SourceConnection 230
Let’s Test It! 235
Conclusion 237
Chapter 14: Implement New Connection �������������������������������������������������������������� 239
Implementing New Connection Functionality 239
SourceConnection Property Value Changes 241
Let’s Test It! 246
Use Case 1 247
Use Case 2 249
Conclusion 252
Chapter 15: Implement Use32bit, Synchronized, and LoggingLevel
SettingsView Properties ��������������������������������������������������������������������������������������� 253
Specifying a 32-Bit Interface 253
Adding the Use32bit Property 253
Test Use32bit 255
Adding the Synchronized Property 257
Examining Execute() Overloads 258
Building the ExecutionValueParameterSet 260
Calling the returnExecutionValueParameterSet Function 262
Adding the Synchronized Property to SettingsView 263
Test Synchronized 263
Adding the LoggingLevel Property 265
Building the LoggingLevel TypeConverter 266
Extending the ExecutionValueParameterSet with LoggingLevel 268
Adding the LoggingLevel Property to SettingsView 271
Test LoggingLevel 272
Conclusion 274
vii
This Book is Available on [Link]
Server Integration Services (The Power of .NET for ETL for SQL Server 2019 and
Table of ConTenTs
Chapter 16: Refactoring SourceConnection ���������������������������������������������������������� 275
Thinking Azure-SSIS 276
Refactor Connection Identification 283
lding-custom-tasks-for-sql-server-integration-services-the-power-of-net-for-etl-for-s
Refactor the SettingsViewSourceConnection Property 288
Identifying Catalog, Folder, Project, and Package 291
Adding the returnCatalogProject Method 293
Adding the returnCatalogPackage Method 296
Adding the returnCatalog Method 299
Adding the returnCatalogFolder Method 300
Let’s Test It! 302
Conclusion 303
Chapter 17: Refactoring the SSIS Package Hierarchy ������������������������������������������ 305
Refactor the SettingsView Catalog, Folder, Project, and Package Properties 305
The Folder Property and Folders Collection 307
The Project Property and Projects Collection 321
The Package Property and Packages Collection 334
Resetting Collections 346
Let’s Test Expressions! 353
Conclusion 360
Chapter 18: Instrumentation and Validation��������������������������������������������������������� 361
Instrumentation 361
Adding Instrumentation 361
Let’s Test It! 370
Validation 372
Validating the SourceConnection Property 372
Building the Connection Validation Helper Function 379
Calling attemptConnection in the Validate Method 382
How I Tested the attemptConnection Validation 384
Validating Folder, Project, and Package Properties 387
Conclusion 398
viii
This Book is Available on [Link]
Server Integration Services (The Power of .NET for ETL for SQL Server 2019 and
Table of ConTenTs
Chapter 19: Crushing Bugs ����������������������������������������������������������������������������������� 399
Execution Timeout Expired 399
Threading, Briefly 400
lding-custom-tasks-for-sql-server-integration-services-the-power-of-net-for-etl-for-s
ExecuteCatalogPackageTaskExecute 401
Designing a Test SSIS Package 402
Implementing a WaitHandleWaitAny Method 410
Refactoring logMessage to raiseEvent 420
Refactoring the Execute Method 429
Let’s Test It! 445
Conclusion 449
Chapter 20: Adding Synchronous Execution Properties ��������������������������������������� 451
Surfacing the New Properties 452
Adding MaximumRetries, RetryIntervalSeconds, and OperationTimeoutMinutes 454
Cleaning Up Outdated Asynchronous Execution 459
Let’s Test It! 462
Conclusion 464
Chapter 21: Testing the Task �������������������������������������������������������������������������������� 465
Manually Testing Use Cases 465
Use Cases 466
Testing Task Existence 466
Testing Task Validation 467
Testing the Task Editor 469
Testing Execution 483
Conclusion 496
Chapter 22: Building the Setup Project ���������������������������������������������������������������� 497
Adding the ExecuteCatalogPackageTaskSetup Project 498
Adding References 502
Configuring Tags 505
Configure the Product tag 505
ix
This Book is Available on [Link]