AtomSphere Platform

Integration Implementation Planning Guide

Project Name Integration Implementation guide

Project Number 0-1

Customer Name Boomi

Prepared By Professional Services

Internal Use - Confidential

 Contents
1 Document Control Information ...................................................................... 19
1.1 Revision History .................................................................................. 19
2 Introduction ............................................................................................ 20
2.1 Overview .......................................................................................... 20
2.2 Concepts .......................................................................................... 21
3 Getting Started ........................................................................................ 23
3.1 General Account Setup and Administration ................................................. 23
3.1.1 User Access/Create Additional Users.................................................... 23
3.1.2 Controlling User Access to Your Account ............................................... 25
3.1.3 User Roles ................................................................................... 28
3.1.4 Privileges .................................................................................... 29
3.1.5 Account Groups ............................................................................. 30
3.1.6 Custom Roles and Privileges .............................................................. 36
3.1.7 Boomi Support Access ..................................................................... 41
3.1.8 Restricting Environment Access .......................................................... 41
3.1.9 Restricting Folder Access in Build ....................................................... 43
3.1.10 Security FAQ’s .............................................................................. 44
3.1.11 Common Platform and User Account Related Issues/FAQ’s ......................... 47
3.2 Atoms, Molecules, and Atom Clouds .......................................................... 51
3.2.1 Atoms ......................................................................................... 52
3.2.2 Molecules .................................................................................... 55
3.2.3 Atom Clouds ................................................................................. 59
3.2.4 How to Install a Local Atom, Molecule, or Cloud ...................................... 62
3.2.5 How to Migrate Your Atom to Another Server ......................................... 70
3.2.6 How to Migrate Processes to a New Atom .............................................. 72
3.3 Runtime Configuration and Mechanics ....................................................... 77
3.3.1 Configuration Files ......................................................................... 77
3.3.2 Custom Scripting and JAR’s ............................................................... 83
3.3.3 Communications ............................................................................ 86
3.3.4 Runtime and JVM’s ......................................................................... 87

Internal Use - Confidential

 3.3.5 Executions ................................................................................... 88
3.3.6 Flow Control................................................................................. 92
3.3.7 Atom, Molecule, Cloud Runtime Administration and Maintenance FAQs and
Common Errors ........................................................................................ 94
3.4 Connection Usage and Licensing FAQ’s ..................................................... 108
3.4.1 How Does Connection Usage and Licensing Work? .................................. 108
3.4.2 What’s the Difference between a Connector and Connection? ................... 108
3.4.3 What Represents a Unique Connection? .............................................. 109
3.4.4 What are Connector Classes? ........................................................... 109
3.4.5 How are Connections Counted? ........................................................ 110
3.4.6 Production vs. Test Connections ....................................................... 113
3.4.7 Do Published Web Services Count as Connections? ................................. 114
3.4.8 Are there any Free Connectors? ........................................................ 114
3.4.9 Do Custom Connectors Developed Using the SDK Count Towards Licensing? ... 114
3.4.10 How Can I Be Smart about My Connection Usage? .................................. 115
3.4.11 Where Do I View My Connection Licensing and Usage? ............................. 116
3.4.12 How Do I Get More Connections Added to My Account? ............................ 116
3.4.13 Common Licensing Issues................................................................ 117
3.4.14 Why Does My Connector List in the Licensing Tab include Connectors that have
been Deleted? If you notice a deleted connector listed in the Licensing tab: ........... 119
3.5 Environments ................................................................................... 120
3.5.1 Overview ................................................................................... 120
3.5.2 Considerations for Determining Your Environments ................................ 121
3.5.3 Software Development Life Cycle ..................................................... 121
3.5.4 Deployment and Version Control....................................................... 121
3.5.5 Environment Variables ................................................................... 122
3.5.6 Deployment Topology .................................................................... 122
3.5.7 User Access and Roles ................................................................... 123
3.5.8 Integration Use Case-Specific Runtimes .............................................. 123
3.5.9 Environment Setup for Different Scenarios .......................................... 124
3.5.10 Environment Extensions ................................................................. 129

Internal Use - Confidential

 3.5.11 Add an Environment ..................................................................... 130
3.5.12 Conclusion ................................................................................. 131
3.6 Approach for Migrating from an Existing Middleware Platform to AtomSphere...... 131
3.6.1 Overview ................................................................................... 131
3.6.2 Migrating from Another Middleware Platform ....................................... 131
3.6.3 Detailed Work Breakdown............................................................... 134
3.6.4 General Implementation Tips .......................................................... 138
4 Build the Integration Development Area ........................................................ 138
4.1 Organization .................................................................................... 138
4.1.1 Establish Naming Conventions .......................................................... 138
4.1.2 Improve Folder Structure ............................................................... 139
4.1.3 Eliminate Redundant Copies of Components and Connections .................... 139
4.1.4 Create Only a Single Copy of Each Process ........................................... 140
4.1.5 Copying Components: Shallow vs Deep Copies ...................................... 140
4.1.6 Avoid Multiple Disk Connection Components ........................................ 140
4.1.7 Specify Only Base Directory in FTP/SFTP Connections ............................. 141
4.2 Accountability .................................................................................. 141
4.2.1 Avoid Using Personal Logins for Endpoints ........................................... 141
4.2.2 Email Alerts ............................................................................... 141
4.2.3 Proper User Management ............................................................... 141
4.3 Process Overview .............................................................................. 142
4.3.1 Data Flow .................................................................................. 142
4.3.2 Data Manipulation ........................................................................ 173
4.4 Process Design .................................................................................. 251
4.4.1 Use Tracked Fields/Document Tracking .............................................. 251
4.4.2 Record Synchronization ................................................................. 259
4.4.3 Rejected Document Processing ........................................................ 260
4.4.4 Branch End ................................................................................ 260
4.4.5 Sensible Process Layout ................................................................. 260
4.4.6 Build from Outside In .................................................................... 260
4.4.7 Query Operation Design ................................................................. 260

Internal Use - Confidential

 4.4.8 Use Process Options to better meet client needs ................................... 261
4.4.9 Process Re-Run............................................................................ 261
4.4.10 Date Handling in Maps ................................................................... 261
4.4.11 Process Decoupling ....................................................................... 261
4.4.12 Business Rules Shape Considerations .................................................. 284
4.4.13 Accessing Data Changed in a Prior Branch ........................................... 284
4.4.14 Caching Considerations .................................................................. 284
4.4.15 Use of a Sub-Process to link Document Streams..................................... 294
4.4.16 Flow Control............................................................................... 294
4.4.17 File Naming When Using File-based Connectors ..................................... 294
4.4.18 Design for Reuse .......................................................................... 294
4.4.19 Test Mode Limits ......................................................................... 295
4.4.20 Component Locking ...................................................................... 295
4.5 Process Options ................................................................................ 295
4.5.1 Process Modes ............................................................................. 295
4.5.2 Options Within Each Mode .............................................................. 297
4.6 Connector Guides .............................................................................. 301
4.6.2 Database Integration ..................................................................... 319
4.6.3 NetSuite Integration ..................................................................... 323
4.6.4 Salesforce Integration ................................................................... 333
4.6.5 SAP Integration ........................................................................... 347
4.6.6 Workday Integration ..................................................................... 359
4.7 Understanding Data Profiles ................................................................. 370
4.7.1 XML (eXtensible Markup Language) .................................................... 370
4.7.2 Flat File .................................................................................... 375
4.7.3 Database ................................................................................... 377
4.7.4 EDI (Electronic Data Interchange) ..................................................... 380
4.7.5 JSON (Javacript Object Notation) ..................................................... 383
4.7.6 Notepad++ and Plugins .................................................................. 387
4.8 Common Integration Patterns................................................................ 389
4.8.1 Incremental Poll (ETL Style Data Synchronization) ................................. 389

Internal Use - Confidential

 4.8.2 Payload Driven Integration (Real-time, Synchronous, Record Payload) ......... 391
4.8.3 Event Driven Integration (Real-time, Synchronous, Event Payload) ............. 392
4.8.4 “Fire & Forget” Integration (Asynchronous) ......................................... 393
4.8.5 Call-back Integration (Asynchronous) ................................................. 394
4.8.6 Accumulate-then-Process Integration ................................................. 395
4.8.7 Decoupling Processing and Transmission ............................................. 396
4.9 Know Your Integration Load .................................................................. 397
4.9.1 One-Way Sync Strategies ................................................................ 397
4.9.2 Two-Way Sync Strategies (Bi-directional Sync) ...................................... 403
4.10 Testing Processes .............................................................................. 410
4.10.1 Overview ................................................................................... 410
4.10.2 Designing Testable Processes ........................................................... 410
4.10.3 Creating Test Harness Processes ....................................................... 411
4.10.4 Advanced Test Harness Process Ideas ................................................. 413
4.10.5 Creating Mock Web Service Endpoints ................................................ 414
4.10.6 Automating Testing ...................................................................... 416
4.10.7 Using Boomi Assure for Platform Regression Testing ............................... 416
4.11 HTTP Concepts ................................................................................. 417
4.11.1 Requests ................................................................................... 417
4.11.2 Request Components..................................................................... 420
4.11.3 Connector .................................................................................. 422
4.11.4 Installation ................................................................................ 428
4.12 Trading Partners ............................................................................... 432
4.12.1 Communication Methods ................................................................ 433
4.12.2 Receive and Send Process ............................................................... 433
5 Deployments ......................................................................................... 435
5.1 Performance Considerations ................................................................. 435
5.2 Requirement Satisfaction and Licensing (Prior to Deployment) ........................ 436
5.3 Deployment Considerations .................................................................. 436
5.3.1 Full Testing Checks ....................................................................... 436
5.3.2 Load Testing Checks ..................................................................... 437

Internal Use - Confidential

 5.4 Component Versioning ........................................................................ 437
5.5 Use of Extensions .............................................................................. 438
5.6 Deployment Procedures ....................................................................... 440
5.6.1 Standard Deployment Procedures ..................................................... 440
5.6.2 Important Deployment and Extension Concepts ..................................... 443
5.7 Automating Deployments with the AtomSphere API ..................................... 444
5.7.1 Overview ................................................................................... 444
5.7.2 Usage Notes and Limitations ........................................................... 444
5.7.3 Steps ........................................................................................ 445
6 Manage the Process ................................................................................. 447
6.1 Error Handling Framework.................................................................... 447
6.1.1 Process Error Handling Logic ........................................................... 447
6.1.2 Common Error Handling Logic .......................................................... 448
6.2 Operational Monitoring ....................................................................... 450
6.2.1 Integration Process Alerts ............................................................... 450
6.2.2 Execution History ......................................................................... 451
6.2.3 Purge History .............................................................................. 452
6.2.4 Local Runtime Monitoring ............................................................... 453
6.2.5 Cloud Runtime Monitoring .............................................................. 455
6.2.6 Integration Logs for Troubleshooting.................................................. 456
6.3 Functional Monitoring ......................................................................... 459
6.3.1 Overview ................................................................................... 459
6.3.2 Functional Monitoring Options ......................................................... 459
6.3.3 A Note about Execution History Retention and Purge Policies .................... 463
6.3.4 Considerations for Monitoring High Volume Low Latency Processes ............. 464
6.4 Event Framework .............................................................................. 465
6.4.1 Overview ................................................................................... 465
6.4.2 How the Event Framework Works ...................................................... 466
6.4.3 Consuming Events ........................................................................ 470
6.5 High-volume Troubleshooting ................................................................ 471
6.5.1 OutOfMemoryException errors ......................................................... 471

Internal Use - Confidential

 6.5.2 The Atom's memory efficiency ......................................................... 471
6.5.3 Repeated exception errors.............................................................. 473
6.5.4 Usage of all available memory on the machine ..................................... 473
6.5.5 If you still have problems with your Atom............................................ 475
6.6 Maintainability and Maintenance ............................................................ 475
6.6.1 Process Design & Build Documentation ............................................... 475
6.6.2 Process Execution Documentation ..................................................... 476
6.6.3 Process Notifications ..................................................................... 476
6.6.4 API Procedures ............................................................................ 478
6.6.5 Identify Who Will Monitor Executions and How...................................... 479
6.6.6 New Release Best Practices ............................................................. 480
7 Boomi Help and Support Options ................................................................. 482
7.1 Introduction ..................................................................................... 482
7.2 Quick Links ...................................................................................... 482
7.3 Community ...................................................................................... 483
7.3.1 Community Features ..................................................................... 483
7.3.2 Accessing the Community ............................................................... 484
7.3.3 Community Forums ....................................................................... 484
7.3.4 Knowledge Center ........................................................................ 484
7.4 User Guide ...................................................................................... 485
7.5 Contacting Technical Support................................................................ 485
7.5.1 Support Options........................................................................... 486
7.5.2 Severity Definitions and Response Times ............................................. 486
7.5.3 Accessing the Support Center and Creating Cases .................................. 487
7.5.4 Case Submission and Follow-Up ........................................................ 487
7.5.5 Best Practices for Contacting Support ................................................ 489
7.5.6 Status Site ................................................................................. 490
7.6 Professional Services .......................................................................... 490
7.6.1 Training and Certification ............................................................... 491
7.6.2 Consulting Services ....................................................................... 491
7.7 Customer Success .............................................................................. 492

Internal Use - Confidential

 7.8 Summary ........................................................................................ 492
8 Appendix - Atomsphere Best Practice Checklist ................................................ 492
8.1 Accountability .................................................................................. 492
8.2 Maintainability ................................................................................. 493
8.3 General .......................................................................................... 495
8.4 Performance .................................................................................... 496
8.5 Final Checks .................................................................................... 497
8.6 Process Development Checklist .............................................................. 498

Internal Use - Confidential

 Figures

Figure 1: User Management Menu ........................................................................ 24

Figure 2: Add/Maintain User Roles ....................................................................... 25
Figure 3: User Roles ........................................................................................ 29
Figure 4: Account Groups Page ........................................................................... 31
Figure 5: Account Groups Ex1............................................................................. 34
Figure 6: Account Groups Ex2............................................................................. 35
Figure 7: Account Groups Ex3............................................................................. 36
Figure 8: Add/Edit Role .................................................................................... 37
Figure 9: Privileges for a Role that Restricts Viewing Data .......................................... 38
Figure 10: Process Reporting Page ....................................................................... 38
Figure 11: Build Read Access Privilege Only ............................................................ 39
Figure 12: Atom Management Read Access ............................................................. 40
Figure 13: Setup >> Account Information ............................................................... 41
Figure 14: Atom Management >> Environments ........................................................ 42
Figure 15: Boomi AtomSphere ............................................................................ 52
Figure 16: Boomi Atom ..................................................................................... 54
Figure 17: The Molecule ................................................................................... 55
Figure 18: molecule Clustering ........................................................................... 57
Figure 19: Atom Cloud ..................................................................................... 60
Figure 20: Sample Process to Move Schedules ......................................................... 74
Figure 21: Map in the Process ............................................................................. 75
Figure 22: Containers Properties File .................................................................... 78
Figure 23: Changing Values in UI ......................................................................... 80
Figure 24: Basic Properties ................................................................................ 81
Figure 25: Advanced Properties .......................................................................... 82
Figure 26: Custom Properties ............................................................................. 83
Figure 27: Upload JAR #1 .................................................................................. 84
Figure 28: Upload JAR #2 .................................................................................. 84
Figure 29: Upload JAR #3 .................................................................................. 85
Figure 30: Upload JAR #4 .................................................................................. 85
Figure 31: Understanding JVMs ........................................................................... 87
Figure 32: Flow Control Shape > Threaded ............................................................. 88
Figure 33: Flow Control Threaded Example ............................................................ 89
Figure 34: Understanding Forked Process Execution .................................................. 90
Figure 35: Flow Control > Forked ........................................................................ 91
Figure 36: Flow Control Forked Example ............................................................... 91
Figure 37: Environment Properties..................................................................... 113
Figure 38: Account menu > Setup > Licensing ........................................................ 116

10

Internal Use - Confidential

 Figure 39: Licensing Window ............................................................................ 117
Figure 40: Environment Model .......................................................................... 125
Figure 41: Environment Model .......................................................................... 126
Figure 42: Environment Model .......................................................................... 128
Figure 43: Environment Model .......................................................................... 129
Figure 44: Environment Extensions via Atom Management ........................................ 130
Figure 45: Understanding Data Flows .................................................................. 146
Figure 46: Data Flow with Business Rules ............................................................. 148
Figure 47: Data Flow with Return Documents ........................................................ 149
Figure 48: Data Flow with Route Shape ............................................................... 151
Figure 49: Data Flow with Process Route ............................................................. 152
Figure 50: Data Flow with Flow Control ............................................................... 154
Figure 51: Data Flow with Try/Catch .................................................................. 156
Figure 52: Set Properties ................................................................................ 162
Figure 53: Using Groovy to Get and Set Dynamic Process Properties ............................ 163
Figure 54: Groovy Script ................................................................................. 163
Figure 55: Groovy Script ................................................................................. 163
Figure 56: Groovy Script ................................................................................. 164
Figure 57: Groovy Script ................................................................................. 164
Figure 58: Sample Message Body ....................................................................... 166
Figure 59: XML Testing ................................................................................... 168
Figure 60: Sample JSON.................................................................................. 168
Figure 61: Sample EDI/Flat File ........................................................................ 170
Figure 62: XML Data ...................................................................................... 174
Figure 63: Adding Cache Data .......................................................................... 176
Figure 64: Flat Files and Input Data ................................................................... 178
Figure 65: Flat Files and Input Data ................................................................... 179
Figure 66: Flat Files and Input Data 2 ................................................................. 179
Figure 67: Flat Files and Input Data 3 ................................................................. 180
Figure 68: Flat Files and Input Data 4 ................................................................. 181
Figure 69: Flat Files and Input Data 5 ................................................................. 182
Figure 70: Flat Files and Input Data 6 ................................................................. 183
Figure 71: Flat Files and Output Data 1 ............................................................... 183
Figure 72: Flat Files and Output Data 2 ............................................................... 183
Figure 73: Flat Files and Output Data 3 ............................................................... 184
Figure 74: Flat Files and Output Data 4 ............................................................... 184
Figure 75: Flat Files and Output Data 5 ............................................................... 184
Figure 76: Flat Files and Output Data 6 ............................................................... 185
Figure 77: Enforce Unique/Flat File ................................................................... 185
Figure 78: XML Profile Sample Data ................................................................... 186

11

Internal Use - Confidential

 Figure 79: Flat File Data and Simple Mapping ........................................................ 187
Figure 80: Output Data 1 ................................................................................ 188
Figure 81: Output Data 2 ................................................................................ 189
Figure 82: Output Data 3 ................................................................................ 189
Figure 83: Output Data 4 ................................................................................ 190
Figure 84: Output Data 5 ................................................................................ 190
Figure 85: Output Data 6 ................................................................................ 191
Figure 86: Output Data 7 ................................................................................ 192
Figure 87: Output Data 8 ................................................................................ 193
Figure 88: Output Data 9 ................................................................................ 194
Figure 89: Elements Defined by Types ................................................................ 195
Figure 90: Sample Data .................................................................................. 197
Figure 91: Output Data .................................................................................. 198
Figure 92: Add Qualifiers and Instances ............................................................... 198
Figure 93: Data ............................................................................................ 199
Figure 94: Array Sample Data ........................................................................... 200
Figure 95: Parent Array in Profile ...................................................................... 201
Figure 96: Matching Qualifiers .......................................................................... 202
Figure 97: Parent Element with Children in it ....................................................... 202
Figure 98: Instance Based on Qualifier ................................................................ 203
Figure 99: EDI Profiles ................................................................................... 204
Figure 100: Document Cache Data Flow .............................................................. 212
Figure 101: Adding Docs to the Cache ................................................................. 213
Figure 102: Multiple Name Elements Example ....................................................... 215
Figure 103: Script ......................................................................................... 218
Figure 104: User Defined Functions .................................................................... 224
Figure 105: Extensions Window ......................................................................... 225
Figure 106: Test Extensions ............................................................................. 227
Figure 107: Connection Settings ........................................................................ 227
Figure 108: Cleanse Shape .............................................................................. 229
Figure 109: Cleanse Shape Window .................................................................... 229
Figure 110: Pull Execution Properties ................................................................. 231
Figure 111: JSON Profile ................................................................................. 240
Figure 112: JSON Profile 2............................................................................... 241
Figure 113: JSON Profile 3............................................................................... 242
Figure 114: Elements Retained in Each Document .................................................. 243
Figure 115: Allowing for Batching ...................................................................... 244
Figure 116: 850 Profile ................................................................................... 245
Figure 117: Splitting on PO1 Segment ................................................................. 245
Figure 118: XML Data ..................................................................................... 246

12

Internal Use - Confidential

 Figure 119: Combining Arrays ........................................................................... 248
Figure 120: Combining Arrays ........................................................................... 249
Figure 121: Combining Arrays ........................................................................... 250
Figure 122: Tracked Field ............................................................................... 253
Figure 123: Add Tracked Fields to Connectors ....................................................... 253
Figure 124: Edit Tracking Tab .......................................................................... 254
Figure 125: Edit Tracking Tab(2) ....................................................................... 254
Figure 126: Document Details Table ................................................................... 255
Figure 127: Show Documents ........................................................................... 256
Figure 128: Show Documents that Match Tracked Fields ........................................... 256
Figure 129: Example Sub-process ...................................................................... 264
Figure 130: Sub-process Example ...................................................................... 265
Figure 131: Multi-source Document Call .............................................................. 265
Figure 132: Calling Process .............................................................................. 266
Figure 133: Multi-source Document Call with Return ............................................... 266
Figure 134: Two-process Comparison >> .............................................................. 267
Figure 135: Orders Sub-process ......................................................................... 271
Figure 136: Employees Sub-process .................................................................... 271
Figure 137: Import File Record Types ................................................................. 272
Figure 138: Process Routing ............................................................................. 273
Figure 139: Main Process ................................................................................ 273
Figure 140: Manipulate Properties ..................................................................... 274
Figure 141: Process Route Shape ....................................................................... 274
Figure 142: Route Parameters .......................................................................... 275
Figure 143: Exception Shape ............................................................................ 275
Figure 144: Component Type List ...................................................................... 276
Figure 145: View Logs .................................................................................... 276
Figure 146: Joining Data from Multiple Sources ..................................................... 285
Figure 147: Mapping Data ............................................................................... 286
Figure 148: Lookup ....................................................................................... 287
Figure 149: Set Propeties > Cache Lookup ............................................................ 288
Figure 150: Parameter Values .......................................................................... 288
Figure 151: Use Temporary Array ...................................................................... 289
Figure 152: Load From Cache Shape ................................................................... 290
Figure 153: Notify shape ................................................................................. 291
Figure 154: Data Flow .................................................................................... 292
Figure 155: Document Cache Lookup in the Map .................................................... 293
Figure 156: Process Options ............................................................................. 299
Figure 157: Process Options ............................................................................. 300
Figure 158: Amazon S3 Connection .................................................................... 301

13

Internal Use - Confidential

 Figure 159: Connection Steps ........................................................................... 302
Figure 160: Choose Property ............................................................................ 304
Figure 161: Set Property ................................................................................. 305
Figure 162: File in Bucket ............................................................................... 305
Figure 163: Set Additional Property ................................................................... 306
Figure 164: File Path ..................................................................................... 306
Figure 165: Set Additional Property ................................................................... 307
Figure 166: S3 Create .................................................................................... 308
Figure 167: S3 Create after Import .................................................................... 308
Figure 168: S3 Process ................................................................................... 309
Figure 169: Message Shape .............................................................................. 310
Figure 170: Email Produced ............................................................................. 310
Figure 171: Uploaded File in Bucket ................................................................... 311
Figure 172: Basic Process ................................................................................ 312
Figure 173: Set Property ................................................................................. 313
Figure 174: File Path ..................................................................................... 313
Figure 175: Retrieve Directory Listing in "02" ........................................................ 314
Figure 176: S3 GET Process.............................................................................. 316
Figure 177: Connector Shape Parameters............................................................. 316
Figure 178: Process Steps................................................................................ 317
Figure 179: Message Shape .............................................................................. 318
Figure 180: Salesforce Process.......................................................................... 346
Figure 181: Local Atom Reference Architecture ..................................................... 351
Figure 182: Outbound Reference Architecture....................................................... 352
Figure 183: Send IDoc to SAP ........................................................................... 355
Figure 184: Receive/Listen for IDoc from SAP ....................................................... 357
Figure 185: Executing a BAPI over RFC ................................................................ 358
Figure 186: Calling a SAP Web Service ................................................................ 359
Figure 187: AtomSphere Workday SOAP Model ....................................................... 360
Figure 188: RaaS Workday Model ....................................................................... 364
Figure 189: Message Queuing ........................................................................... 366
Figure 190: Outbound Subscriptions ................................................................... 367
Figure 191: XML Data ..................................................................................... 370
Figure 192: Single Value ................................................................................. 371
Figure 193: XSD Document .............................................................................. 372
Figure 194: Two "Note" Elements ...................................................................... 373
Figure 195: Different Prefixes .......................................................................... 374
Figure 196: URI as an Identifier ........................................................................ 375
Figure 197: Flat File Contents .......................................................................... 376
Figure 198: Data Positioned ............................................................................. 377

14

Internal Use - Confidential

 Figure 199: SQL Statement that Retrieves a Set of Fields ......................................... 378
Figure 200: SQL Statement that Retrieves a Set of Fields ......................................... 379
Figure 201: X12 Document .............................................................................. 381
Figure 202: Loops ......................................................................................... 382
Figure 203: JSON .......................................................................................... 384
Figure 204: Object ........................................................................................ 385
Figure 205: Array ......................................................................................... 385
Figure 206: Array of Objects ............................................................................ 386
Figure 207: 3 Elements in an Array .................................................................... 387
Figure 208: Example >> Last Modified Date .......................................................... 389
Figure 209: Increment Poll (ETL Style) ................................................................ 390
Figure 210: Real Time (Synchronous, Record Payload) ............................................. 391
Figure 211: Real Time (Synchronous, Payload[Msg=ID]) ............................................ 392
Figure 212: Real Time (Asynchronous, "Fire & Forget") ............................................ 393
Figure 213: Real Time (Asynchronous, Callback) .................................................... 394
Figure 214: Accumulate-then-Process ................................................................. 395
Figure 215: Decoupling Processing and Transmission ............................................... 396
Figure 216: Two Way > Secondary to Master ......................................................... 404
Figure 217: Two Way > Master to Secondary ......................................................... 404
Figure 218: Flow Representing First Half of Two-way Sync ........................................ 410
Figure 219: Logic Breakdown of a Process ............................................................ 411
Figure 220: Conceptual Web Service Listener Process .............................................. 411
Figure 221: Simple Test Harness ....................................................................... 412
Figure 222: Test Set Against Known Values ........................................................... 413
Figure 223: Test Harness Process ...................................................................... 414
Figure 224: Mock Endpoint Web Service .............................................................. 415
Figure 225: GET HTTP .................................................................................... 418
Figure 226: POST HTTP .................................................................................. 418
Figure 227: HTTP Response ............................................................................. 419
Figure 228: HTTP Client Operation > Get ............................................................. 423
Figure 229: HTTP Client Operation > Send ........................................................... 424
Figure 230: Resource Path............................................................................... 426
Figure 231: Set Properties ............................................................................... 427
Figure 232: Data Flow .................................................................................... 427
Figure 233: X12 850 Document ......................................................................... 432
Figure 234: Revision History ............................................................................ 437
Figure 235: Set Extensions .............................................................................. 438
Figure 236: Extensions >> Connection Settings ...................................................... 439
Figure 237: Test Extensions ............................................................................. 439
Figure 238: Deploy Latest Revision .................................................................... 441

15

Internal Use - Confidential

 Figure 239: Processes .................................................................................... 442
Figure 240: Deployment Confirmation................................................................. 442
Figure 241:Process Error Handling .................................................................... 448
Figure 242: Error Message ............................................................................... 448
Figure 243: Settings ...................................................................................... 451
Figure 244: Event Framework Graphic ................................................................ 466
Figure 245: Enable Events in Notify Shape............................................................ 470
Figure 246: Script to Show Notifications .............................................................. 477

16

Internal Use - Confidential

 Tables

Table 1: Privileges and Associated Roles................................................................ 30

Table 2: Property Structure ............................................................................... 79
Table 3: Licenses and Scenarios ........................................................................ 110
Table 4: License Scenario ............................................................................... 113
Table 5: Three Environment Table..................................................................... 124
Table 6: Four Environment Table ...................................................................... 125
Table 7: Five Environment Table ....................................................................... 127
Table 8: Five Environments with Separate Tracks ................................................... 128
Table 9: Phase 1 .......................................................................................... 135
Table 10: Phase 2 ......................................................................................... 136
Table 11: Phase 3 ......................................................................................... 137
Table 12: Phase 4 ......................................................................................... 137
Table 13: Phase 5 ......................................................................................... 137
Table 14: Connectors, Get/Set Properties............................................................ 165
Table 15: Special Characters............................................................................ 168
Table 16: Message Shape and Single Quotes ......................................................... 171
Table 17: Message Shape and Variables ............................................................... 172
Table 18: Date/Time Formatting ....................................................................... 205
Table 19: Date Formats .................................................................................. 206
Table 20: Time Formats ................................................................................. 206
Table 21: Attributes ...................................................................................... 207
Table 22: Formats ........................................................................................ 208
Table 23: Number Formats .............................................................................. 209
Table 24: Number Formats .............................................................................. 209
Table 25: Percentage .................................................................................... 210
Table 26: Writing Data ................................................................................... 210
Table 27: Read Data ...................................................................................... 211
Table 28: Write Data ..................................................................................... 211
Table 29: Cross Reference Table ....................................................................... 216
Table 30: Matches Regular Expression................................................................. 217
Table 31: String Related Functions .................................................................... 219
Table 32: Numeric Related Functions ................................................................. 220
Table 33: Repeating Element Functions .............................................................. 221
Table 34: Date/time Functions ......................................................................... 223
Table 35: Lookup Functions ............................................................................. 223
Table 36: XML Data Combos ............................................................................. 246
Table 37: Parameters .................................................................................... 303
Table 38: SAP Approach and Considerations ......................................................... 350

17

Internal Use - Confidential

 Table 39: Monitoring Techniques ....................................................................... 453
Table 40: Monitoring Techniques ....................................................................... 456
Table 41: Logs for Troubleshooting .................................................................... 456
Table 42: Process Monitoring Options and Best Practices .......................................... 463
Table 43: Four Types of Events ......................................................................... 468
Table 44: Event Channels................................................................................ 471
Table 45: Records and Usage ........................................................................... 473
Table 46: Resources and Links .......................................................................... 483
Table 47: Coverage and Support ....................................................................... 486
Table 48: Severity and Response ....................................................................... 487

18

Internal Use - Confidential

 1 Document Control Information

1.1 Revision History

Revision # Author Date Comments

1 Mike Hiltwine, Matt 08/19/2019 Initial version

Krebsbach

19

Internal Use - Confidential

 2 Introduction

The intention of this guide is to provide general development standards and best practices for
integrating systems and applications within an organization using the Boomi technology.

2.1 Overview
Standardization is a way for an organization or enterprise to help reduce the risk of
uncontrolled and poorly developed processes impacting the day-to-day operations where the
impact affects users, data quality, and revenue for the organization.

This guide will provide a list of standard integration patterns and techniques commonly used
and implemented by the Boomi platform. Additionally, it will provide guidelines that
Integration Developers can reference and utilize to maintain a best practice format while
creating, deploying, and maintaining processes.

20

Internal Use - Confidential

 2.2 Concepts

• Process – An AtomSphere integration process.

• Execution – An instance of a process running on a runtime. It is single-threaded.
• Java Virtual Machine (JVM) - A single operating system process, running on the Java
platform.
• Cluster – One or more JVMs working together as a logical Molecule or Cloud.
• Node – A single Molecule or Cloud JVM running as part of a cluster.
• Atom – A single-tenant, single-node AtomSphere runtime engine.
• Molecule – A single-tenant, multiple-node AtomSphere runtime engine.
• Cloud – A multiple-tenant, multiple-node AtomSphere runtime engine.
• Hard Disk or Storage – The persistent, long-term data storage available on the
server/machine.
• Heap – The transient working memory for a JVM. This is a subset of memory that is owned
and managed by a Java process.
• Garbage Collection – The algorithm used by Java to manage the Heap usage, constantly
running in the background.
• Thread – An executing code path within a JVM.
• Memory or RAM – The transient working memory available for the CPU.
• Forked Execution – Node executes each process in separate JVM, forked each time that
another process executes.
• Threaded Execution – An executing code path within a JVM. All threads share the same
heap space.
• Computer – A single computer, which can be a physical or virtual machine.
• CPU – A processor in a computer, including physical, not virtual, cores.
• Data - Actual document data files used during execution and subsequent document data
viewer.
• Counter - Upon first use of a counter parameter or trading partner.
• Logs - Daily container logs, http server logs, local user logs.
• Userlib - Contains additional libraries (jars) for some types of connectors, custom scripts,
etc.
• Message - Staging area for messages sent back to the platform.

21

Internal Use - Confidential

 • Queue - Persisted Atom Queue contents. Does not contain actual document data, instead
contains pointers to document data residing in data directory.
• Doccache - Temp directory for Document Cache component.
• Tmp - the general temp directory, i.e. “scratch space” for just about anything. tmpdata
is actually the “data” directory used when “immediate purge” is enabled for an account.
• Latency – The delay from input into a system to desired outcome.
• Connection – Represents a specific instance/location of an application or data source.
• Connector - The adapter that moves data in and out of integration processes.

22

Internal Use - Confidential

 3 Getting Started

3.1 General Account Setup and Administration

Within the AtomSphere Integration platform, users have the option of adding additional users
to work within the owner’s development area.

Additional users are given access to the AtomSphere Integration platform through Setup >
User Management. Users are granted different levels of access through a role-based access.
Roles are collections of individual privileges that control access to specific functionality
within the application.

Integration provides several default roles out of the box or you can create custom roles. Users
may be assigned multiple roles in which they can get a multitude of different privileges.

You can add any number of users in your Integration account.

3.1.1 User Access/Create Additional Users

Only users with a valid user account can access the Integration platform. To create a user, go
to the account Setup section of the Integration platform, which can be found in the upper-
right section of the user interface (UI), and click the drop-down arrow to the right of the
message icon. Click Setup > User Management to access the “Users” administration section.

23

Internal Use - Confidential

 Figure 1: User Management Menu

Click the plus icon in the Users menu. This will populate a “Add/Maintain User Roles” Menu.
Fill out the menu and select the appropriate role for the user. Select OK and the user will
automatically be saved.

24

Internal Use - Confidential

 Figure 2: Add/Maintain User Roles

3.1.2 Controlling User Access to Your Account

User Management

The primary method for managing user access to your account is through User Management.
Here you create new users and assign roles to control access to various functionality within
the application.

To configure User Management, go to Account menu > Setup > User Management.

For more information see Understanding User Management and Custom Roles.

25

Internal Use - Confidential

 Boomi Support Access

By default accounts permit access by the Boomi support team. Access to your account can be
helpful or necessary when working with support to troubleshoot errors and review
configuration.

Access is controlled by selecting the role that will be applied to support personnel accessing
your account. By default this is the Administrator role but you can choose any standard or
custom role available in your account. For example you could select a "read only" role that
allows support personnel to "look but not touch" setup within your account. To disable support
access entirely, select "--No Access--".

To configure support access, go to Account menu > Setup > Account Information > Support
Access Role.

Single Sign On

Requires the Advanced User Security feature.

You can enable Single Sign On (SSO) for your Integration to use an external identity provider
to authenticate users. Users must still be configured within Integration's User Management
because the platform performs the authorization (what they are allowed to do once in
AtomSphere). However, upon login, users are first authenticated against an external identity
provider of your choosing before accessing Integration.

To enable and configure Single Sign On for your account, go to Account Menu > Setup > SSO
Options.

Using SSO can be a convenience to end users who may be accessing Integration from a
company portal or another application because they do not have to manually log into
Integration and remember their credentials. SSO is beneficial to organization administrators
by allowing them to manage users' access to Integration from their central identity provider.
For example, if an employee leaves the company, their access can be terminated from the

26

Internal Use - Confidential

 central identity provider instead of having to remember to remove them from Integration
explicitly.

Considerations:

• Once SSO is enabled, all users except those with the Administrator role will not be
able to log into Integration directly.
• Administrators will continue to log into Integration directly.
• User roles and authorization are still managed within Integration.
• Integration SSO uses SAML 2.0.
• There are special considerations for users with access to multiple accounts
• See How to Use Single Sign-on for more considerations

See Single Sign-On with SAML Authentication for more information.

Account Groups and Child Accounts

Requires the Account Groups feature, typically enabled for partner and parent-child account
hierarchies.

Partners with child Integration accounts will additionally use Account Groups to manage user
access to child accounts. The Account Groups feature is only enabled in the parent account.
You can define any number of Account Groups that represent logical groupings of child
accounts. Users are added to an Account Group which grants them access to the child
accounts within that group.

It may be helpful to think of it like this: User Management controls access to the parent
account. Account Groups control access to all the child accounts.

27

Internal Use - Confidential

 Considerations

• Account Group users are separate from User Management users. Account Group users
do not have exist as User Management users.
• To add the same user as both a User management and Account Group user, simply use
the same user ID/email.
• Roles are assigned per user per Account Group. This means two users in the same
group can have different roles, or the same user can have different roles in different
groups.

As a best practice to facilitate user administration, manage access for your partner users to
all child accounts through Account Groups in the parent account. Do not add partner users
directly to the individual child accounts through each child account's User Management.
However, if end customers require access to their individual child account, it is best to
manage those customer users with through the child account's User Management.

3.1.3 User Roles

User Roles and Privileges, Boomi User Guide

When setting up user accounts, you can decide to use the default roles provided in the
platform to assign a new user, or you can create custom roles. The privileges associated with
these default roles are granted to the user. You can also set up a user with a combination of
roles to provide extra privileges; note that the role with greater privileges takes precedence
over the other roles.

28

Internal Use - Confidential

 Figure 3: User Roles

3.1.4 Privileges

Privileges are rights reserved to a role that allow a user to access or perform actions in a
specific area of the Integration platform. Privileges have been defined by Boomi at a level
that allows for fine-tuning of custom roles. For example, you can restrict specific roles from
viewing data on the Boomi platform via the User Management interface (figure 1). Or, you
could restrict a role from having access to the Process Deployment privilege so that a user
couldn’t deploy changes. Similarly, you can control the access to the Build, Deploy, and
Manage tabs by adding or removing privileges. (Custom roles section).

Note: The privileges associated with the default roles can’t be modified. To find the list of
privileges associated with default roles, choose the roles in the Inherits Role dialog box
(figure 4).

29

Internal Use - Confidential

 Listed below are possible roles and their associated privileges:

Table 1: Privileges and Associated Roles

3.1.5 Account Groups

Note: If you use a single Boomi account, account groups are not available and not required.
Master accounts and the Account Groups tab are for administrators of a master account who

30

Internal Use - Confidential

 manage multiple accounts. If you manage multiple accounts, you must contact Boomi to have
Master accounts and the Account Groups tab enabled.
The Account Groups tab, located on the Setup page, enables administrators of master
accounts to define and manage relationships between various Boomi AtomSphere
platform resources and the Boomi accounts that use those resources.

Figure 4: Account Groups Page

Platform features and resources are enabled for accounts, not for individual users. Users have
access to an account and therefore have access to the account's features and resources.

Partners can use account groups to manage:

• a large number of customer accounts and to distribute access rights. For example, a partner
manages 1000 Boomi accounts and has five different support teams. The master account
administrator can create an account group for each of the five support teams and then assign
200 accounts to each account group.
• partner users and direct customer users. Setting up account groups allows your partner user
base to have strategic access to specific groups of accounts where some users have limited
versus administrative access. The direct customers can still enable users and roles by using
the User Management tab.

31

Internal Use - Confidential

 • a partnership where systems integrators are used. The master account administrator can
create an account group for each systems integrator and assign the appropriate accounts to
each account group.

Types of user access:

If you use account groups, there are two types of user access:

• Direct Access — The user is assigned role(s) directly on an account by using the User
Management tab.

• Account Group Access — The user is assigned role(s) through one or more account groups by
using the Users in Group list.

Users added to the Account Groups tab in the Users in Group list are independent from the
users added directly to the User Management tab. If two instances of the same user exist for
both Direct access and Account Group access, the Direct access user role and privileges takes
precedence when that user logs into the account.

Note: There is no more inheritance-based user access. You can no longer switch into an
account based on having access to a parent account. Account groups take the place of a
parent/child account relationship.

Support access role:

On the Account Information tab, there is a Support Access Role field that allows direct
customer users, enabled under User Management, to control the level of access that outside
users should have upon logging into the account. If a partner user is enabled within an
account group, the Support Access Role setting takes precedence and limits the account
group user to this setting. This is only applicable when the partner user is not enabled as a
Direct Access user in the account.

User Roles in Account Groups

Account group access is controlled by giving users access to the account group and by
assigning them roles within the account group.

• Multiple users can be assigned to an account group and each user can have a different role
within the group.

32

Internal Use - Confidential

 • A user can have multiple roles within an account group.

• A user can be a member of multiple account groups and can be assigned a different role in
each account group.

Note: Users must log in to the Boomi AtomSphere platform before they can be added to an
account group.

Account Groups Example

This example shows a Documentation master account, who is a partner, managing four
customer accounts.

The four customer accounts are:

• Doc ABC Company

• Doc DEF Company

• Doc GHI Company

• Doc JKL Company

The partner has four users that need to share the development and support duties for these
customer accounts. The users are:

• ServUser1@boomi.com

• ServUser2@boomi.com

• ServUser3@boomi.com

• ServUser4@boomi.com

The partner's business scenario is:

33

Internal Use - Confidential

 • All four users need to have the Support or Production Support role so that they can effectively
troubleshoot and retry deployed processes.

• The users ServUser1 and ServUser2 need to have the Standard User role so that they can
effectively develop and test processes in the Doc ABC Company and Doc DEF Company
accounts.

• The users ServUser3 and ServUser4 need to have Standard User role so that they can
effectively develop and test processes in the Doc GHI Company and Doc JKL Company
accounts.

Two account groups can be added to handle the different Services teams and to distribute
account access.

The users ServUser1 and ServUser2 are added and granted the Standard User role so that they
can handle all development and managements tasks for the Doc ABC Company and Doc DEF
Company accounts.

Figure 5: Account Groups Ex1

34

Internal Use - Confidential

 The users ServUser3 and ServUser4 are added and granted the Standard User role so that they
can handle all development and management tasks for the Doc GHI Company and Doc JKL
Company accounts.

Figure 6: Account Groups Ex2

In the All Accounts account group, all four users are added with the Support or Production
Support role so that each user gets a union of the roles and privileges that are distributed
across account groups.

35

Internal Use - Confidential

 Figure 7: Account Groups Ex3

3.1.6 Custom Roles and Privileges

If the default roles in the platform don’t meet your organization’s needs, you can create
custom roles and assign them to a user. These roles can then be assigned privileges that are
in line with your company’s established development lifecycle practice. All privileges must
manually be assigned to custom roles. Additionally, when you’re creating custom roles, the
platform lets you assign inherited privileges associated with existing standard or custom roles.
Additional privileges can then be granted, if required. Within the User Management menu,
select the Custom Roles tab. Select the plus icon to populate the Add/Edit Role menu.

36

Internal Use - Confidential

 Figure 8: Add/Edit Role

Custom roles for restricting access to data and documents

You can allow users to view process logs and/or execute and rerun processes but not view the
resulting data and documents from the processes.

For example, a person who is responsible for running processes that manage personnel or
financial data needs to run processes and ensure that they execute successfully. However,
they should not have access to sensitive data that flows through the processes.

The “View Data” privilege allows users to view data and documents on the Process Reporting
page. All user roles delivered with Boomi Integration have the View Data privilege turned on.
Therefore, you must create a custom role that allows users to view process execution activity
and logs and/or to execute processes on the Process Reporting page, but it restricts them
from viewing data.

Note: If you set up a custom role in a master account, that role can be shared with other
accounts through account groups.

The privileges for a role that restricts users from viewing data should be set as follows:

37

Internal Use - Confidential

 Figure 9: Privileges for a Role that Restricts Viewing Data

Users who are assigned a custom role with these privilege settings have access to the Process
Reporting page. Accessed through the Manage drop down >> Process Reporting.

Figure 10: Process Reporting Page

However, if users select a process and view the document results on the bottom of the page
they do not see the following options:

• View Document on the Inbound Data and Outbound Data tabs

• Run Document in Test Mode option on the Inbound Data tab
• Re-run Document if the Execute privilege is off on the Inbound Data tab

Custom roles for giving read-only access to the Build page

You can allow users to view components but not create, edit, copy, delete, or save
components.

38

Internal Use - Confidential

 For example, your Support group needs to view components to assist users with
troubleshooting. However, Support should not be able to create or change any components.
Another example is a partner who wants to give your customers the Build Read Access
privilege to your account so that they can copy components but not change them.

The Build Read Access privilege gives users read-only access to the Build page. You can create
a custom role that includes the Build Read Access privilege and assign it to these users.

Note: If you set up a custom role in a master account, that role can be shared with other
accounts through account groups. For more, access the User Guide here: Controlling User
Access, Boomi User Guide

Users who are assigned a custom role with the Build Read Access privilege turned on see the
following:

Figure 11: Build Read Access Privilege Only

Note: If a user has the Build Read and Write Access privilege AND the Build Read Access
privilege turned on, the Build Read and Write Access privilege adds the ability to create, edit,

39

Internal Use - Confidential

 copy, delete, and save components. In other words, the user has full access to all actions on
the Build page.

When you replace a user’s “Build, Read, and Write Access” privilege with the “Build Read
Access” privilege, their privileges change immediately. If the user is logged into the
Integration platform when you make this change, and if the user tries to change a component,
they are unable to save it even though they can still see the Save buttons on screen. Instead,
the user receives an error message. The user must log out and log back in to see the changes
in the user interface.

Custom roles for giving read-only access to Atom Management

You can allow users to view but not modify the properties and settings of Atoms, Molecules,
and Atom Clouds on the Atom Management page.

For example, your Support team needs to view the Release Control Scheduling settings for an
Atom but does not need to change those settings.

The Atom Management Read Access privilege gives users read-only access to the Atom
Management page. You can create a custom role that includes the Atom Management Read
Access privilege and assign it to these users.

Note: If you set up a custom role in a master account, you can share that role with other
accounts through account groups.

Figure 12: Atom Management Read Access

40

Internal Use - Confidential

 Note: If a user has the Atom Management privilege and the Atom Management Read Access
privilege turned on, the Atom Management privilege adds the ability to modify properties and
settings. Users with these privileges have full access to all actions on the Atom Management
page.

3.1.7 Boomi Support Access

In Setup > Account Information, you can manage the Boomi support teams to access your
account. You can choose to revoke access completely or grant access as a specific user role.

Figure 13: Setup >> Account Information

Note that granting access allows the support team to more easily assist with issue
troubleshooting if needed.

3.1.8 Restricting Environment Access

It is a best practice to establish separation of duties and not allow all Integration users
complete access to all environments. For example, developers should be able to access the

41

Internal Use - Confidential

 Build tab, deploy to and configure extensions for a “dev” environment/Atom, but should not
be able to view or modify production environment/Atom. Similarly, a production support user
should be able to view execution results and perhaps re-run failed documents in the
production environment but should not have access to build to deploy.

This type of access control is configured by restricting roles to one or more environments
within the Atom Management screen. By default, all roles have access to all environments,
however once a role is explicitly assigned to an environment, users will only be able to see
that environment and perform privileges within that environment as defined by the role.
Select the Manage dropdown menu then select Atom Management within the dropdown. Next
select the environment you wish to customize. Figure 5 shows the Production menu. In the
configuration section you can customize the “Roles with Access”.

Figure 14: Atom Management >> Environments

Restricting roles to environments has implications in several areas of the application

including:

• Dashboard
• Deploy tab
• Process Reporting
• Atom Management

42

Internal Use - Confidential

 • AtomSphere API

Environment restrictions do not apply to the Build tab.

It is possible for a user to be assigned to multiple roles and those roles to be assigned to
different environments. In this situation, the actions a user can perform within a given
environment will vary based on the role.

For example, consider the following scenario:

• “Developer” role includes privileges Build Read and Write, Deploy, Execute Process,
and Process Reporting.
• “Developer” role is assigned to the “DEV” environment.
• “Support” role includes privileges Process Reporting and Dashboard.
• “Support” role is assigned to the “PROD” environment.

A user is assigned both the “Developer” and “Support” roles. The user will be able to do the
following:

• In Build, view and modify all components.

• In Deploy, view and deploy to the DEV environment only.
• In Dashboard, view the PROD environment only.
• In Process Reporting, view execution history for both DEV and PROD but execute
processes in DEV only.

Note: Be aware that the Environment Management privilege grants the ability to view and
modify all environments within the Atom Management screen. To only permit viewing and
modifying of a specific environment, configure the role to only include the Atom Management
privilege.

3.1.9 Restricting Folder Access in Build

Requires the Advanced User Security feature.

In the Build console, user access can be restricted per folder. The optional folder permissions
configuration can limit the user roles allowed to edit components within that folder. It does
not hide the folder from view or prevent viewing or copying components within the folder.

43

Internal Use - Confidential

 Consider using folder restrictions for the following situations to avoid inadvertent
modifications by general users:

• Restrict folders containing common/reusable components especially connections to

"lead developers"
• Restrict folders containing template/example processes to "lead developers"
• Restrict folders for different projects to the respective development teams
• The user's role must have the "Build Read Access" and/or "Build read and write access"
privilege to be able to view the Build console in general.
• The user's role must have the "Build Read and Write Access" privilege to potentially be
able to modify components. If it only has "Build Read Access" all components will be
read only regardless of folder permissions.
• If a folder has no role restrictions, any user with Build Read and Write Access privilege
can view and modify components within that folder. This is the default permissions for
new folders.
• If a folder has role restrictions, only users with that role may modify components with
that folder. The components will be read only for all other user roles.
• Only users with the Administrator role can manage folder permissions. Administrators
do not automatically have Write Access to restricted folders, however they can
manage the permissions to grant themselves access.

Folder restrictions only apply to the Build tab Component Explorer and do not apply to other
tabs including Deploy, Atom Management, and Process Reporting.

3.1.10 Security FAQ’s

Can you please further elaborate on the usage and storage of the
Private Key, and why this key is stored in the Boomi Data Center?

An account is associated with one public/private key pair. Connector passwords are encrypted
and stored using the account’s public key, and we use the account’s private key to decrypt
them at runtime. Processes are run from the data center. So, the private key must be
available to us when we run processes.

44

Internal Use - Confidential

 Can you please provide the Boomi user account password strength
requirements, e.g., minimum length, numbers, letters, symbols, caps,
etc.?

This can be configured by the account admin. The Password Policy tab on the Setup page is
used to establish rules for passwords used by users of the account. You can turn on or off any
of the password rules.

If you change the account's password policy and there are already users of the account whose
passwords may or may not match the new policy that you put into place, the next time that
they log into the Boomi platform they will see a message informing them that the password
policy has changed. The message lists the new rules that are in place and informs users that
they must enter their current password as well as a new password that conforms to the new
policy. If a user's current password happens to match the new policy, they will be able to
continue to use it by entering the current password in all of the fields.

Does Boomi implement user password expiration on a periodic

basis? If so, what is this period of time before a password will expire /
needs to be changed?

This can be configured by the account admin. The following rules are available when you set
a password policy for an account. You can select none, any or all of these rules:

• Passwords expire every 90 days.

• Passwords must have a minimum length.
• Passwords must contain characters from at least two of these groupings: alpha,
numeric, and special characters.
• Passwords must not contain sequences of three or more characters from the user ID.
• Password must not match any of the previous four passwords.
• Passwords must not contain a sequence of two or more characters more than once,
e.g. a12x12.

45

Internal Use - Confidential

 Is there a lockout policy after a certain amount of incorrectly
entered user passwords?

After 6 incorrect login attempts, the user is locked out and the account administrator will
receive an alert email indicating what account was attempting to be logged into
unsuccessfully.

Does Boomi retain a log that captures user login attempts / fails /
successes for a period of time?

This is logged as part of the general platform logs. This information is not publicly available
and is managed by the Boomi team.

When Boomi updates are available, does the Atom automatically

install these or does it prompt the user to install the update?

By default, the Atom will automatically download and install platform updates on the release
date. However, the Release Control feature allows you to apply new Atom (or Molecule or
Cloud) and connector updates at any time during the two weeks before the full release. This
allows you to select a time within that window that’s convenient for you to get and test the
updates. The pending updates are applied per Atom. You can update your “Test” Atoms and
test the changes against your processes before the full release. If you are happy with the test
results you can apply the changes to your “Production” Atoms and move the updates into your
production environment before the full release.

On the full release date all users will receive enhancements and user interface changes. Users
who have not adopted the Atom and connector updates in advance will get those updates as
well.

Boomi AtomSphere Platform users are located all over the world. They work in many different
industries. Their business hours vary. They can have Integration processes running at any time
of the day or night. Therefore, no particular date, day of the week, or time of day will be
convenient for everyone to receive a new release. Boomi realizes this and wants to minimize
the impact of new releases on your business.

A detailed Q&A on Release control is available here: Release Control

46

Internal Use - Confidential

 Are Java Platform (e.g., JRE) updates part of the periodic updates
that the Atom recognizes / is alerted of, even if the Atom is installed on
a Windows server?

No, in general Java updates must be applied manually by the customer. They are not included
as part of the regular AtomSphere release.

Can you please confirm that once an Atom is installed locally on a

server, it will initiate all communication with the Boomi Data Center
and it will not open any ports on the server which the Atom is deployed
to?

Yes, all communication is initiated by the Atom. The Atom will not attempt to change any
port settings you have configured on your network. Traffic is defaulted to outbound traffic
only, but with the Web Services Connector, the Atom can listen to incoming traffic if the port
is opened.

3.1.11 Common Platform and User Account Related Issues/FAQ’s

Why am I getting "Login failed, please check credentials and try

again"?

• Your training account expired already. Please follow this, What do I do if my master
Training Account expired? to restore your account
• Your username/password combination is wrong, try resetting your password

How to enable a feature?

Please contact your Boomi account manager or sales representative to enable the feature
and/or for more details of cost and information. Common features include:

• Advanced User Security - to create custom roles and enable the SSO options tab
• Version Control - for revision history
• Component Locking - lock components so other non-admin users cannot edit it
• API Management - more API types (API type can be Advanced) and options, API
management, CORS options

47

Internal Use - Confidential

 • B2B and EDI - enable Trading Partner, AS2, EDI, and B2B capabilities
• Any features or components not included in your edition of the platform

You will have to contact your Boomi account manager also for:

• Molecule and cloud licenses

• Atom workers
• Connector licenses

How to reset my password?

It would be best if you reset your password when you are available to receive the email since
the password link only lasts for 30 minutes.

1. Go to https://platform.boomi.com/
2. Type in your username, it should be your email address
3. Click "Help. I forgot my password"
4. This link is only valid for 30 minutes

Why do I keep getting "Unauthorized" error messages when trying

to switch to a training or another account?
Most likely you have SSO enabled on another account so that will prevent you from logging
into the platform directly. If you try to do it, it'll give unauthorized error messages. To log
into a non-SSO enabled account, you will need to use the account ID in the following
URL: https://platform.boomi.com/#build;accountId=<<account_id>>;

It'll be best to bookmark that other account if you plan to use it frequently. If you are an
administrator, you'll be able to log into the platform normally.

48

Internal Use - Confidential

 I keep resetting my password but I'm not receiving an email

If you are resetting your password for a trial or training account and not receiving any emails,
your account is most likely expired, please follow this, What do I do if my master Training
Account expired? to restore your account. Once restored you should be able to login again, if
not reset your password.

Why am I seeing a "not enrolled in active account" message when

logging in?

You are most likely not part of any account anymore. Your user account was removed,
whether by accident or not please contact your Administrator on that account.

I just made a training account and received the email to login but I
cannot login?

The account is most likely still being created. Wait up to 30 minutes. If you are still not able
to login, contact support@boomi.com.

My training account is about to expire, how do I extend it?

If you have a training account, your account ID will include "training" in it. Enroll in any
course on the train.boomi.com site to extend your expiration date. You can also complete
and enroll in the Boomi Essentials course to extend the expiration date. You will be able to
complete and re-enroll in the Boomi Essentials course as many times as you want. You can
also follow this, What do I do if my master Training Account expired?

49

Internal Use - Confidential

 My trial account is about to expire, how do I extend it?

Contact your account manager and let them know how much long you need it. Once they
approve, you can cc support (or the account manager can extend it) and support can extend
it for longer.

Can support add a new user to my account?

Boomi Support is unable to add users for security reasons pending other circumstances like
your administrator leaving or no one in your company has administrator access, etc. Your
administrator will have to add your new user. If your administrator is not available, you can
contact Boomi Support and cc your account manager so they can verify your request.

How to enable the support portal?

Training and trial accounts do not have access to a support portal. Active accounts can open a
support case or send a case to support@boomi.com with the email address, first and last
name or user, and account ID. Only one email address can be associated with a support portal
for an account. If you have a support portal for company A and want a support portal for
company B, you will either have to switch your support portal to company B or have a second
login (with a different email address) have a support portal for company B.

If you are a partner's account (not a direct Boomi client), you have to contact your partner to
enable your support portal since Boomi isn't able to modify your account information.

How do I change my account name displayed at the top right?

1. Hover over your account name at the top right, click "Setup"
2. Click on the "Account Information" tab
3. Change your "Account name" to desired display and click save

50

Internal Use - Confidential

 How many times can a user try to log in unsuccessfully?
A user is allowed 6 consecutive unsuccessful login attempts before their account is locked.
After the 3rd unsuccessful attempt they will see a countdown of how many more attempts.
The account administrator will receive an email alert indicating the failed attempt and lock
out. Once the account is locked, the account will remain locked indefinitely until the user
resets their password.

3.2 Atoms, Molecules, and Atom Clouds

Boomi provides a cloud-based integration platform. It supports two deployment models: an in-
the-cloud deployment model that is used when all the integration endpoints are cloud-based
and an on-premise deployment model that is used when any of the integration endpoints are
within a corporate network.
• If you use the in-the-cloud model, you can deploy your integration processes to a Boomi Atom
Cloud.
• To support the on-premise model, Boomi provides a capability called an Atom, which is a
lightweight Java application that is deployed on a host with Internet access.
• If you use the on-premise model and want to have a highly available, load balanced solution,
or if you want to change your Atom's processing time or volume handling, or if you are
worried that your Atom's processes may not run because of a computer outage, consider using
a Molecule.

• If you need a multi-tenant solution to accommodate multiple customer accounts, consider

setting up your own private Atom Cloud.

• Understand more about the Atom and Molecule here: Boomi Chalk Talk Series: Boomi Atom
and Molecule

51

Internal Use - Confidential

 Figure 15: Boomi AtomSphere

3.2.1 Atoms

The Boomi Atom is a lightweight, dynamic runtime engine. Once your integration processes
have been deployed to your Atom, the Atom contains all the components required to execute
your processes from end to end, including connectors, transformation rules, decision
handling, and processing logic.
A local Atom is a single-tenant, single-node AtomSphere runtime engine. Single tenant means
only available for one account.

If you need to connect to applications and data locations within your local network, you will
need to install an Atom on a machine within the network.

You do not need a local Atom on each application or server that you are trying to integrate
nor does the Atom need to be installed on the same machine as the application. The Atom
needs to be able to connect to the other on-premise applications or servers.

Considerations
There are some minimum hardware requirements, but they are pretty basic:

52

Internal Use - Confidential

 • OS - Windows or Linux machines that support Java 7 or 8 Runtime
• Processor - 1.8 GHz or higher Pentium 4
• Memory - 2GB RAM (min. 1GB dedicated to Atom)
• Hard Disk - 50MB for runtime and configuration, 10GB for data archiving

So you will choose an Atom runtime if:

• You want to access local or on-site endpoints like database, file shares, on-premise
APIs, or cloud-based platforms and APIs.

• You are able to procure and manage local server environments.

• Your organization restricts locating data in shared environments.

• Your performance or through-put requirements are not excessive.

• Under normal sizing an Atom can handle processing of around 100,000 documents per
month
Super-sized Atoms refers to an Atom with more system resources allocated to it. It is not a
separate entity from a normal Atom.
By default, the machine allocates 1GB of RAM for the Atom heap. You can allocate more heap
to the JVM to increase through-put.

• You can upgrade the machine’s CPU resources to have more simultaneous executions.

Pros and Cons

Pros

• Access local resources (file read/write, database, on premise systems).

• Can also access cloud-based systems and APIs.
• More control over resources, configuration, and authorization modes.

Cons

• Must procure, set up and manage servers.

• Not suitable for large volumes, high performance.
• Does not ensure high availability standalone.

53

Internal Use - Confidential

 Figure 16: Boomi Atom

Local Atom versus Atom in the Cloud

You can choose to use an Atom in one of two ways: you can install one locally to a machine in
your network, or you can use one in the cloud. Which option you choose depends upon your
specific integration scenario.

• Local — If your integration scenario includes connecting to resources or applications behind

your firewall, such as a database, file system directory or other on-premise applications, you
have to install the Atom locally. The Atom must be installed on a machine that has access to
all the required resources. This is how the on-premise to SaaS integration problem is solved!

• In the Cloud — If your integration scenario includes only connecting to resources or

applications accessible via the Internet, such as web applications and FTP sites, you can
choose to use an Atom in an Atom Cloud. This hosted option provides a "zero-footprint"
integration solution, with no software or hardware to be installed because all computing is
performed in the Boomi data center.

Atoms and environments

If test connections are enabled in your account and if you are using environments, which are
classified as test or production, you can attach your Atoms to either type of environment.
When you attach a local Atom to an environment, the appropriate connection licenses are
calculated based on the classification of the environment. If you move a local Atom from one

54

Internal Use - Confidential

 type of environment to another, the production and test connection licenses are
recalculated. For more information, see the topic about test connection licensing.

3.2.2 Molecules

The Boomi Molecule is a single-tenant, clustered Atom that allows for multiple Atom
processes to run concurrently. It is the enterprise-grade version of an Atom that can be
deployed across multiple servers to enhance load balancing and ensure high availability for
mission-critical integration processes.
A Molecule consists of multiple "nodes" that are installed on multiple on-premise machines.
The nodes on the machines are grouped together to form the Molecule. Users will see only
one "Atom" instance on the Boomi AtomSphere platform's Process Reporting and Atom
Management pages, but if the Molecule is enabled it is truly a grouping of multiple nodes that
distribute processing across one or more physical machines.
The Molecule is best used in integrations that will receive and/or generate a high volume of
documents. Review the Flow Control shape to configure multi-threading and processing within
your Boomi process design.

Figure 17: The Molecule

55

Internal Use - Confidential

 What is clustering?
Similar to an Atom, a Molecule executes integration processes that are managed on
the AtomSphere platform. The Molecule distributes these requests among all the machines
that form the cluster. This results in balanced computational work among these different
machines, which improves the overall performance of the cluster system.
A single node in the cluster is defined as the "head" node. The head node:

• Is elected from the remaining nodes in the network and can migrate between nodes ("failover
support")

• Retrieves messages from the platform

• Schedules processes

• Maintains the health of each node

• Tends to send processes to other nodes for processing, because the head node has to take
care of the “administrative” tasks above

56

Internal Use - Confidential

 Figure 18: molecule Clustering

Do you need a Molecule?

Below are some general questions that you should consider before implementing a Molecule.

Are you satisfied with your on-premise Atom's processing time and volume handling?

• Depending on the count, size and variation of your document types, you may want to consider
Molecule processing across multiple machines. Refer to the High Volume Troubleshooting
topic to understand what could be causing memory issues and/or slowness among
your AtomSphere processes.

How much do you care if a process does not execute due to a computer outage?

• Molecules support failover, so if one "head" node dies, another node attached to the Molecule
can support the management, scheduling and execution of your processes.

Is this setup worth the extra configuration time, hardware allocation and maintenance?

57

Internal Use - Confidential

 • Atom, Molecule and Cloud technology are enabled in a Boomi Atom Cloud for production
deployment. If you are building integrations that do not require on-premise resources behind
your firewall, you can deploy these integrations in the Cloud.

Misconceptions
Faster than Atoms

• While overall throughput can be greater (more nodes working at the same time),
Molecules are NOT inherently faster at an INDIVIDUAL execution on a single node level.

Load balances REAL TIME executions

• Molecules automatically load balance SCHEDULED executions, however an external load

balancer is required for inbound web services requests.

Provides MID-EXECUTION failover

• A process failure does not automatically continue/resume on another node.

Automatically processes large data sets in parallel

• While it is possible to split large data sets and process in parallel across multiple threads
within a single node or even across multiple nodes (or both), it does not happen
automatically. Processing data in parallel is a conscious design decision using the Flow
Control shape.

Downtime during version upgrades

• Molecule can do a rolling restart of each node to apply monthly AtomSphere updates.
This means there is no overall downtime for the Molecule. These settings can be
configured in the platform.

Pros and Cons

Pros

• Can access local resources (file read/write, database, many behind-the-firewall systems)
and also access cloud-based systems and APIs.
• High-Availability architecture supports fail-over and scheduled execution load balance.
• Suitable for large volumes, high performance.

58

Internal Use - Confidential

 • More configuration, tuning and set-up options.
• No additional data volume fees.

Cons

• Must procure, set up, and manage server set.

• Higher level of setup, configuration, and management required.
• Possible additional cost for molecules if not included in the purchased edition –
enterprise or enterprise plus editions get 2 free Molecules, everything else will require an
additional purchase.

3.2.3 Atom Clouds

When referring to Atom Clouds there are Boomi Atom Clouds and Test Atom Clouds in the
Americas, Europe, and the Asia-Pacific, all of which are maintained and run by Boomi. You
can deploy your cloud-based integration processes to any of these Atom Clouds to which your
account has access. You can also set up your own private Atom Cloud.
An Atom Cloud is similar to a Molecule, with several exceptions:

• It is multi-tenant, whereas a Molecule is single-tenant. An Atom Cloud can be used by

multiple accounts.

• It uses forked execution, which allows each integration process to execute in a separate Java
virtual machine and which is not enabled in a Molecule by default.

• It uses a High security policy, whereas Molecules use a Low security policy.

An Atom Cloud consists of one or more Cloud Molecules installed on multiple machines. The
Cloud Molecules on the machines are grouped together to form the Atom Cloud. The
Boomi AtomSphere platform assigns each account to a particular Cloud Molecule so that
processing is spread across the available Cloud Molecules. This is automatic based on the

59

Internal Use - Confidential

 number of accounts that are assigned to each Cloud Molecule. The AtomSphere
platform attempts to maintain an even number of accounts.

Figure 19: Atom Cloud

Why have a private Atom Cloud

If you would like to do any of the following, you should set up your own private Atom Cloud.

• Share your Molecule across multiple customer accounts — A typical scenario is that a
Boomi partner sets up an Atom Cloud and shares it with their accounts.
• Execute and store customer data within your own infrastructure — This may be required for
security reasons, because of contractual requirements between you and your accounts, or to
comply with regulations concerning data storage.

• Process very high volumes of data — If you need to do this you need to increase the amount of
memory available per Molecule to more than the 512 MB that is allocated by default.

• Control all hardware resources.

• Provide your own public URL (for example, connect.yourcompany.com).

60

Internal Use - Confidential

 Production and test Atom Clouds
If you are using environments and if test connections are enabled in your account, when you
add a private Atom Cloud you must select a classification. The choices are Production or Test.
Production Atom Clouds can be attached only to production environments. Test Atom Clouds
can be attached only to test environments. Production Atom Clouds run in a separate physical
environment from test Atom Clouds. This is a best practice that Boomi follows. Partners who
want to install and manage their own Atom Clouds should do the same. The classification can
be set only when you add an Atom Cloud. You cannot change it later.
Note: Atom Clouds added prior to the January 2014 release are classified as production Atom
Clouds and they cannot be changed. They are attached to production environments.
The classification also determines which type of license is used when a process is deployed to
an environment attached to this Atom Cloud. If you want to use a test Atom Cloud with test
environments, you must have test connection licenses. See the topic about test connection
licenses.

Configuring and maintaining a private Atom Cloud

Once you have installed an Atom Cloud you may want to modify how it runs, change default
settings, or even remove it. See the topic on Atom, Molecule, and Cloud management.

Atom Clouds use a High security policy. Custom connectors that run in an Atom Cloud have
additional security restrictions.

Deploying processes to a private Atom Cloud

When you deploy processes to a private Atom Cloud:

• If you install your own Atom Cloud, your scheduled processes will run according to the system
time of the machines on which the Cloud Molecules in your Atom Cloud are installed.

• There is currently no limit to the number of processes that can be deployed to a Cloud
Molecule in an Atom Cloud.

• Only a certain number of process executions can run simultaneously on a Cloud Molecule. If
many processes are scheduled to run at the same time, they are put into a queue. At peak
processing times, your processes start no later than three (3) minutes after the originally
scheduled time.

61

Internal Use - Confidential

 • The Boomi Atom Clouds’ default character encoding is UTF-8, but you can configure your
private Atom Cloud differently.
The File Encoding property on the Properties panel controls the character set that the Atom
uses to transform bytes to characters (and characters to bytes) if no explicit character set is
provided. Here is a list of encoding sets.

Pros and Cons

Pros

• Multi-tenant architecture supports multiple organizations seamlessly.

• High-Availability architecture supports fail-over and load balance.
• High security, each process has own JVM.
• No additional data volume fees.

Cons

• Must procure, set up and manage server set.

• Higher level of setup, configuration and management required.
• Possible additional cost of an Atom Cloud for each molecule.

3.2.4 How to Install a Local Atom, Molecule, or Cloud

Overview

This article will summarize the system requirements and installation steps required to install
and configure an Atom, Molecule, or Cloud, on customer hosted equipment. The information
below is designed to provide a high level view of the overall process. Please refer to the
Reference material for more detailed information.

Atoms vs Molecules vs Clouds

These three options, should be thought of as distinct types of Run Time Engines, all of which
are capable of executing AtomSphere processes. An Atom can be thought of as a single, stand
alone Java machine, running on a single server. In contrast, the Molecule and Clouds are
comprised of a cluster of JVMs, running across a suite of machines.

62

Internal Use - Confidential

 It is important to understand that the Java process running on a Molecule node is not an
Atom. These are three separate products, and they do work slightly differently.

Hardware Requirements

http://help.boomi.com/atomsphere/GUID-0815001F-8949-42EF-9808-3C0B53A29A77.html

Servers

All three types of installs, require one or more Linux or Windows machines. For best
performance, these machines should be dedicated Boomi resources, without any other
applications installed ( Exception: There are a few connectors/applications that require the
Atom be installed locally ).

Molecules and Clouds, are both clustered systems, which require multiple machines ( referred
to as Nodes ), which each should be configured identically.

Clustered systems also require Network Storage, shared across the multiple nodes.

Operating System

The installation of all three products is supported on Linux or Windows. The choice of
Operating System, should be based on the Operations/IT resources currently in place within
your organization. For example, if you already have other Linux resources within your
infrastructure, then install a Linux Atom. This way, you can utilize the current skill set of your
team, as well as the monitoring tools that are already in place. The same would apply if you
primarily use Windows within your organization.

3.2.4.2.2.1 Service User

For each product, there will be a Java process, which runs as a service on each machine.

This service should never be run under the user account of a specific individual within your
organization, or as the "root" user. Instead, it should run as a Service User (and Group). This is
typically referred to as the "Boomi" user. ("boomi:boomi" for user:group)

63

Internal Use - Confidential

 In a Windows environment, this service user needs to have Local Administrative rights.

In a clustered environment, the Boomi user and Group should be created within your Network
Domain. This way, the same User can be used across each machine in the cluster. That user
will need read and write access to the file system as described below.

3.2.4.2.2.2 Java Version

The Cloud installer requires that you download and install the Java JDK prior to installing the
Cloud. However, for Atoms and Molecules, if no Java version exists on the new machines, the
installer will automatically download and install the correct version (within the Installation
directory). Optionally, you may install the Java JRE by hand, on each machine, prior to
installing the Atom or Molecule.

If you chose to install Java independently, as a best practice, we recommend running the
same version of Java as the Boomi Public Cloud (All of our QA / Regression Testing is done
against this version)

The Java version can be changed post install if required. Common reasons for this include:

• Security Updates
• The Boomi Public Cloud has updated, and you want to stay in step
• Newer Java versions may have performance improvements (Note: There is an
outstanding issue running Clouds on Java versions greater than Java 8 update 75 )

Storage

3.2.4.2.3.1 Local Storage

The Atom installation should be stored on a disk/partition local to the machine itself. As a
best practice, create a partition/drive ( i.e. D:/ ) separate from the Operating System. This
protects the OS from disk utilization issues, in the event that the Atom logs fill a partition.

As mentioned above, the clustered Molecules and Clouds, are primarily stored on a Network
attached system. However, Molecules and Clouds MUST be configured with "Local
Working Storage." When executions run on a particular Molecule node, temporary data will be

64

Internal Use - Confidential

 written to the local directory, rather than across the network, dramatically improving
performance.

Molecule and Atom Cloud working data storage

Optionally, "Local Temp" storage is configurable in all three products. By default, the JVM will
use Java temp (In Windows, this is typically the Boomi user's home directory. In Linux, this is
typically /tmp).

In clustered environments, both the Local Working Storage, and Local Temp directories, must
be local to each machine, and must be configured the same way on each machine.

For all three installation types, the Installation directory, Local Working Directory, and Local
Temp directories, must all be owned (read and write access) by the Boomi service user.

3.2.4.2.3.2 Network Storage

As mentioned above, Molecules and Clouds require dedicated network storage that supports
"Advisory Locking," and either NFS version 3 or 4.

As a best practice, the shared drive should be dedicated to the Boomi systems. This helps to
avoid inadvertent performance issues caused by other applications running on other systems.

The network drive should be mounted to the same directory path on each node of the cluster,
and it should be configured to remount when the machine restarts.

Just like with Local Storage mentioned above, the Boomi User must have read and write
access to the installation directory on this network drive.

3.2.4.2.3.3 Antivirus

For all three installation types, Antivirus software should be configured to ignore any
files/directories from the installation directory and below.

65

Internal Use - Confidential

 Network

3.2.4.2.4.1 Firewall

All Network traffic between an Atom and the Platform, originates from the local system.
Therefore, it's typically not necessary to white-list the IP addresses for the AtomSphere
Platform

Hostnames and IP addresses for connecting with Boomi

Note: When hosting Web Services, or AS2 connections on an Atom within your network, you
may need to open up inbound rules, specifically for those Client Applications.

3.2.4.2.4.2 Proxy Settings

Proxy settings, for outbound traffic (from your Atom), can be configured during the install,
but can also be updated post install if necessary.

Changing HTTP proxy settings on an Atom or Molecule

Per the article above, you can also configure non-proxy-hosts if required (perhaps to connect
to an on-premise web application).

3.2.4.2.4.3 Unicast vs Multicast

In clustered environment, the individual nodes will communicate with one another, by default
using Multicast. Before doing the install, you should determine whether your network rules
will allow for this, and whether or not you will need to run in Unicast mode. Note: Many VM
and Cloud environments do not support Multicast.

The Unicast and Multicast ports are configurable.

3.2.4.2.4.4 Load Balancer (LB)

In a clustered environment, the Molecule or Cloud will load balance only scheduled
executions. Inbound Web Services or AS2 connections, will be handled by the Node that
receives them. Therefore, as a best practice, configure a Load Balancer in your network, that
will distribute the requests across the cluster. (F5 for example)

66

Internal Use - Confidential

 Typically, this LB might expose a public facing IP address to your client applications or trading
partners, and then distribute the requests to the internal network. You can configure port
translation as well.

Depending on requirements, the LB can be configured to terminate SSL, and pass un-
encrypted traffic to the individual nodes, or it can pass SSL through to the nodes for
decryption.

3.2.4.2.4.5 Reverse proxy vs DMZ

As a best practice, we typically install the Molecule/Cloud within the corporate firewall, and
NOT in the DMZ. The F5/Load Balancer in these configurations can be configured as a reverse
proxy if required.

Installation
Downloading

The installer can be downloaded from the Atom Management page. The installer could be
updated each month as part of our normal release cycle, therefore, as a best practice, you
should always download the latest version before doing your install.

Permissions

During the installation, you will be asked to either enter your Platform credentials, or to
enter an Installation Token (generated when downloading the installer). If the account is
configured to use SSO, then the credentials you use must be those of a user that has "Default
Boomi Administrator" privileges.

Downloading the local Atom installer

Install

The Atom installation is run on the machine that will host the Atom service. The Molecule and
Cloud installers should be run on only a single Node in the cluster. It is not necessary to run
the same installer on subsequent nodes.

67

Internal Use - Confidential

 For Windows installs, the installer must be run with elevated permissions. Typically, we
suggest "Right" clicking on the installer executable and running "As Administrator."

For Linux installs, we typically suggest using the "su" command to assume the role of the
Boomi user, and then running the install. Note: during the install, you will be asked whether
or not you want to create symbolic links. Typically, the Boomi user will not have permission
to modify files in /etc., and therefore you should skip this step.

If, during the install, you choose to enter your Platform credentials, it is important to
understand that these credentials are not stored within the installation directory itself. These
credentials are only used for the initial communication with the Platform. Once the Atom is
installed, future communication with the Platform will be authenticated using a key specific
to that install, and it will not need to re-use your personal credentials.

In clustered environments, remember to enter a Local Working Directory.

After installing the Windows service, edit the service and change the "Logon As" to make sure
the service starts with the correct Boomi user.

Setting up the Molecule's Windows service

After installing on Linux machines, configure the Java process to start during the Boot
sequence.

Creating a Linux systemd service to start the Atom

3.2.4.3.3.1 Unicast vs Multicast

As mentioned above, Nodes in a clustered environment communicate via Multicast by default.

If your network will not support Multicast traffic, after installing the first Node in the cluster,
you can configure the Molecule/Cloud to use UNICAST

Setting up unicast support

http://help.boomi.com/atomsphere/GUID-DAC921FC-F37A-49BF-9157-587B760AC1BC.html

68

Internal Use - Confidential

 If configuring UNICAST, you must define the "Initial Hosts For Unicast" property as mentioned
in the article above. This list of IP addresses does not need to include every machine in your
cluster, but it must define at least one machine that will be online at any given time. You can
add additional nodes in the future, for example, without updating this property.

If there are other services running on the individual nodes that might use ports that conflict
with the default Unicast/Multicast ports, please specify ports that are available.

Changing the unicast port

Changing the multicast address and port

3.2.4.3.3.2 Installing Secondary Nodes

Installing additional Molecule nodes on Windows

Installing additional Molecule nodes on Linux

Installing additional Cloud Molecules on Linux

Installing additional Cloud Molecules on Windows

Verification

After installing each machine, verify via the Atom Management UI, that the service is online,
and in a good state. This implies that the service is online and communicating with the
Platform.

If running a clustered environment, use Cluster Status to ensure that all nodes are online, and
that only one node is indicated as the "Head" node.

Deploy and execute a test process and verify that you can retrieve a process log from Process
Reporting.

You may choose to shut down or restart the head node to verify that "Headship" correctly
transfers to the remaining nodes. By restarting nodes, you can also verify that each node in
the cluster is capable of executing processes.

69

Internal Use - Confidential

 3.2.5 How to Migrate Your Atom to Another Server

Overview

You may need to move an Atom (or Molecule or Cloud), either to a new location on the same
computer or to a new location on a different computer. You would like to preserve schedules,
process properties, deployment, and other data.

Some typical reasons for moving an Atom are:

• New hardware / hardware upgrade / hardware failure

• Different Atom installation directory
• A resource with more disk space

After moving the Atom Installation folder from one location/machine to another, you must
change certain properties, including machine names and file paths, so that they point to the
Atom's new location.

Important Notes:

Moving an atom/molecule installation, to another directory, another host, or even changing IP

addresses is NOT the recommended approach. If this is necessary, you should consider
installing a new atom, and migrating the processes from one to another instead, see How To
Migrate Processes to a New Atom. This article is here for reference, to document the
files/configuration that may contain host/IP information only.

If you copy an installation from one machine to another (or image a machine for use on
another server), please note that YOU CANNOT RUN DUPLICATE ATOMS in this way (a.k.a.
active-active). The atom is identified by our platform with an internal ID, and if you have two
installations with the same internal ID, this would cause significant issues. If you must copy a
server image or atom installation, you must not start both atoms at the same time. You
SHOULD delete the atom installation of the unused installation once the migration is
complete.

70

Internal Use - Confidential

 Instructions

Note: If you are creating an exact mirror of the current machine/VM, you should only need to
verify these settings vs. changing values.

If you move an Atom's Installation folder (<atom_install_directory>) to a different directory

on the same computer, you must verify/change the following properties:

• container.properties file (located in <atom_install_directory>\conf)

o com.boomi.container.localDir
• start.properties file (located in <atom_install_directory>\conf)
o com.boomi.container.trustStorePassword
o com.boomi.container.keyStoreLocation
• atom(w).vmoptions file (located in <atom_install_directory>\bin)
o -classpath/p
o -Djava.endorsed.dirs
o -Djava.io.tmpdir
o -Djava.library.path
• pref_jre.cfg file (located in <atom_install_directory>\.install4j) - this is the path to
the preferred JRE, which may or may not have changed
• inst_jre.cfg file (located in <atom_install_directory>\.install4j) - this is the path to the
JRE installation directory, which may or may not have changed.

Note: Where atom(w) means: atom = running as a service, atomw = running in

console/desktop mode

If you move an Atom from the computer on which it was installed to a different computer,
you must verify/change these properties:

• container.properties file (located in <atom_install_directory>\conf)

o com.boomi.container.cloudlet.initialHosts
o com.boomi.container.sharedServer.http.baseUrl
o com.boomi.container.sharedServer.http.host
o com.boomi.container.sharedServer.http.externalHost - Make sure that
externalHost points to the load-balanced VIP pool containing Atom Workers.

71

Internal Use - Confidential

 • atom(w).vmoptions file (located in <atom_install_directory>\bin)
o -Djgroups.bind_addr

Service/Startup scripts:

• Windows: The Windows service will need to be re-installed on the new machine. It
could be modified if only the installation directory changes. Alternatively if you only
move the atom installation directory, the path to the atom.exe file defined in the
service should be changed. This can be accomplished by editing a registry key
(see Modifying the "Path to executable" of a windows service - Stack Overflow).
• Unix: The init.d scripts will need to be re-installed/modified

Web Service and other listeners:

• If you move an Atom to a different computer and you are using Web Service Publishing
or other listening connectors (like SAP), you may also need to consider making changes
to the following:
o Any external applications that call Boomi AtomSphere services.
o The connect.boomi.com URL-- either change in the application, DNS, proxy

Java:

• You may need to re-import any java certs into your new JRE.
• Also copy the Unlimited Strength Cryptography files into your JRE if necessary.

3.2.6 How to Migrate Processes to a New Atom

Content

Below is a high level, basic outline for migrating processes from one atom/molecule to
another. The following instructions assume that you have environments with extensions
enabled on your account. If not, work with a support team member to discuss alternative
steps if necessary.

72

Internal Use - Confidential

 Licenses

During the migration, you will be attaching your processes to both the old and new atom, so
you'll need double the licenses you're currently using. If you do not have enough licenses,
please contact the Boomi Support team and your account manager. Discuss with the support
team member and the account manager how long you need to complete the migration and
they will be able to provide temporary licenses to use during the migration.

Configure New Atom or Molecule

If your old atom/molecule has any specific settings already set, do the same for your new
atom/molecule. A couple of configurations to consider:

• You may need to import Java certificates into the new JRE keystore
• You may need to install Unlimited Strength Cryptography files
• You should check the container.properties, atom.vmoptions, and other molecule
related settings files like the procrunner, procworker, etc and apply the settings in
there to the new atom/molecule accordingly
• You should check the Shared Web Server settings from Atom management and make
necessary changes
Note: If it is a new atom on the same server, you must use a different port within
Shared Web Server settings

Custom jar Files

If any of your processes use custom scripts or drivers (.jar files), the contents of the userlib
folder must be migrated to the new atom. You can either move them directly over or manage
them via the custom libraries component. The deployed custom library will automatically
take effect and add the jar files in it to your new atom when you attach the atom to the
environment.

Note: If you are changing OS or architecture, you may require different jar files or drivers to
meet the new system's requirements

73

Internal Use - Confidential

 Attach the New Atom to the Existing Environment

After you have checked the above, you are ready to attach your new atom to the
environment.

Extensions

You will need to enter in your passwords for the environment extensions again. The passwords
in extensions will not be applied automatically to your new atom. If you have a lot of
passwords though, you can do the following to move passwords from one atom to another:

1. Go to <<atom_installation_directory>>\settings\atom on the old atom

2. Find the component ID of the connectors you are using
3. Copy the folder(s) with the same component ID you found in step 2
4. Paste them into the same location (<<atom_installation_directory>>\settings\atom) in
the new atom
5. Restart the new atom

Migrating Schedules

The processes deployed on the environment should show up in your deployed processes tab on
the new atom but without any schedules. To set the schedules you can either set them up
again if it's just a few processes or you can use the Atomsphere API.

Here's a sample of the process you can use to move schedules over:

Figure 20: Sample Process to Move Schedules

1. Use the Query operation on the Process Schedule object to pull the current schedule
for the old atom. The filter should be using the old atom ID

74

Internal Use - Confidential

 2. Map the query response profile straight across to the update request profile since they
all use the same field. You will add a default value for the atomID since it will be the
new atom ID now

Figure 21: Map in the Process

3. Use the Update operation to pass the schedule to the new atom
4. Confirm schedules have been copied to new atom. The schedules will be advanced
instead of the Minute, Hour, or Day type you might have used before since the API
translate the schedules as advanced automatically. The schedules should still be
equivalent

Deployment History

Because you are attaching to the same environment, your deployment history will be carried
over automatically.

75

Internal Use - Confidential

 Process Properties, EDI Sequence #s, CDC Data

You may choose to migrate all processes at once or one at a time to allow for
testing/validation on the new atom. For each (non-listener) process you wish to move:

1. Stop the schedule(s) for the process on old atom

2. Move persisted process properties like the "Last Successful Run Date"
a. Go to ./execution directory on the old atom
b. Find out the component ID of your process (you can look in the process URL or
revision history window in the build tab)
c. There should be a file in the executions folder named with the
<<component_id>>.properties, copy it from old atom
d. Paste it into the ./execution directory of the new atom
3. Move counters/EDI Sequence #s/etc
a. Go to the ./counter directory
b. Similar to step 2, copy the ./counters/<<component_id>> folder from the old
atom to the new
4. Move the Find Changes file
a. Go to the ./work/cdc directory
b. Similar to step 2, copy the ./work/cdc/<<component_id>> folder from the old
atom to the new
5. Start the schedule(s) for the process on new atom
6. Repeat as needed for each process

Listener Processes (HTTP Server, AS2 listener, etc)

For listener processes, you may test the new listeners by using the port defined on the new
atom's in Shared Web Server settings. A couple of considerations:

• If you are directing client requests to your service via a router configuration, you may
simply be able to re-route those requests to the new server/port
• If you are using an external load balancer, you may be able to redirect the requests
using the balancer
• You may also choose to have your clients update their connection points to connect to
your new servers

76

Internal Use - Confidential

 • You may choose to detach (or stop) all the listener processes from the environment,
then change the web server settings on the new atom to the port number of the old
atom (if it's on the same server) and start the listener again

Execution History

Execution history will stay with the old atom. If information is required for past executions,
you can look in process reporting or query the execution records object using the old atom
name/ID.

Removing the Old Atom

Once the migration is complete and all processes and listeners are now running on the new
atom you can detach the old atom from the environment. Your account owner will remove
the temporary licenses from your account. You can uninstall the old atom if you do not need
anything else from it.

3.3 Runtime Configuration and Mechanics

3.3.1 Configuration Files

Atom VMOptions
By default, the local Atom or Node will get 512MB of heap space allocated to it. If you want
to allocate more RAM then we need to access the local directories to make this change.

The atom.vmoptions file is located in the atom's installation directory, under the bin folder.
(depending on your Windows browsing window you may only see the file name as Atom and
the Type column as VMOPTIONS file).

This file contains the server level configurations to enable remote settings, client time out
settings and enable remote JMX (Java Management Extensions). Also, as being displayed on
this slide, the runtime memory allocation is configured. Edit the amount of MB and save the
file. Then you will need to restart the runtime for the change to take effect.

77

Internal Use - Confidential

 Runtime Properties
Local runtimes:

• Local Atoms
• Molecules
• Private Atom Clouds

Changes reflect in the container.properties file.

Each property consists of three different elements:

• Name
• Default Value
• The Format

When the runtime properties are changed they are reflected in the container.properties
file.

AtomSphere runtime properties are available for accounts containing local Atoms,
Molecules and Private Atom clouds.

Each property consists of 3 different elements, the name, default value and format of the
element.

Figure 22: Containers Properties File

The next configuration file is the Container Properties located in the atom's installation
directory, under the conf folder.

78

Internal Use - Confidential

 This file contains all of the settings for your runtime configuration including the Atom or
Molecule name, proxy information and log settings.
Although text editor can be used to edit the file’s properties, you are also able to edit
the properties in the Properties panel on the AtomSphere Platform. This is in Manage >>
Atom Management >>Properties panel.
One item of note: some of the changes made in the Atom files require a restart.
Table 2: Property Structure

The property structure contains the Property Name, Default Value, Format and a description
(it is not listed here but is contained in the reference guide).
Many properties require a restart to the local Atom, Molecule, or private Atom Cloud after it
is set or changed. Checking the Restart on Save option before saving the property forces the
necessary restart. Only Atom, Molecule, and Cloud owners can restart them. Accounts having
an Atom in a Cloud cannot restart the Cloud.

79

Internal Use - Confidential

 Figure 23: Changing Values in UI

Within Atom Management, click on the on the Atom being used, in this case it is our local
Atom and select the Properties menu option. If selecting a cloud atom, it would be called
“Account Properties”

80

Internal Use - Confidential

 Figure 24: Basic Properties

Basic properties consist of 3 property types, Basic, Advanced and Custom.

Let’s begin with basic properties.
• Atom Name - is used to edit the display name of the selected Atom attached to an
Atom Cloud.
• The Host Name – are the locally installed runtimes which require distinctions between
nodes/atoms to correctly point to and update. This is only available for a local
atom.
**Note: Without this each molecule node would fill the role of head node and not load
balance requests to ‘child’ nodes
• Purge Data Immediately – If checked, the purge takes place immediately after a
process execution ends. This setting does not affect when logs are purged. If this
setting is off, the process option takes precedence.
• Purge History after x days - Sets the number of days following process execution the
purge. The default is 30 days.
• Force restart after x minutes – If a value is contained in this field it forces a restart
after the number of minutes This is only available for a local atom.

FYI Only:

81

Internal Use - Confidential

 Display Name Property Name
Atom Name com.boomi.container.name
Purge Data Immediately com.boomi.container.purgeImmediately
Purge History After X Days com.boomi.container.purgeDays
Force Restart After X Minutescom.boomi.container.forceRestart

Figure 25: Advanced Properties

The advanced properties tab contains many different properties to set. Cloud atoms can only
set Low Latency Warning threshold, Purge schedule for logs, Purge schedule for Processed
Documents and Purge schedule for Temporary Data. However Local atoms as displayed in this
slide, contain a multitude of properties to set. For more information about the properties,
visit the reference guide and located in Search>Integration management>Atom, Molecule, and
Atom Cloud management and configuration >Atom Management >Properties panel.

82

Internal Use - Confidential

 Figure 26: Custom Properties

The Custom tab is available only for a local runtime or Atom Cloud. It is not available for
account properties. It is used to set properties not available on the Basic or Advanced tabs
and is used primarily for adding custom properties Boomi Support may instruct you to add.
To create a custom property, enter:

Name Description
Add CustomProperty Adds a pair of blank Property Name and Property Value fields to
the Custom Properties table.
Property Name Used to enter a custom container property name. The complete
property name must be entered. For
example, com.boomi.container.name.
Property Value Used to delete a custom container property value.

3.3.2 Custom Scripting and JAR’s

In Integration, JAR files reference custom Java classes developed outside Integration for your
local or cloud atom.

There are three types of JARs:

• General that support multiple functions (these JARs are stored in /userlib).
• Scripting that support custom scripting (these JARs are stored in
/userlib/script).
• Connector that support a connector (these JARs are stored in
/userlib/<connType>).

Each time a class is referenced the runtime must be restarted to be available to use it.

83

Internal Use - Confidential

 Custom Library components (JARs) must be deployed independently — they are not
referenced by processes or any other components.

Figure 27: Upload JAR #1

Step 1: Upload custom JAR to Account Libraries (Setup > Account Libraries).
Each Boomi account has access to 100 MB of space for uploaded files.

Figure 28: Upload JAR #2

Step 2: Create a Custom Library component in Build tab.

Add one or more of the uploaded JAR files in the account library to the component.
When you create a Connector custom library, you also specify a connector type, such as
Database. Once you create a Custom Library component, you cannot change its type.

84

Internal Use - Confidential

 Figure 29: Upload JAR #3

Step 3: Deploy the Custom Library component

When you deploy a Custom Library component, the JAR files that it references are deployed
to the appropriate /userlib folder.
In the Deploy tab you need to change the drop down list from Processes to Customer Libraries
to view the custom libraries components.

Figure 30: Upload JAR #4

85

Internal Use - Confidential

 3.3.3 Communications
The local runtime is constantly checking for updates, several times a minute. They serve as
“keep alives” for the platform. If enough of these heartbeats are disrupted for long enough
the Platform will consider the runtime to be down and mark it as offline.

What is the runtime checking for?

It is looking for configuration changes, either to the runtime or a process. Remember some
changes to the runtime are managed in the UI, so it will get those changes.

Schedule changes and execution requests

If you deploy or redeploy a process and edit extensions

Atom also uploads process metadata – count and size of documents. This is not uploading the
data, only execution statistics.

Document view requests are a re-direct to the local atom, document is not persisted on the
platform.

What happens when the communication breaks down? Well you can imagine the runtime is not
able to get all the changes from the Platform that I just listed.

• Process reporting won’t have the latest process metadata

• Schedule changes or updates will not be communicated

• However, scheduled processes will continue to execute

• Deployments will not propagate

• Creates a Retry Loop error in the atom logs – when the communication is reestablished
the log and metadata will be sent to Process Reporting for the scheduled processes
that did run during the outage.

• Platform sends out “Atom Down” notification – this will generate an event log which
you can alert you by email or RSS feed.

It is good to note that upon failure/outage – the duration between attempts to communicate
slows down

86

Internal Use - Confidential

 3.3.4 Runtime and JVM’s
• Once again, a single JVM runs on a single machine. So, an Atom only has one JVM to
handle the workload associated with that Atom. A Molecule has a single JVM per Node on
the cluster, by default. A Molecule can be configured with Forked Execution in which
case there can be multiple JVMs per Node, similar to a cloud.

• WSS = Web Services Server. This is how you make Boomi a RESTful web service or SOAP
service to your customer or company. Instead of sending requests to consume the service,
you ARE the service and Boomi receives the requests.

• JMS = Java Messaging Service is a queue for storing and retrieving messages.

Figure 31: Understanding JVMs

• An Atom has a single JVM.

• A Molecule has a head-node and can have multiple child nodes.
• The Atom Cloud has a head-node, a specific JVM for handling Atom Queueing. We could
have also included a JVM that handles real-time executions called an Atom Worker. There
are JVMs for each execution running simultaneously on that cloud. The Atom Cloud will
always have multiple JVMs per Node because a property called Forked Execution is on by
default. This is because the Atom Cloud is a multi-tenant environment and necessitates
forked execution to provide process isolation.
• There is a user control over turning on forked execution in a Molecule, it is off by default.

87

Internal Use - Confidential

 3.3.5 Executions

Threaded Executions
A thread is an executing code path within a JVM. The JVM is doing many “jobs” at one time –
it is executing the process, communicating with platform, maintaining health…. All those jobs
are handled on different threads. All the threads are sharing the same heap space (allocated
memory, default is 512 MB). If your process is moving a lot of documents and data, it is
important to know how your heap space is being used.

A single Atom runs different processes as separate threads in same JVM. A Molecule also uses
threads per execution by default unless Forked Executions is turned on.

You can configure the Flow Control shape in your process to allow it to execute across
multiple threads. This is how we can enable some parallel processing within our process. But
those parallel threads are again utilizing that same heap space.

Figure 32: Flow Control Shape > Threaded

88

Internal Use - Confidential

 Figure 33: Flow Control Threaded Example

In our Flow Control we have set the document batch to 5 and the unit scope to threads.
In this example our start shape is going to receive 9 documents. Because of our 5-document
batch setting the Flow Control creates 2 groups and creates a different thread for each group.
They flow concurrently through the process and are sent to Salesforce at roughly the same
time.

Forked Executions
• Forked Execution is an option on a Molecule, but it is the default behavior in the cloud.

• What this does is for every process that executes the Node will fork off a separate JVM.

• Remember each JVM heap space is carved out memory from the total machine resources,
so with Forked Execution you can use more machine resources when needed and not have
to have it always allocated. But be mindful to set a maximum forks value and be aware of
how much memory you have available.

• This is good for scheduled/batch/large data volumes because it allows you to insulate
entire runtime assigning its own JVM and heap space.

89

Internal Use - Confidential

 • Forked Execution is not recommended for web services or other real-time listeners,
because of startup latency and it can quickly consume your system memory.

• Another situation, which is atypical, if you are running multiple Atoms on the same
machine you cannot enable forked execution for the same reason.

Figure 34: Understanding Forked Process Execution

With Forked Execution turned on, Node 1 will spin up a new forked JVM for each process
execution. The main Node will always exist because it must maintain status and
communication to the other Nodes in the cluster.

When the process executions are complete the forked JVMs with terminate and return the
heap back to the machine resources.

Those resources are taken from the JVM, so you must be sure there is enough RAM available
w/o crashing the system. The size of forked nodes can be specified in atom.vmoptions. For
example, the persistent node gets 512 MB, and the two forked nodes also are at 512 MB each
– so you’re using 1.5GB of system RAM

In the cloud its usual to see 512MB for the persistent node with 1GB for forked nodes. In a
molecule forked nodes are usually set to 512 MB.

90

Internal Use - Confidential

 Figure 35: Flow Control > Forked

Figure 36: Flow Control Forked Example

In this example we are setting the Flow Control shape to 5 document batch again, but this
time the Unit Scope is set to Processes.

91

Internal Use - Confidential

 Once again, we see the 9 documents flowing in. This time the Flow Control creates 2 groups
and creates a JVM for each group. They get sent to Salesforce from different machines.

3.3.6 Flow Control

In a Molecule scenario, the Flow Control shape provides the ability to implement multi-
threads or multi JVMs during the process build or design time.

The forked JVM is created on-demand as the process executes.

The Unit Scope is a field within the Flow Control shape that you set at build or design time.
When the runtime cannot accommodate a forked execution request it will default to using
threads.

So on an Atom the scope will be threaded because forked it not available and on a Molecule
the scope will be process as long as you haven’t reached the forked max, then the process
would use threads. On a Molecule you can also configure multiple Flow Control shapes and
fork your process across Nodes and fork across threads.

Pros and Cons

Pros:

• Speeds up data processing (divide and conquer).

• Does not affect licensing (Molecule is single entity).

Cons:

• Certain configurations (like reading and writing from a document cache) in forked
execution mode could cause the process to slow down.
• Must be configured by developers during Build.

The benefit of Flow Control is that it speeds up data processing by dividing and conquering.

The caution of Flow Control is reading/writing from a document cache can slow a process
down.

92

Internal Use - Confidential

 In large data loads it is best to split up the load across multiple JVMs or multiple threads if
you have the heap space available.

Most important is to use the Flow Control shape to take advantage of the capabilities of your
runtime environment.

***Con Notes: If all threads need to access the same cache, cache might move from local to
NFS share, this will cause a slowdown

Note: reference article about – Cases where the Atom writes the cache to disk rather than
keeping it in memory – “Where and how long Cache is stored” – I MB limit for cache – over will
write to disk. Matters most for high performance (web service) processes

Batching and Threads

In the Flow Control shape you can;

• Groups documents

• Create number of units

• Assign doc groups to each unit

• Distribute units to node(s)

• If you enable "parallel processing" and also "run individually" in the Flow Control shape,
the "parallel processing" is applied first before the "run individually". So, if you have 2
threads chosen and 20 documents, then the 20 documents will be split into 2 batches
(1-10, 11-20). A thread will be started for each batch, and within each thread, the 10
documents will be run individually.

To achieve your desired mechanics, you can use a series of Flow Control steps to split into
processes/JVMs and then again into threads per JVM.

***If the number of units is greater than nodes, processes or threads then they will be
“doubled” up on available nodes.

93

Internal Use - Confidential

 3.3.7 Atom, Molecule, Cloud Runtime Administration and Maintenance FAQs
and Common Errors

If required, AtomSphere users can install atoms, clouds, or molecules locally on their own
systems. System Administrators can then customize these installations to take full advantage
of system resources. One common reason to do this, is to add more memory to the Java
machines, for higher volume processing. This article will primarily focus on maintenance and
administration issues, including some common errors and questions.

Common Errors
Error: \\<path to
install>\boomi\accounts\<account>\execution\history\...\executionResponse.dat
(The system cannot find the file specified)

Depending on the volume of executions on a particular molecule or cloud, this issue can be
caused by network latency between a particular node, and the file share. Check and remove
the external switches between the nodes.

Error: Error-193-0xc1-Windows-could-not-start-the-atom-service-on-the-local-
machine

Error 193 0xc1 is occurred when the application manager system of windows fails to find the
exact path of the .exe file of some program to run the service.

3.3.7.1.2.1 Solution 1:

Check if there is any file which has the duplicate name in the Atom installation drive.

• For example if folder C:\Boomi AtomSphere & C:\Boomi.txt are present ,rename the
text file Boomi.txt to Boomi1.txt
• Start the service.This will remove your Error 193 0xc1

94

Internal Use - Confidential

 3.3.7.1.2.2 Solution 2:

If you don’t have any duplicate program file in C drive and still facing this error then it means
the path set to executable file is modified. In this case , you have to set this path manually.

• Click on Start,Run and type services.msc

• Right click on the Atom,click on properties & in properties window check Path to
executable.Take a note of this path.
• Click on Start,Run and type Regedit.
• Go to this path HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ and select
that atom service and check the Image path.Take now of this path too.
• Check if both path are matching exactly. If not then it means you have detected the
reason of error. Now you just need to make updations in these paths one by one. First
try to change path under service’s properties windows and if you still get error then
try to update path in registry. Hopefully this will remove your Error 193 0xc1.

3.3.7.1.2.3 Solution 3:

If the problem still persists

• Open the path C:\Windows\System32(64).

• Change the sort order on the page to Alphabetical Order, and locate msiexec.exe
• Now look right above msiexec.exe you should see a new file called msiexec with no
extension. The size of the file should also be 0kb. Delete this file.
• Now open services, and start the Atom service. It should now successfully start!
•

If the problem still persists please contact your Desktop Administrator or

contact https://support.microsoft.com/en-us

95

Internal Use - Confidential

 Error: Starting a Windows atom as a service gives Error 1: Incorrect function

When an atom is installed, Java 7 or 8 is required. The atom installer will use the latest
version it finds, or it will download and attempt to install Java.

To determine which Java your atom is configured to use, view the contents of:

<atom_installation_directory>\.install4j\inst_jre.cfg and pref_jre.cfg *(The pref_jre.cfg file

takes precedence.)*

Verify that the JRE specified by the files above is not corrupted. If it is corrupted, missing, or
an incorrect path, create/modify a pref_jre.cfg file and add the location of the JRE or JDK
that the Atom, Molecule, or Atom Cloud should use. Stop and start the AtomSphere service.

For examples, refer to the guidance at this link:

http://help.boomi.com/atomsphere/GUID-58EF2B6A-4CF7-461C-BA67-CC56FCD0E29F.html

If you continue to have trouble, check the atom.vmoptions file to see if the atom memory
request does not exceed the physical memory

of the server, or the VM. The same error would be generated, if for example the VM has 2 GB
of RAM, and the atom is requesting 3 GB.

Error: java.io.IOException: Unable to tunnel through proxy. Proxy returns

"HTTP/1.1 407 Proxy Authentication Required"

3.3.7.1.4.1 Description:

An existing Atom's configuration is changed to tunnel through a proxy, following these

instructions:

96

Internal Use - Confidential

 http://help.boomi.com/atomsphere/GUID-20C748C6-05EA-41FF-8750-F47D4873FA9D.html

However, now the Atom goes on and offline intermittently. The following entries can be
found in the container log for this Atom:

[com.noelios.restlet.ext.net.HttpUrlConnectionCall sendRequest] An error occurred during

the communication with the remote HTTP server.

java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy
Authentication Required"

at
sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:1585)

at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegate
HttpsURLConnection.j

Installing a new atom, with the same proxy settings, result in a Bad Username/Password
error.

3.3.7.1.4.2 Solution:

This error results from invalid proxy credentials, as indicated by failure of the new install.
Please reach out to your Network team to determine the accurate credentials corresponding
to the proxy host provided for the Atom.

97

Internal Use - Confidential

 Error: Caused by: java.io.IOException: No space left on device
((org.hsqldb.HsqlException)) Caused by: No space left on device
((java.io.IOException))

On Linux, you can use the df command to get a full summary of disk space, and inode usage
of the system

1. To check the file system disk space statistics in “human readable” format, means it gives
the details in bytes, megabytes and gigabyte.

Command: df -h

2. To check the file system type of your system use the option ‘T‘. It will display file system
type along with other information.

Command : df -T

3. To display the information of number of used nodes and their percentage for the file
system.

Command: df -i

Error: Failed initializing store file; Caused by: There is not enough space on the disk

3.3.7.1.6.1 Description:

Processes fail on a molecule with the above error.

98

Internal Use - Confidential

 3.3.7.1.6.2 Solution:

1)Check and verify whether the file share server (NAS) is running out of inodes. For NetApp,
the default file cap in terms of volume is three million. You may have to consider increasing
the maxfiles to say 50 million as described in the link below :

hard drive - Linux - Help, I'm running out of inodes! - Server Fault

If you are hitting an inode limit on a particular directory, you can change the directory level
nesting level, of both the data and execution directories:

Atom Data DirectoryStructure

Process ExecutionDirectory Structure

You may also choose to modify the purge settings to reduce the number of files in these
folders.

2) It may not be possible to add more disk resources to the molecule. If necessary, you may
have to consider installing a new one along with a separate file share. Or, see the KB article
titled: How to Migrate Your Atom to Another Server

3) In order to eliminate the log entries in process executions, Low latency option is
recommended:

http://help.boomi.com/atomsphere/GUID-A813E1FE-7E91-41B3-ACF8-0FAE1A054276.html

Error: Your atom did not restart after a release and appears to be waiting

Examine the container log for the following lines:

Mon dd, yyyy hh:mm:ss z INFO [com.boomi.container.core.AccountManager updateStatus]

Updating account manager status from STARTED to STOPPED

99

Internal Use - Confidential

 Mon dd, yyyy hh:mm:ss z INFO [com.boomi.container.config.ContainerConfig setStatus]
Container status changed from STOPPING to STOPPED: Atom is restarting in order to apply
updates

Mon dd, yyyy hh:mm:ss z INFO [com.boomi.container.core.BaseContainer

restartContainerProcess] Atom restart initiated.

If it appears that the atom only tried to restart once, then you may have an older version of
the restart.sh script. There is a new script available that will attempt to restart the atom five
times.

Please create a support case within the Support Portal and supply the following information:

• URL from your account management page

• Attach the atom container log for the day of the release that this issue occurred
• Attach the restart.bat, restart.ps1 and restart.sh file from the <atom installation>/bin
directory

FAQ
What is jgroups and how is it used for clustering in molecules/clouds?

JGroups (from http://www.jgroups.org/) is toolkit for reliable messaging that is written in

the Java language. It is used to create clusters whose nodes can send messages to each other.

Why is this important to know? Boomi uses JGroups as the clustering across atom nodes
within a molecule or cloud. Boomi is a development platform and as such it is important to
understand this technology that is an important aspect of communication within molecule and
cloud clusters.

Boomi molecules and clouds requires nodes in the cluster to be able to communicate across a
network. By using JGroups, nodes within a molecule or cloud are able to join/leave the
cluster. A single atom node in the cluster is elected as the “Head” atom. The nodes within a
cluster are able to detect membership of other nodes in the cluster and recognize if other
nodes have joined, left or crashed in the cluster. If nodes are detected as left or crashed,

100

Internal Use - Confidential

 they are removed from the cluster and if the node that crashed or left is the leader (Head), a
new head is then elected.

Messaging across the nodes is handled by JGroups. JGroups comes with a large number of
transport protocols, for the molecule and cloud we support UDP (Multicast) or TCP (Unicast).
Nodes may “see” messages from other molecules or cloud clusters in the same network, and
will ignore them, only acknowledging messages from nodes within the same molecule or cloud
cluster. Some environments support unicast over multicast for better reliability and unicast
requires specific ip addresses and ports be designated for the nodes in the cluster. With
JGroups, lost messages get retransmitted. The head node retrieves messages from the
AtomSphere Platform, schedules processes, and the headship migrates to a new node if the
"head" dies.

If the nodes are unable to communicate within the cluster, then multiple atoms could be
designated as the head atom which will lead to conflicts, duplicate executions, etc… so the
cluster is dependent on a reliable system network configuration.

It is Important that you monitor the state of health of your molecule or cloud cluster on a
regular basis. This link contains recommendations in the Reference Guide for Cluster
Monitoring that must be implemented for every molecule and cloud.

Monitoring Atoms,Molecules and Clouds

For best performance, we recommend upgrading your molecule or cloud to jgroups3 as

described at this link.

To Upgrade ClusteredAtoms to JGroups 3

How to find program path for an exe on windows machine?

If you know the windows service/process name, there are several ways to determine the
executable path for this service or process:

101

Internal Use - Confidential

 1. You can press Win+R or from windows START search RUN then type services.msc . Right-
click on the service and chose properties . You will see it under the executable path.

2. You can press Win+R or from windows START search RUN , type msinfo32.exe into the Run
dialog box and click OK which will run the System Information program.

Expand Software Environment > Running Tasks and take note of the Process ID column. All you
need to do is match the PID number from the Task Manager with the Process ID in System
Information. The Path column will show the program’s path.

If you want to see how it works on different windows version then please check here:

Determine Program Path from Task Manager for Running Process • Raymond.CC

How do I use Release Control Scheduling?

Release Control Scheduling is used to provide a scheduled update, which allows you to test,
monitor, and develop using the latest version of the Atomsphere Platform, before the 'forced'
release date. This is an available feature that must be requested to be enabled before it is
available to an account.

To locate Release Control Scheduling, navigate to the Atom Management page in Integration
(Manage > Atom Management), select your Atom on the left, and click the Release Control
Scheduling tab located at the bottom of the page.

How to Calculate Scheduling:

Scheduling is based off of 4 simple options:

• Schedule Type
• Scheduled Day
• Scheduled Hour
• Scheduled Timezone

102

Internal Use - Confidential

 3.3.7.2.4.1 Schedule Type Details:

This is most likely the least self-explanatory of the above options. Let's use the following
"Calendar" as an example:

Wednesday (1st) is the date that Release Control becomes available.

Wednesday (15th) is the date that the Release is pushed automatically to users that have not
updated yet.

First Week = 1st-7th

Last Week = 8th-14th

For Example:

Last Week - Thursday - 2am - EDT = Thursday, the 9th @ 2am EDT.

Last Week - Tuesday - 2am - EDT = Thursday, the 14th @ 2am EDT.

Note: this is just an example. For a listing of actual Release Control and Release Dates,
please visit: http://trust.boomi.com/notifications

Why are older connector versions left in the atom after a release?

Boomi may retain older connectors in the atom. There are functionality and performance
trade-offs to consider and therefore we decided not to remove each connector after each use
or after each release.

103

Internal Use - Confidential

 The connector files take up very little space and older connector files should not pose an
issue. Also, a typical use case is once a connector version is used, the atom could use it again,
so in general we do not remove them even after a release and rollback window is complete.

The Manage -> Atom Management -> Atom Connector/Versions tables may also contain
connectors in the list that are no longer used in deployed processes.

If older connector files are causing an issue for you or consuming disk space that you need,
please open a support case and we can help mitigate the issue for you and advise on how to
address the concern.

If there is interest in this feature to remove old connectors, we could consider it in a future
release. Please submit a request via Feedback item and our Product Management team can
look into your request in more detail as well.

How can I enable UNICAST on my Molecule/Cloud if it cannot communicate with the

platform?

The reference guide describes how to make this change if the atom is running and properly
communicating with platform.boomi.com. However, if the atom/molecule/cloud will not
start, you cannot modify its properties from Atom Management.

Add the following lines to <atom_install>/container.properties ( substitute IP address and

port number as applicable )

com.boomi.container.cloudlet.clusterConfig=UNICAST

com.boomi.container.cloudlet.initialHosts=10.172.4.17[7800]

104

Internal Use - Confidential

 How to associate the Atom with the physical machine that is hosting the Atom?

On the main Atom Management page, the "Atom Host" will identify the hostname of the
machine running the atom.

In one case, there was a customer who had multiple machines with the same name. In this
case, we created a test process, with a program command, in which we were able to retrieve
the output of the ifconfig command. This allowed us to identify the IP address of the specific
machine that ran the atom.

How is my atom impacted if the date/time on my server is incorrect?

3.3.7.2.8.1 Description:

The server is accidentally started with an incorrect date/time for the physical location of the
server.

3.3.7.2.8.2 Solution:

In this case, scheduled processes will execute, but the platform will not accept requests from
the atom due to SSL security. Therefore, the atom will not be able to report the status of
those executions.

Once the server time is corrected, the execution information will appear, HOWEVER, it will
have the date/time that was associated with the execution when it was run using the
incorrect date/time on the server. This means that you could potentially have executions that
appear to have been run in the future or in the past.

The atom timezone is based on the system clock setting of the machine that the atom is
running on. To change the atom timezone, you would need to change the setting of clock of
the machine. This is by design.

105

Internal Use - Confidential

 How are lock files used in an atom or process?

The atom.lck file prevents the atom from running as a service (or executable) as more than
one OS process (e.g. "pid")

The process.lck file prevents a process from running simultaneously. This is directly tied to
the process option to allow processes to run simultaneously being unchecked.

Scheduling to clear .lck files on a regular basis is not recommended.

In order to determine why .lck files are not getting cleaned up, each occurrence should be
looked at in detail to determine why the process.lck file did not get deleted as part of the
execution completion clean up. There are many reasons for this, for example:

• the atom had a catastrophic failure and clean up did not occur or did not complete
• the atom server had a catastrophic failure and clean up did not occur or did not
complete
• a process was manually cancelled, and the process cancellation clean up did not
complete
• a long running process got hung up (for any number of reasons) and clean up did not
occur
• the cluster, atom server or fileshare could be constrained to the point that it is not
able to handle the current load and as a result file I/O does not keep up with the
application

If .lck files block a process execution on a frequent basis, an architecture review is

recommended.

106

Internal Use - Confidential

 How do I restart a local atom?

Your atom runs as a server on the host machine it resides. You need to login to the host
machine and stop and restart the Atom service. If your atom is running on a Windows server,
you can follow the below article:

http://help.boomi.com/atomsphere/GUID-CD9CAAA7-616D-4D3B-ACC1-75BD26A31C7D.html

If you don't have access to the host machine go to Atom Management page and open Atom
Properties. Click on "Restart on Save?" checkbox. Refresh the page and confirm that atom was
restarted by checking Last Atom Restart in Startup Properties tab.

Why can't a particular user see the Folder Permissions Dialog?

The Available Roles field shown in the link below cannot be seen when setting folder
permissions. Only the Assigned Roles field can be seen.

http://help.boomi.com/atomsphere/GUID-421F756C-AB0B-49B1-9AEF-5CB8EA63AC33.html

The user trying to set folder permissions must be have an Administrator role to the account
where folder permissions are being set. Any Administrator to the account can give users
Administrator privileges.

My Atom will no longer start, what can I do?

If you are having trouble starting the atom service on Linux:

1) First check the status of atom using below command,

./atom status

2) Kill the demon thread process using below commands,

ps -eaf|grep atom

107

Internal Use - Confidential

 You will get process ids, then you need to kill those processes using below command,

kill -9 pid

Removing lock files from atom installation bin directory,

rm -r *.lck

3) Then start/restart the atom using one of the below commands:

./atom start

./restart.sh

3.4 Connection Usage and Licensing FAQ’s

3.4.1 How Does Connection Usage and Licensing Work?

• Generally, your account will be provisioned with some number of connections licenses
based on your subscription. This means you can deploy up to that many connections across
your various integration processes. Connection licenses are consumed as you deploy
processes containing unique connections.
• If you attempt to deploy more connections than provisioned in your account, you will be
blocked from deploying processes containing net-new connections until more connection
licenses are provisioned. You can try to “free up” connection licenses by detaching unused
processes.
• You can view connection usage within Integration, account menu > Setup > Licensing. See
Licensing for complete details.

3.4.2 What’s the Difference between a Connector and Connection?

• A connector is the adapter that moves data in and out of integration processes. Connectors
abstract much of the technical details for establishing connectivity to a given application

108

Internal Use - Confidential

 or data source API, allowing you to focus on the record-level interaction and business
requirements. More info about connectors can be found here.
• A connector is really a combination two components: a connection and an operation. Think
of the connection as where you want to go and the operation as what you want to do
there.
• A connection represents a specific instance/location of an application or data source. Keep
reading below for more information.
• Licensing is based on connections not connectors. You are not provisioned specific types of
connectors (e.g. database, Salesforce). You are provisioned the total number of endpoints
to which you want to integrate. See "What are Connector Classes?" below.
• The full list of connectors can be found here.

3.4.3 What Represents a Unique Connection?

• A connection represents a specific instance/location of an application or data source. For

example, you would need a separate connector per:
o FTP host with user credentials
o Logical database on a given server with user credentials
o REST web services base URL
o SOAP web services WSDL/endpoint
o Business application (e.g. Salesforce, SAP) credentials
• Technically speaking a connection represents a unique set of configuration values (e.g.
username, password, host, etc.). Whenever a different set of values is required, a
separate connection must be used.
• However, a separate connection is NOT needed for each action you wish to perform
against a given endpoint/API (for example, “query Salesforce customer” vs. “create
Salesforce customer”). Multiple different actions (represented in Integration as operation
components) are intended to be paired with a single connection across different
integration processes.
• Note that two connections with the same name and identical configuration are still two
separate connections. Similarly, a copy of a connection is still a separate connection.

3.4.4 What are Connector Classes?

• Connection licenses are categorized and provisioned by connector class: Enterprise,

Standard, Small Business, and Trading Partner. For example:

109

Internal Use - Confidential

 o Small Business - QuickBooks, Sage 50/Peachtree
o Standard - SFTP, Disk, Mail, Salesforce, NetSuite, HTTP Client, Web Services SOAP
Client
o Enterprise - SAP, Oracle E-Business
o Trading Partner - Trading partner (X12, HL7, EDIFACT)
• The full list of connectors and their classes can be found here.
• You can deploy any type of connection within that class. The specific connector type is not
important. For example, you may intend to use your three standard connections for
Salesforce, a database, and an FTP server but technically you can deploy any three
standard connections.
• Trading Partner licenses work the same way however count unique trading partner
components instead of connection components. Note that trading partner components
effectively contain embedded connections.

3.4.5 How are Connections Counted?

• When you deploy a process, the unique connection components referenced within that
process (either by a connector step or embedded within another step or component) are
counted.
• A connection license is consumed for each unique connection component deployed to a
given runtime: Atom, Molecule, or Atom Cloud. If your account has multiple attachments
to the same Atom Cloud, those attachments are considered separate runtimes.
• Only connections referenced within a deployed process are counted against licensing.
Connection components created in the Build console but never deployed do not count.
• Some common scenarios:

Table 3: Licenses and Scenarios

Scenario Connection Licenses Used

A single process
with 2 unique 2 connections
connections is
deployed to a
single Atom.

110

Internal Use - Confidential

 Scenario Connection Licenses Used

Unique connections = A + B = 2
2 connections

Two processes
each containing
the same 2
unique
connections are
both deployed to
the same single
Atom.

Unique connections = A + B = 2
A single process 4 connections
with 2

111

Internal Use - Confidential

 Scenario Connection Licenses Used
connections is
deployed to an
environment
with 2 Atoms
attached

Unique connections = Atom 1 (A + B) + Atom 2 (A + B) = 4

The process to deployed to both Atoms. Each Atom is considered a separate logical runtime
engine.
2 connections

A single process
containing 2
connections is
deployed to a
Molecule cluster
consisting of 3
nodes.

Unique connections = A + B = 2

A Molecule is a single logical runtime engine regardless of how many nodes (servers) it spans.

112

Internal Use - Confidential

 3.4.6 Production vs. Test Connections

• This applies to accounts with the Test Connection Management feature enabled.
• Environments are classified as either “Test” or “Production”. This is a user-defined
classification however note that certain Atom Clouds (most notably the Boomi Atom Cloud)
may only be used in a production environment.

Figure 37: Environment Properties

• Whether a deployed connection counts as Test or Production depends on the classification

of the environment to which it is deployed. There is no distinction in the connection
component itself.
• A process deployed to a “Production” environment will count against Production
connections. The same process deployed to a “Test” environment will count against Test
connections.
• Test connections are provisioned and used separately from Production connections.

Table 4: License Scenario

Connection Licenses
Scenario
Used
A single process containing 2 connections is deployed to both a "Test"
2 Test connections and 2
environment containing a single Atom and a "Production" environment
Production connections = 4
containing a Molecule...

113

Internal Use - Confidential

 3.4.7 Do Published Web Services Count as Connections?

• No. If you publish a web service process, the listen/receive action in the beginning of the
process does not consume a connection because it uses the Atom’s shared web server. In
fact, those types of connectors do not even have a connection component. Connections to
other endpoints elsewhere in the process do count towards usage as normal.
• These connectors are:
o AS2 Server
o MLLP Server
o Web Services Server
• The ability to publish web services is an account-level feature. Once enabled you can
publish any number of web services listener processes.
• Keep in mind if you use an HTTP Client or Web Services SOAP Client connector to invoke
another Integration web service process you've published, that client connection DOES
count against licensing.
• Note some connectors that implement the Listen action such as JMS and SAP do have
connection components and therefore count towards usage. Technically these are client-
initiated subscribers and do not leverage the Atom's shared server.

3.4.8 Are there any Free Connectors?

• Yes. The connectors for AtomSphere Platform features do not count against licensing. You
can deploy these connections for free.
• As discussed above, the shared web server connectors do not count towards licensing
either.

3.4.9 Do Custom Connectors Developed Using the SDK Count Towards

Licensing?

Yes, connections for custom connectors developed using the Connector SDK count towards
licensing.

114

Internal Use - Confidential

 3.4.10 How Can I Be Smart about My Connection Usage?

• Reuse connection components! Look to reference an existing connection within your

account before creating a new one. This will reduce confusion and simplify maintenance in
addition to minimizing connection usage.
• To help with reuse, as best practice, organize and consolidate all connection components
into a single common folder within the Build console. You may wish to restrict access to
this folder to administrators or super-users.
• As part of your process deployment procedure, confirm only “official” connections (those
residing in the common folder) are referenced in the to-be-deployed process. Use Compare
Deployment to compare the list of components in the to-be-deployed process version to
the previously deployed version to determine if any new connections have been introduced
by accident.
• Use environment extensions instead of having separate connections for each environment,
such as sandbox vs. production.
• Sometimes separate connections are required while developing and testing in Test Mode,
requiring you to configure connection values directly vs. extensions. In these situations,
create a separate connection but be sure to replace with the “official” connection before
deploying. Adopt a naming convention to avoid confusion (e.g. “Database ABC DEV DO NOT
DEPLOY”).
• Understand and take advantage of the configuration and parameterization options of
certain connectors:
o Disk - Unique per file path however this can be parameterized. Recommend only
ever having a single Disk connection for your entire account and parameterize
everywhere.
o FTP - Unique per FTP host and credentials. Remote directory path is configured
separately in the operation component.
o Database - Unique per logical database and credentials on a given database server.
For example, if you have multiple logical databases on the same server, you will
need separate connections. Similarly, if you require different user names to access
different schemas in the same logical database, you will need separate
connections.
o HTTP (e.g. REST) - Unique per base URL. Recommend configuring only the base URL
(e.g. www.myendpoint.com) in the connection and pairing with different or
dynamically-configured operations to connect to multiple resource paths.

115

Internal Use - Confidential

 3.4.11 Where Do I View My Connection Licensing and Usage?

You can view the number of connections provisioned and used in your account within Integration,
account menu > Setup > Licensing. See Licensing for complete details.

Figure 38: Account menu > Setup > Licensing

(1) Connections by Class - The red section (top) shows a summary of connections by purchased
(provisioned), deployed (used), and available by connection class.

(2) Connectors Deploy in Class - The blue section (bottom left) shows the connections deployed for
the class selected above.

(3) Deployments - The green section (bottom right) shows deployed connections along with the
process and Atom/Molecule/Cloud. This list can be filtered by selected the class above and/or
connection to the left.

3.4.12 How Do I Get More Connections Added to My Account?

If you need more connections provisioned in your account to be able to deploy additional
processes, please contact your Boomi Account Manager. Your Account Manager is listed in the
Support Portal under “Account Info”.

116

Internal Use - Confidential

 3.4.13 Common Licensing Issues

How Do I Determine Where a Specific Connection is Used?

To determine in which deployed processes a given connection is used, do the following:

1. Within Integration, go to account menu > Setup > Licensing.

2. Determine the connection class for the given connection (for example, Standard) and click
the Connector Class in the top section “Connections by Class”. This will filter the bottom
left section “Connectors Deployed in Class”.
3. Next in that bottom left section, click the connection name (e.g. “My Salesforce
Connection”). This will filter the bottom right section “Deployments”.
4. Finally, “Deployments” will show all deployments (processes and Atoms) containing the
given connection.

You can use this information to determine if the connection is accidentally referenced by a
process or incorrectly deployed to an Atom.

In example below, the licenses are filtered for a SFTP “Test” connection named “New SFTP
Connection.” It shows the connection referenced by two processes, “Test” and “Disk Test”.
However, because both processes are deployed to the same Atom (“personalDev”) only one
Standard (Test) connection license is used.

Figure 39: Licensing Window

117

Internal Use - Confidential

 Deployment Warning Message: “Deployment Rejected: attempted
to deploy X more Standard Connection(s). Doing so would exceed your
purchases for this connector class...”

This warning means the process(es) you are attempting to deploy will result in more deployed
connections than are allowed in your account for the given connector class. To resolve, try the
following:

1. Review your process configuration for redundant or extra connections.

• Modify your process to reuse an existing deployed connection vs. creating a new
connection (if appropriate).
• Remove extra/unused connection references. Note connections referenced by
disconnected steps (perhaps left over from development/testing) will still count
towards usage so remove those shapes from the process.
• Remember connections can be referenced by shapes other than Connector shapes,
such as Decisions, Set Properties, Map Functions, etc.
• Confirm the number of available connections for the given connector class using
the Licensing tab. If there are not enough connections available, try the following
to free up connections:
2. Reduce your deployed Standard connections, by using the "How to Determine Where a
Specific Connection is Used" step above.
• Look for opportunities to consolidate redundant connections that are really
connecting to the same endpoint or can be parametrized for reuse. See "How Can I
Be Smart about My Connection Usage?" above.
• Look for processes that are no longer used (perhaps left over from testing or proof
of concept development) and detach them from the environment or Atom.
• If using environments, look for and remove unused environments. Similarly, within
a given environment, detach any extra Atoms.
3. Contact your Account Manager to obtain additional connection licenses.

118

Internal Use - Confidential

 3.4.14 Why Does My Connector List in the Licensing Tab include
Connectors that have been Deleted?

If you notice a deleted connector listed in the Licensing tab:

• Select the connector on the left side of the screen and note the process name(s) from
the listing on the right side of the screen.
• From the Deploy tab, select the appropriate process names and use the gear icon to
select Compare Process Deployments for the most recent deployment of this process.
• Click on Next to view the comparison of the latest deployment to the Latest Revision
of the Process.
• Highlighted in orange you will see the component(s) that have been deleted since this
process was last deployed. One of the orange highlighted components will be the
connector from the Licensing tab.

What does this mean?

While the connector is not being used in the latest revision of the process in the Build
tab, it is part of the latest revision of the process that is deployed. Connector
licensing is determined by connectors that are deployed by class.

How do I fix this?

The deployed process is using a connector that has been deleted from the Build tab.
First, determine if the connector was appropriately deleted. That is, what connector
should be used in place of the deleted connector? Then, once it has been verified that
the latest revision of the process is ready to be deployed, deploy the process by
clicking on "Deploy Latest Revision of Process".

The Licensing tab should no longer list this process for the deleted connector and the
appropriate Atom Name once the process has been re-deployed.

The connector deleted from the Build tab can be viewed and restored by doing the

119

Internal Use - Confidential

 following:

• The deleted connector can be viewed in the Build tab by using the Internal ID listed in
the Licensing tab
• Typing “id:<Internal ID>” in the Component Explorer box in the Build tab and press
enter. [Please note that the Internal ID on the Licensing tab and the Component ID are
synonymous.]
• Saving the component will restore the component in the Build tab.

3.5 Environments

3.5.1 Overview

One of the first tasks when setting up the AtomSphere Platform is establishing your runtimes.
Whether it be in the Boomi Cloud, a local Atom, or a Molecule, how these runtimes interact
with the platform is encapsulated by the AtomSphere environment object. Environments can
be thought of as logical containers for the runtime. Planning your environment strategy is
crucial to establishing your deployment methodology, user security, and your testing
environments.

Environments play several roles in the platform:

• Abstraction of the runtime for deployment. Processes are deployed to an environment,

not directly to a runtime.
o Attach Atoms, Molecules, or Clouds and effectively become the container for
the runtime.
o More than one runtime can be attached to the same environment
• Restrict user access to runtimes using custom roles / segregation for different
groups/projects/teams
• Classify environments as test or prod for purposes of AtomSphere connection licensing

120

Internal Use - Confidential

 • Software development life cycle (SDLC) and version control for taking snapshots of
your integration configuration
• Define process configuration variables ("extensions") for processes deployed to a given
Environment

3.5.2 Considerations for Determining Your Environments

In order to lay out how many environments are needed, consider:

• SDLC stages and other app instances

• Runtime deployment topology
o Cloud vs. local/hybrid
o Multiple data center
• User access
o Privileges
o Segregating projects or regional teams
• Use-case specific runtimes

3.5.3 Software Development Life Cycle

Environments lay out the journey a process will take from initial configuration on the build
tab, all the way to production. Processes typically are deployed from the build tab to the
initial development environment, then promoted up to other testing environments and
eventually to production.

3.5.4 Deployment and Version Control

Within the platform, processes are deployed directly to environments, and only indirectly to
runtimes. A runtime can also be attached or detached from one environment to another. This
allows a new Atom installation (think server replacement) to not cause the loss of deployment
version history. These versions, or deployment snapshots, are maintained in the AtomSphere
Platform and associated with the environment. A process can be deployed to the
environment, the Atom detached from the environment, then later re-attached to show the
prior version history.

121

Internal Use - Confidential

 Using this, we can setup multiple environments, even if we do not have a separate runtime
for each environment. Environments are typically a one to one with an Atom or Molecule
installation. However, we can have multiple environments setup and only have one runtime
setup. Think of maintaining a Regression Testing environment that only needs to be
periodically used. The Atom could be resident of the Test Environment but then detached for
a period of time to be used in the regression environment, where a different deployment
version of an integration resides.

3.5.5 Environment Variables

Environments also contain the override variables to be used by an integration, defined by

extensions within processes, including endpoint connections. By maintaining environments
and using extensions, we can deploy the same code across multiple environments, with
connection parameter variables defined at the environment rather than within the process
configuration. This allows us to construct environments to meet the requirements of the SDLC
- no code changes needed when promoting a deployment.

How many instances of an endpoint exist for testing could justify additional environments. If
there is only one sandbox and one production, at least two environments are needed. If there
is a sandbox, preview, and production tenant, you may wish to maintain three environments.

Traditionally, the SDLC landscape would comprise DEV, TEST, and PROD. DEV would be an
unstable testing ground, TEST would mirror production and be used for UAT, and then
ultimately land in production. Some business requirements require additional environments
for regression testing, load testing, etc. These should be factored into your environment
landscape and SDLC.

3.5.6 Deployment Topology

Hybrid deployments and distributed deployments will also justify additional runtimes, and
therefore environments as well. Think of running some integrations in the Boomi Cloud and
running other integrations locally in an on-premise Atom for local applications. As companies
move to the SaaS space, legacy systems are often still in play. Establishing a local and cloud
environment can help during this transition.

122

Internal Use - Confidential

 We can also have distributed deployments, a group of Atoms or Molecules, that all run the
same processes and use the same environment variables. Here we are taking advantage of the
many to one relationship runtimes have to environments. We can install multiple Atoms or
molecules all grouped under one environment. This allows for the same integration to be run
on multiple runtimes using the same deployment version and extension variables. Keep in
mind with the approach that although deployments are at the environment level; schedules,
persisted process properties, passwords, and license usage is all done at the Atom, or runtime
level.

3.5.7 User Access and Roles

We can define which Boomi users can access that environment. In this way only, users with
the correct permissions can access the environment, including deployment, process
monitoring, or execution abilities.

Boomi is a shared platform for configuring your integrations. However, some Integration
requirements deal with sensitive data and may not be accessible to all your Integration
builders.

Boomi uses environments to control user access to the integration process execution. With
custom roles, users can be setup to be restricted to an environment that holds the sensitive
integrations. Then only they would be able to execute integrations or see execution history
for those integrations that reside there. The runtime restrictions work in tandem with any
permission restrictions setup in the custom role (e.g. Build only, process monitoring only).
Keep in mind that on the Build tab, all Integration Developers have access to components.

Additional runtimes could be required to create isolated environments to control user access.

3.5.8 Integration Use Case-Specific Runtimes

Sometimes diverse types of integrations can be grouped according to key patterns. Depending
on the nature of your integrations, the runtime configuration may vary and require separate
runtimes by integration type. For example, batch or ETL processing often has different
resource demands than high volume real-time processing. Consequently, you may decide to
segregate these integrations into optimized runtimes for each.

123

Internal Use - Confidential

 3.5.9 Environment Setup for Different Scenarios

Let’s look at a few scenarios to see how our environment landscape takes shape.

Basic SDLC

• For SDLC, we will ideally maintain three environments: DEV, TEST, and PROD.

• Some of our applications have two sandboxes, some only have one.
• Integration users have access rights to all integrations however users have different
permissions depending on role.
• Single runtime needed per environment (no distributed runtimes)

Recommendation: Three environments to match SDLC.

Table 5: Three Environment Table

Environment Dev Test Prod

Runtime Atom Molecule Molecule

• Deployer: Deploy,
• Developer: Deploy,
Manage Extensions
• Developer: Deploy, Monitor, Execute
Role • Prod Support: Monitor,
Monitor, Execute • Business User: Monitor,
Restrictions Execute
Execute
• Business User: Monitor

Sandboxes: Sandboxes: Production:

Endpoint App • App 1 • App 1 • App 1 PROD

• App 2 • App 2 (UAT) App 2 PROD

124

Internal Use - Confidential

 Figure 40: Environment Model

Sensitive Data

• For SDLC, we will maintain at least 1 test environment and production.

• Only HR developers can access HR integrations. Non-HR Boomi users own other
business integrations.
• Single runtime needed per environment (no distributed runtimes)

Recommendation: use separate environments to restrict user access for sensitive integrations
vs. non-sensitive.

For example, if your SDLC has only two tiers (e.g. test and production) and you want to
maintain isolation throughout test and production, you would need 4 environments.

Table 6: Four Environment Table

Environment HR Test HR Prod Biz Test Biz Prod

Runtime Atom Atom Atom Atom
Deployer: Promote to
HR Developer: HR Admin: Promote to
Role Developer: Deploy, Production, set
Deploy, Monitor, Production, set environment
Restrictions Monitor, Execute environment variables
Execute variables, view HR executions

125

Internal Use - Confidential

 Environment HR Test HR Prod Biz Test Biz Prod
Prod Support: Monitor,
Execute

Business User: Monitor

Sandboxes: Sandboxes:
HR App 1 Prod App 1 PROD
Endpoint app • HR App1 • App1
• HR App2 HR App 2 Prod • App2 App 2 PROD

Figure 41: Environment Model

Additional setup:

• Custom user roles setup per environment.

• White list sensitive data runtime IP at application level
• On the Build tab:
o Folder permissions enabled, and restricted by custom user role
o For connection components, force extensions (no credentials saved in
connector)

126

Internal Use - Confidential

 o If credentials must be saved (for browsing), use a separate connection
component with dummy data or user permission restricted to schema only (no
data), or whitelist IP to Boomi HR server only

Hybrid Deployment

• For SDLC, we will ideally maintain three environments: - DEV, TEST, and PROD.

• Applications may have multiple test environments, some are in the cloud and some are
behind a firewall
• On premise integrations and cloud-based integrations
• Integrations can be separated by either cloud-to-cloud, or on-premise to cloud.

Recommendation: 5 environments. One shared development environment, with separate

tracks for on-premise deployment vs cloud deployments.

Table 7: Five Environment Table

Environment Shared Dev Local Test Local Prod Cloud Test Cloud Prod

Runtime Atom Atom/Molecule Atom/Molecule Boomi Test Cloud Boomi Prod Cloud

• Deployer:
Deploy,
• Deployer: Deploy,
• Developer: • Developer: Manage • Developer:
Manage
Deploy, Deploy, Extensions Deploy,
Role Extensions
Restrictions
Monitor, Monitor, • Prod Monitor,
• Prod Support:
Execute Execute Support: Execute
Monitor, Execute
Monitor,
Execute

Production:
Sandboxes:

Sandboxes:
• Local App1
• Local App Sandboxes: Production:
1 PROD
Endpoint • Local App2 • Local App1
• Local App • SaaS App 1 • SaaS 1 PROD
app • SaaS App1 • Local App2
2 PROD • SaaS App 2 • SaaS 2 PROD
• SaaS App2 • SaaS App3
• SaaS App 3
• SaaS App3
PROD

License usage: 10 Test, 5 Prod

127

Internal Use - Confidential

 Figure 42: Environment Model

Optimizing Specific Integration Use Cases

• For SDLC, we will ideally maintain three environments: - DEV, TEST, and PROD.
• Some application overlap between ETL and Real-time jobs

• ETL integrations and Enterprise Real-time integrations (real time/event-based vs.

scheduled/batch. Maybe even a third for very large ETL migrations)
• Separate environments → separate runtimes → fine tune configuration for specific use
cases (high volume & small data vs. lower volume & big data)

Recommendation: 5 environments. One shared development environment, with separate

tracks for ETL processes and Real-time processes.

Table 8: Five Environments with Separate Tracks

Environment Shared Dev Real-time Test Real-time Prod ETL Test ETL Prod

Molecule
Molecule
Runtime Atom Molecule Molecule
(w/forked
(w/forked processing)
processing)

• Deployer: Deploy, • Deployer: Deploy,

• Developer: • Developer: • Developer:
Manage Manage
Deploy, Deploy, Deploy,
Role Extensions Extensions
Monitor, Monitor, Monitor,
Restrictions • Prod Support: • Prod Support:
Execute Execute Execute
Monitor, Execute Monitor, Execute

128

Internal Use - Confidential

 Environment Shared Dev Real-time Test Real-time Prod ETL Test ETL Prod

Sandboxes:
Sandboxes: Production: Sandboxes: Production:

Endpoint • App1
• App1 • App 1 PROD • App2 • App 2 PROD
app • App2
• App2 • App 2 PROD • App3 • App 3 PROD
• App3

License usage: 7 Test, 4 Prod

Figure 43: Environment Model

3.5.10 Environment Extensions

Set extension values for one or more processes deployed to Atoms attached to the selected
environment.

Clicking Environment Extensions in an Environment Properties tab opens the Extensions

dialog.

Note: You must have the Atom Management privilege (as well as Environment Management) to
set extensions on the Extensions dialog. If you have the Atom Management Read Access
privilege, you can only view existing extensions and open the Audit Logs dialog.

129

Internal Use - Confidential

 Figure 44: Environment Extensions via Atom Management

3.5.11 Add an Environment

Procedure
1. Go to Manage > Atom Management

2. On the Environments page, click and select Environment from the list.
3. Type a name for the environment.
Restriction: Environment name is restricted to 80 characters. There are no restrictions to the type
of characters allowed.
4. (For accounts with Unlimited environment support) In the Environment Classification field,
select Production or Test to indicate the environment type.
The classification can be set only when you add an environment. You cannot change it later.
Note: The Environment Classification field is not available for accounts with Basic environment
support. All environments are classified as production environments.
5. Click Save.

130

Internal Use - Confidential

 3.5.12 Conclusion

During your initial AtomSphere setup, laying out the runtimes is often a first step. With this,
environment features should also be considered to look at how this may affect your
deployment scheme. From establishing the change management path, to user access, to
unique runtime deployments, understand how environments can help you meet your
functional and non-functional requirements.

3.6 Approach for Migrating from an Existing Middleware

Platform to AtomSphere
3.6.1 Overview

This document provides a high-level approach for migrating interfaces from an existing
middleware platform to Boomi AtomSphere Platform. It discusses the unique opportunities
and challenges when migrating platforms and offers guidance for dividing the effort into
various project phases. However, this document is not intended to serve as a detailed project
plan.

3.6.2 Migrating from Another Middleware Platform

In many ways a migration project from an existing integration platform is similar to a net-new
integration project, but there are differences. It is important to be aware of both the
advantages and challenges of a migration project to seize improvement opportunities and
avoid operational disruption.

Advantages

• Familiarity with use cases - The team has familiarity with integration use cases, end
applications, and known issues. Integration management procedures are typically in
place and (hopefully!) some interface documentation exists.
• Known scope - Scope is generally well-defined, although there could be a large
number of interfaces to move.
• Historical data exists - Historical information exists for sample data, volumes and
performance as a basis for comparison. However, be aware exact metrics may differ in
comparison to AtomSphere depending on design.

131

Internal Use - Confidential

 • Management tooling - Supporting management tooling likely exists such as reporting
dashboards, alerting channels, system monitoring hooks, documentation, change
control procedures.
• Improvement opportunity - The migration project presents an opportunity to
optimize integration designs such as streamlining “bolt-on” changes over the years,
consolidating and parameterizing similar functionality, and deprecating outdated logic
and interfaces.

Challenges

• Learning curve - AtomSphere greatly reduces the learning curve and complexity of
middleware platforms and integration development, but it is still a new tool for the
team to adopt.
• New integration approaches - All middleware platforms aim to solve the same general
integration problems, but each tool will approach the problem different--AtomSphere
is no exception. Interface design will typically need to evolve from the legacy system
to take advantage of the architecture and capabilities of AtomSphere. Because of
fundamental differences in approach, rebuilding is generally required and usually more
efficient vs. migrating legacy assets. The Boomi professional services team can assist
in helping translate legacy implementations into AtomSphere designs.
• Maintain production operations - The cutover must be carefully coordinated with
existed interfaces to minimize disruption to production operations.
• Avoid impact to clients - Impact to existing clients should be avoided/minimized.
Ideally the migration would be transparent to client applications but usually that is not
entirely possible. Assess impact to client applications (especially those outside the
organization) as early as possible. Common changes can include URL endpoints,
authentication schemes and credentials, certificates, and whitelisted IPs/hosts. Data
formats should be preserved to avoid more costly parsing logic changes for clients.

Grouping Interfaces

There are several common ways to organize the interfaces into logical groups for migration.
The best grouping is the that makes the most sense for your project and should take into
account the number of interfaces and endpoints, commonality across interfaces, project
timelines, and external dependencies.

132

Internal Use - Confidential

 • Endpoint - Interfaces connecting to/from a given endpoint or set of endpoint
applications.
• Business process - Interfaces enabling a given business process such as order-to-cash
or customer-sync.
• Technical pattern/style - Interfaces using a given integration style such as batch vs.
real time, or involve a specific processing requirements. See Technical Patterns
below.
• Criticality - Interfaces supporting critical, frequently used, automated processes vs.
less critical processes.
• Individual interface - Grouping by individual interfaces with no other discernible
commonality.

Technical Patterns

Thinking in terms of technical patterns can be a great way to better understand the nature of
the interfaces and break them down into individual solvable pieces.

A pattern-based approach to migrating interfaces to AtomSphere can help organize the work
effort, ease the transition, and minimize risk. The big idea is to:

1. Survey existing interfaces to identify common patterns and/or endpoints

2. Group interfaces by pattern
3. Create templates for each pattern
4. Implement and cutover interfaces by pattern

Patterns can be generic or specific, simple or complex, functional or nonfunctional, but

typically not end-to-end workflows. Some examples:

• Endpoint connectivity application/protocol (e.g. SAP, MQ, database, REST) and API
interaction (e.g. specific queries/joins, upserts, etc.)
• Interoperability with existing integration infrastructure components such as ESB,
monitoring channels, job tracking dashboards
• Authentication scheme (especially when publishing web services)
• Notable transformation or validation requirements (e.g. validate against custom
database of allowable values)

133

Internal Use - Confidential

 • Integration “style” (scheduled vs. event-based, web services vs. messaging,
synchronous, vs. asynchronous.)

The goals for cataloging and testing technical patterns are:

• Identifying external changes required (most notably for real time interfaces such as
web services)
• Assessing level of effort and clarity of requirements for prioritization
• Discovering opportunities for reuse
• Gaining experience with the AtomSphere platform

3.6.3 Detailed Work Breakdown

Migration Phases Overview

The migration effort can be organized into several phases, each with a handful of parallel
tracks. When migrating a large number of interfaces, it can make sense to divide the
interface-related work (such as requirements gathering, development, and testing) into
multiple waves. These waves can be tackled in parallel or sequentially, depending upon
resource availability of the implementation team and inter-dependencies.

Keep in mind the first set of interfaces will be the most effort and longest duration because it
will include all the one-time activities such as common framework development and runtime
infrastructure build-out. However subsequent waves should be streamlined because those
activities will have already been completed.

Below is an overview of suggested phases and tracks:

• Phase 1 - Discovery and Planning

o Track 1 - Survey, Proof of Concept, Project Planning
• Phase 2 - Templates, Framework, and Detailed Requirements
o Track 1 - Template and Framework Development (multiple waves)
o Track 2 - AtomSphere Account Administration
o Track 3 - Detailed Business Requirements (multiple waves)
• Phase 3 - Implementation
o Track 1 - Interface Development (multiple waves)

134

Internal Use - Confidential

 o Track 2 - Runtime Infrastructure Build-Out (if hosting integrations on-premises)
o Track 3 - Performance Testing
• Phase 4 - Testing, Production Readiness
o Track 1 - Functional Testing with End Users
o Track 2 - Production Readiness
• Phase 5 - Production Cutover
o Track 1 - Production Deployments (multiple waves)

Phase 1 - Discovery and Planning

Track Activities
1. Conduct high level survey of existing interfaces to identify core
patterns. This should not be an extensive, detailed analysis of
business requirements.
• A pattern can be a fairly loose term referring to specific
business logic or technical capability.
2. Group interfaces by pattern(s).
3. Perform minimally-viable proof-of-concepts for each pattern (some of
this may have been performed during the sales cycle).
4. Assign, prioritize, and schedule interface groups into implementation
waves.
Track 1
• Look for “low hanging fruit”: relatively simple yet meaningful
- Survey, Proof
interfaces that can help gain comfort with the new platform.
of Concept,
Don’t try to migrate the most complex, critical interfaces
Project
first. Allow time for the team to acclimate to the new
Planning
platform and refine operational and support procedures.
• Consider dependencies including infrastructure,
organizational, client or framework changes outside of
AtomSphere, availability of clients for testing, and sensitive
dates for cutovers.
• If working with aggressive deadlines, consider de-prioritizing
interfaces that are less critical or those that can be mitigated
by manual processes. Involve the business team in the
prioritization decisions.

Table 9: Phase 1

Phase 2 - Templates, Framework, and Detailed Requirements

Track Activities
1. Develop initial template processes for patterns end-to-end
Track 1 - Template and 2. Review and test template process.
Framework Development • Identify and refactor common framework
(multiple waves) components
• Refine template and iterate

135

Internal Use - Confidential

 Track Activities
3. Develop framework components (e.g. subprocesses) for
nonfunctional requirements such as logging and error
handling. Hook into existing reporting and alerting
channels.
4. process monitoring, doc tracking

1. Configure AtomSphere account including:

Track 2 - AtomSphere o Deployment environments and procedures

Account Administration o End users, roles, and access
o Component organization and naming conventions

1. Gather or create business requirements for the specific

Track 3 - Detailed legacy interfaces. Depending on availability of
Business Requirements documentation this may require essentially reverse-
(multiple waves) engineering the legacy interfaces.

Table 10: Phase 2

Phase 3 - Implementation
Track Activities
1. Develop the specific interfaces for given pattern group
Track 1 - Interface
using the pattern templates and business requirements
Development (multiple
created in Phase 2.
waves)

1. Provision hardware servers and storage required for the

local AtomSphere runtime (Atom or Molecule).
2. Install and configure the runtime.
3. Perform sanity testing, establish connectivity to
Track 2 - Runtime
endpoints.
Infrastructure Build-Out (if
4. Configure infrastructure “plumbing”, especially when
hosting integrations on-
publishing web services (firewalls, load balancers,
premises)
etc.).
5. Implement runtime monitoring (e.g. JMX, log files),
hook into operational alerting systems.

1. Establish performance/SLA requirements based on

business needs.
2. Identify and create test cases based on historical
Track 3 - Performance trends. Variables will include number of records, size of
Testing records, concurrent process executions, and
simultaneous requests.
3. Repeat tests on variety of runtime configurations (CPUs,
memory) to dial in optimal throughput.

136

Internal Use - Confidential

 Track Activities
4. Adjust server and AtomSphere runtime configuration
accordingly.

Table 11: Phase 3

Phase 4 - Testing, Production Readiness

Track Activities
1. Perform end-to-end functional system testing.
Track 1 - Functional
2. Coordinate testing with end users and client applications.
Testing with End Users

1. Business continuity planning including high availability

Track 2 - Production
cluster (Molecule), backup/DR failover testing.
Readiness
Table 12: Phase 4

Phase 5 - Production Cutover

Track Activities
1. Plan production cutovers at times that are least impactful to end
users and client applications. Schedule around special dates such as
month ends or large batch jobs.
2. For scheduled/batch interfaces, the cutover can often be as simple
as disabling the interface in the legacy system and enabling in
AtomSphere. Be sure to migrate any stateful values such as last run
dates, counters, etc.
3. For web services and other event-based real time interfaces, there
are a couple options. Deploy the new AtomSphere interface then:
Track 1 -
Production o If the client application must be modified to accommodate
Deployments the new AtomSphere endpoint, then the AtomSphere
(multiple waves) interface will simply be ready whenever the client
application is modified to point to new AtomSphere
endpoint.
o If the client application will not be modified, look to
configure routing rules in the load balancer to route traffic
to the new AtomSphere endpoint. You will need to consider
the granularity of the API design/resources if doing a partial
release. Often this will involve inspecting and rewriting the
target URL based on pattern matching rules.

Table 13: Phase 5

137

Internal Use - Confidential

 3.6.4 General Implementation Tips

• Document patterns and framework components in a common place, such as an internal

wiki.
• Establish and test connectivity to application endpoints as soon as possible to work
through network access issues.
• If deploying locally, submit any internal hardware and networking provisioning
requests as soon as possible.
• Incorporate modular design elements (e.g. using subprocesses and common
components) to minimize effort to make template/pattern changes in the future.
• As the number of interfaces implementing a given pattern increases, spend more
cycles refining the template. The additional upfront design and testing effort will
greatly reduce the implementation effort and minimize rework.

4 Build the Integration Development Area

4.1 Organization

4.1.1 Establish Naming Conventions

• Label shapes descriptively when creating them.

o Start and Stop shape labeling is discouraged. Map labels are unnecessary
with good component naming (The component name will appear on the
canvas).
o For shapes without components (ex: set properties, data process,
decisions), use descriptive labels that relate to the function of the
shape.
• Example: Decision: “Sync Flag Populated?”
• Example: Set Properties: “Set DPP_ACCT_ID”
• Example: Data Process: “Split”
o Labels are optional for branch, route, process route.

138

Internal Use - Confidential

 • Name components descriptively upon creation.
o All components should have descriptive names, not Integration default
names (“New Map” or “New Profile”).
o Each component should have a unique name. Without unique names,
multiple components (aka different component IDs) will appear similar
but are entirely separate entities. Copying a component to a separate
folder will generate one or more new components with identical names
that should be changed after copying.
o Rename any components that have had a numeric suffix added by
Integration to enforce uniqueness. For example, “SF account query 1”
and “SF Account query 2” could become “SF account query by id” and
“SF Account query by name”.

4.1.2 Improve Folder Structure

• Name and Organize folders thoughtfully – group similar processes in their own
folders and name the folder accordingly.
• Create a folder such as “#COMMON” for components shared between processes,
especially connections.
• Keep only a single copy of each connection in the account. Duplicated
connections lead to confusion and, frequently, unintended Integration license
consumption upon deployment.
• In general, avoid adding components in the root folder. For small accounts,
putting shared connections in the root folder is acceptable.
• Create new components directly in the folder where it should reside.

4.1.3 Eliminate Redundant Copies of Components and Connections

• In Integration, components can have identical names but different component

IDs. If you have multiple copies of components with identical names, it’s
impossible to tell which ones are used in a process without doing further
research.
• Create only one connection component per endpoint for deployment (and
possibly 1 more for browsing). For example, rather than create separate TEST

139

Internal Use - Confidential

 and PROD connections, consider using extensions to switch a connection
between test and production. Reminder: Extensions can be set in test mode.
• Use shared components where appropriate. If you expect a component to
require two different versions during later implementation, create properly
named versions initially.

4.1.4 Create Only a Single Copy of Each Process

• "Test copies" of your processes are discouraged – they lead to duplicated

components in the account, and revision history will be reset in the copied
process.
• Changes are normally done in your single process, unit tested in test mode,
deployed to a Test or QA environment for further testing, then promoted to
production.
• If you are planning extensive changes to a process, a v2 of your process is
acceptable.
• A backup copy of a folder is acceptable only if done infrequently; consider
copying to another Integeration account if one is available.

4.1.5 Copying Components: Shallow vs Deep Copies

• Shallow Copy: Copy Component Dependents is unchecked

If we revise the component(s) in our copy of the process and save it, it will
change the component(s) in the original process because both processes
reference the same component(s).
• Deep Copy: Copy Component Dependents is checked
If we copy over the process to the same folder level, when we copy the
common component(s), it will create duplicate component(s).
If we revise the component(s) in our copied process and save it, it will not
change the component(s) in the original process because both processes do not
reference the same component(s).

4.1.6 Avoid Multiple Disk Connection Components

• Directories can be specified in a set properties shape or connector parameters

tab.

140

Internal Use - Confidential

 4.1.7 Specify Only Base Directory in FTP/SFTP Connections

• Directories can be specified in a set properties shape or connector parameters

tab.
• Multiple connections are required only for entirely separate FTP/SFTP
destinations.

4.2 Accountability

4.2.1 Avoid Using Personal Logins for Endpoints

Example: Configure your SalesForce connection to use BoomiIntegration rather

than JoeSmith.
• Rationale: Changes will be seen as coming from JoeSmith. If JoeSmith changes
his password, the integration will fail. If JoeSmith is logged in at the same
time as an integration executes, there could be a license concurrency problem.
Also, changes made by JoeSmith (person) will be indistinguishable from ones
made by the integration version of JoeSmith.

4.2.2 Email Alerts

• Encourage one or more representatives from each client to sign up for Email
Alerts as basic error handling, especially if other error handling routines are
not developed.
• Email Alerts, Boomi User Guide

4.2.3 Proper User Management

• Coordinate with the client to have them add consultants to accounts in the
proper roles.
• Determine if custom roles are required – build and allocate as necessary.
• Determine if folder-based permission is required. Assign roles accordingly.

141

Internal Use - Confidential

 4.3 Process Overview

4.3.1 Data Flow

Process Types
For Integration, people use a variety of terms for their processes, but they fall under a couple
of categories based on function and how they are used. All processes use process options such
as "Allow Simultaneous Executions" to allow multiple instances of the same process to execute
at the same time or "Purge Data Immediately" to purge process data right away so none is
stored. There are two process modes:

• General mode - logging is enabled and you can view in process reporting
• Low latency mode - logging is minimized, data is not stored anymore, you cannot view
in process reporting. Only certain connectors can be used with low latency mode
• See Process Options section for more information.

Process
A set of shapes that flow in a logical order to perform some set of actions. It can be
scheduled to run at certain intervals. Processes must start with a start shape. The start shape
can be a:

• Connector - connector to somewhere to get some data to start the flow

• Trading partner - wait for EDI data to start processing it
• Data passthrough - picks up the data right before it was called by something, typically a
main or master process. If nothing is passing data through, then an empty document
will be passed down the process
• No data - an empty document starts the process. This is mainly used for ease of moving
process starting points or testing

Main Process
Used to call on many other subprocesses, initialize information or gather information before
calling subprocesses. Also known as master process.

Subprocess
A process that performs a specific set of actions possibly to be used later. It is called by a
process, main process, or other subprocesses via a process call shape. You can think of

142

Internal Use - Confidential

 subprocesses as often used functions in programming. When a subprocess is:

• Data passthrough - the data coming from the process calling the subprocess is passed
through and used further along the subprocess
• No data - nothing is passed into the subprocess even if there is data from the calling
process

Listener
A process that waits to be triggered or waits for something else to call it. It is also called real
time integration or event-based integration.

Connectors
Connectors are where data usually comes from and goes to. Connectors are generally broken
up into two categories, Technology and Application. Technology connectors are the more
general-purpose connectors that can connect to multiple places depending on the
configuration and options. They do a specific purpose like the Disk connector gets and sends
files from a disk or storage system and the HTTP connector makes HTTP requests somewhere.
Application connectors connect only to one place (that particular application it was made for)
like the Salesforce connector only connects to the Salesforce API or the NetSuite connector
only connects to the NetSuite application and then the connector does only functions related
to that specific application.

Connectors usually start the data flow (even after a No Data start shape) for most clients. For
support, the connector is not needed after getting the data from it the first time the
connector runs. A connector is made up of two parts, the connection (the where is it
connecting to) and the operation (what is your connection going to do).

Connection
The connection is the where you are connecting to. Here you would have URLs, location
information, configuration settings specific to the application or general things for technology
connectors. Sometimes you'll see a test connection button on the top right and this will allow
you to test a connection to see if it is working or not. Connections are used to determine

143

Internal Use - Confidential

 licensing.

Operation
The operation is what to do essentially. The operation dictates what to retrieve back, how
much, what to send, etc, what actions/operations to take/do. There is a concept of action in
connectors, but it is either getting data or sending data. Technology connectors usually have
the action as Get or Send. Application connectors will have more options like Get, Query,
Update, Delete, etc depending on the endpoint capabilities but they also fall into the same
two categories of Get and Send actions.

Each operation is tied to an action, so when you change actions, the connector will filter out
the operations and remove the currently selected. If you do not have an operation for the
action selected, you'll need to make a new operation. Operations are not tied to licensing, so
you can have as many as you want. You can generate multiple operations for a single
connection.

Importing

For technology operations, you define what to do since it's more general purpose. However,
for application connectors you usually must import an operation because it is already pre-
defined, and you're only allowed to do a specific set of actions to specific data. If you do not
import an operation, you'll run into errors. When you import an operation, there will usually
be a list of objects (the specific thing you're doing the action to mentioned earlier) to choose
from. This allows you to further define your action, you must define what you're getting from
or sending to specifically. For example, the import defines you are doing a GET from the
Account object or an UPDATE to the Cases object since you must get from something/where
or update to something/where.

Before you import, the operation usually looks bare but after you import more fields and
options should be available. You will usually see a profile associated with the operation now.
It might have a request profile and response profile. These are predefined profiles from the
endpoint since they control what is retrieved or sent. The profiles have elements and behave

144

Internal Use - Confidential

 the same as profiles defined by the user.

Note: when you import an operation, the profile is associated with it. You should use the
profile imported in the operation and edit the profile associated with the import if you need
to. Otherwise, you might see strange behavior.

Filter and parameters

Many operations will allow you to define filters, these are essentially inputs to narrow down
the data your retrieve or where you send to. When you define a filter, you should be able to
use that filter as a parameter in the connector's parameters tab. The parameters tab will
allow you to define the value for the filter. Let's say I only want data after a certain date (the
endpoint must have this concept otherwise you can't do it), the endpoint has a field or
element in their profile that defines a date, let's call it CreatedDate. You select this field to
use for filtering. Once you've selected it, you can now import this CreatedDate field in the
parameters tab of the connector as the input and give it a value. Whenever this connector
shape is executed, it will know to go to the endpoint and request information that is related
to the value you set for the field i.e. after X date for field CreatedDate.

Some actions will have predefined filters you can use like a “Get”, and you will not be able to
add more. If you want to add more, it'll depend on what the endpoint lets you do, but usually
a Query allows for more customization.

Understanding Data Flows

Data goes or flows through a process in a very specific way. Usual rules of thumb is:

• Data must complete a path before it can start a new path. Even if it visually looks like
paths join together, the data must complete a path fully before it can go on another
path
• All data must start at the Start shape, no matter if it is data passthrough, no data, etc
• Data goes to shape and if there are multiple documents (multiple XML files, multiple
rows in a flat file, etc), all documents complete the shape's action before moving on
• All data must complete processing at a shape before moving onto the next shape
• Shape source data is the data before it gets processed by a shape

145

Internal Use - Confidential

 A piece of data that flows down a path is called a document. If the document splits into
multiple pieces of data, each one is called a record. All shapes will affect the data flow in
some way or end/start it. Only the notify and set properties shape will not affect the data
flow when they are included. There are two primary ways to view the data flow, either by
looking at the process and knowing where things go or using the process logs to follow the
process and see where things erred or ended up. We'll look at viewing via the process alone
for now. Viewing via the process logs is very similar, except now you have exact numbers and
everything should add up. More complex processes will be a combination of the below.
Here's an example of some common process paths:

Figure 45: Understanding Data Flows

Data flow on a straight line

Let's look at the first process:

1. There is 1 empty document from the no data start shape, it goes to the connector
2. The connector produces 5 documents and goes to the next shape

146

Internal Use - Confidential

 3. All 5 documents go to the stop shape
4. If there were more shapes all 5 documents would go to shape A, when all 5 are finished
processing at shape A they all move as a group to shape B, once finished processing at
shape B, they move together to shape C, etc

Data flow with branches

Let's look at the second process:

1. There is 1 empty document from the no data start shape, it goes to the connector
2. The connector produces 5 "A" documents and goes to the next shape, branch
3. The branch makes a copy of the 5 "A" documents and sends them down to branch 1
a. The 5 "A" documents go down branch 1 and reach the cache shape, all 5 get
stored in the cache
4. The branch shape makes a copy of the 5 "A" documents and sends it down branch 2
. 5 "A" documents get processed at the message shape, for our purposes, they'll use the
same data to make 5 "B" documents; 5 "B" documents are sent to the next shape.
Message shapes replace the incoming data so it doesn't have to be the same source
data anymore
a. 5 "B" documents are sent to the map shape, they are processed, 5 "C" documents
(since they have been transformed from B to C) are sent to the connector
b. 5 "C" documents are sent through the connector and 5 "D" response documents are
retrieved
c. 5 "D" responses documents go to the stop shape
5. The branch shape makes a copy of the 5 "A" documents and sends it down branch 3
. 5 "A" documents are sent to the map shape, 5 "B" documents are created and sent to
the message shape
a. The 5 "B" documents might be turned into 1 or 5 "C" documents based on the
message shape configuration
b. The 1 or 5 "C" documents are sent to the connector
Branches make copies of the data going into it (shape source data) and send it down each
path. The data goes down each path until completion before it starts on the next path.

Data flow with decisions

Let's look at the third process:

147

Internal Use - Confidential

 1. There is 1 empty document from the no data start shape, it goes to the connector
2. The connector produces 5 "A" documents and goes to the next shape, decision
3. The decision will split up the 5 "A" documents based on a criterion, all documents must
complete here still
4. Let's say 2 documents are determined to be true and 3 are determined to be false
5. The 2 true documents are sent down the true path
a. 2 documents are sent to the message shape, processed
b. 2 documents are sent to the map shape, transformed
c. 2 documents are sent to the connector
d. 2 response documents are sent to the stop shape
6. The 3 false documents are sent down the false path
. 3 documents are sent to the stop shape

Decision shapes group documents based on a criterion and send some amount of documents
together down a true path and then down a false path. It is best to give the decision shape a
label/name to find it easier in the process logs.

Data flow with business rules

Figure 46: Data Flow with Business Rules

The business rule shape is essentially a bunch of decision shapes in one. You can set multiple
conditions instead of the one condition in a decision shape. If a document passes all of the
rules and conditions, it goes into the accepted path, if not the rejected path. Documents are
grouped together in accepted and rejected before being sent to the next shape. The accepted
path executes before the rejected path.

148

Internal Use - Confidential

 1. There is 1 empty document from the no data start shape, it goes to the message shape
2. The message shape replaces the empty document with a document
3. Doc 1 goes to the data process and becomes 5 documents
4. 5 documents go to the business rule, 3 pass and 2 fail the rules
5. 3 documents go to the notify shape, map, and then connector
6. 2 documents go to the notify shape and then mail connector
A business rule shape is a bunch of decision shapes. Instead of having 5 decision shapes, you
can have 1 business rule shape.

Data flow with return documents

Figure 47: Data Flow with Return Documents

In branches, the data must flow through a path until completion before it can move onto the
next path. If you want to retrieve all documents produced by all branches or a certain number
of branches, you can use the return documents shape like in the above example. The return
documents shape waits for all branches to complete before it gathers all source data sent to
it and returns it back to whatever called it, in this case the main process using a subprocess
call. The returned data is now grouped together as a batch and all sent to the next shape.
Then the same data flow principles apply. Combining everything is a common example of why
someone wants to retrieve all documents from a branch. Let's go through the data flow:

149

Internal Use - Confidential

 1. Main process has a no data start shape so it sends 1 empty document to the process call
shape, starting the "Return all documents" subprocess
2. The "Return all documents" subprocess starts with a no data start shape so it sends 1
empty document to the branch shape
3. Branch 1 starts and produces 3 documents from the connector, those 3 documents are
transformed in the map shape, then sent to the return documents shape
4. The return documents shape holds onto those 3 documents
5. Branch 2 starts and produces 4 documents connector, those 4 documents are
transformed in the map shape, then sent to the return documents shape
6. The return documents shape holds onto those 4 documents, bringing the total to 7
documents on hold
7. Branch 3 starts and produces 5 documents, those 5 documents are transformed in the
map shape, then sent to the return documents shape
8. The return documents shape gets those 5 documents and brings the total to 12
documents
9. There are no more branches to the return documents gathers up all 12 and sends it back
to the main process
10. The main process's process call shape outputs 12 documents, those 12 documents are
sent to the next shape, the data process shape to be combined
11. There is 1 document produced from the combine and sent to the stop shape
Note: you can only combine valid data if you want to use it later on, you can't combine an
XML and a JSON together and expect that data to function properly later. Your combine or
any other data processing has to make sense.

150

Internal Use - Confidential

 Data flow using the route shape

Figure 48: Data Flow with Route Shape

The route shape uses a criteria to send incoming data down paths that match the values of
the criteria such as an element value. In this case, there are certain values, New, Current, On
Hold we're looking for. If routing element does not equal any of these, it'll go down the
default path. Everything works the same, documents are completed in batches, etc. Let's
take a look at the data flow:

1. The no data start shape sends 1 empty document to the connector

2. The connector pulls 20 documents to send to the route shape
3. The route shape compares the Account Status value in those documents to New,
Current, and On Hold, let's say the route shape determined 2 are New, 4 are Current,
and 10 are On Hold, and 2 are Closed, and 2 are Unknown
4. 2 documents that are New go down the New path, 2 documents go to the data process,
then 2 to the disk shape
5. 4 documents that are Current go down the Current path, 4 documents go to the map,
they are transformed and 4 documents go to the disk shape
6. 10 documents go down the On Hold path, 10 go to the set properties shape. Different
property types will interact differently with the data
7. 4 documents go down the default path (why is that?), then it goes to a decision shape
(let's say 3 are true and 1 is false, what is the data flow?)

151

Internal Use - Confidential

 Route shapes group documents together based on some criteria and send those groups down
the corresponding paths, one path at a time.

Data flow using a process route

Figure 49: Data Flow with Process Route

A processing route is a combination of the route and process call. The two processes above
show a process route being used and the second process is an example of one of the processes
the process route uses. Important notes:

• The process route shape on the canvas (1st process) must have route parameters
defined

152

Internal Use - Confidential

 • The route parameters are configured the same way as the right side of the set
properties shape (view 2.4) and the end value must equal a route key value in the
process route component (bottom right of screenshot)
• The process a process route calls on uses return documents to send data back,
otherwise nothing is sent back
• If there are no matching route parameter and key, the data goes down the default path
• The default path is executed last, path execution order is determined by Return Path
Execution Sequence in the process route shape not path name
With that in mind, let's go through the data flow:

1. The message shape mimics 5 documents (with a split, it'll make it look like 5 documents
are being sent through) so 5 documents are sent to the process route
2. The process route calculates the route parameter for each document and sends them to
the process that corresponds with the key value
3. Let's say doc 1 has a calculated value of 30, doc 2 30, doc 3 60, doc 4 90, and doc 5 120
4. Doc 1 goes to the process that corresponds with the key 30, let's say it goes down the
true path in the second process in the screenshot. Now that resulting document is
returned on the Success path of the process route because Success is linked with the
return documents shape called "true"
5. Doc 2 goes to the same process Doc 1 goes to (why?) and let's say it goes down the false
path. Now that resulting document is returned on the Broken path of the process route
because Broken is linked with the return documents shape called "false"
6. Doc 3 goes to the process that corresponds with the key 60 and ends up being returned
to the Broken path
7. Doc 4 goes to the process that corresponds with the key 90 and ends up being returned
to the Broken path
8. Doc 5 goes to the process that corresponds with the key 120 does not go to any process
(why?)
9. The process routing shape has finished executing all of the processes and it is getting all
of the returned documents now, it determines 1 goes to Success path, 3 go to Broken
path, and 1 goes down the default path
10. 1 document goes to Success path, to notify, map, and then the connector
11. 3 documents go to the Broken path, to notify then mail connector
12. 1 document (which document?) goes to the default path, to the notify

153

Internal Use - Confidential

 The process route shape is a combination of the route shape and the process call shape. It
collects documents before sending them to different paths just like any other shape.

Data flow using flow control

Figure 50: Data Flow with Flow Control

Flow control controls how documents go down a path. They do not have to go as one group
anymore. Test mode does not show threading (simultaneous/concurrent/parallel processing)
so you won't be able to see it there, but it will happen when the process is deployed and
executed.

4.3.1.4.7.1 Run Each Document Individually

If you select this option (leave number of units at 0), the documents will be sent one at a
time down the next shape in the process path until completion. Once that document
completes, the next document will be sent down to completion, this repeats until there are
no more documents. Let's look at the flow in the above process:

1. 5 documents are retrieved from the disk

2. All 5 docs go to the flow control shape and the setting is run each document individually
3. Doc 1 goes to the data process shape, then to the connector then the stop shape
4. Doc 2 goes to the data process shape, then to the connector then the stop shape
5. Doc 3 goes to the data process shape, then to the connector then the stop shape
6. Doc 4 goes to the data process shape, then to the connector then the stop shape
7. Doc 5 goes to the data process shape, then to the connector then the stop shape, end
of process

154

Internal Use - Confidential

 4.3.1.4.7.2 Run as Batches

If you select this option (leave number of units at 0), the documents will be grouped together
in batches set by the number you have selected.

1. 8 documents are retrieved from the disk

2. All 8 docs go to the flow control shape and the setting is run in batches of 3 documents
3. Doc 1-3 go to the data process shape (there's 3 slots open and there are 8 docs), then 3
docs to the connector then the stop shape
4. Doc 2-6 go to the data process shape (there's 3 slots open and there are 5 docs), then 3
docs to the connector then the stop shape
5. Doc 7-8 goes to the data process shape (there's 3 slots open and there's 2 docs), then 2
docs to the connector then the stop shape, end of process

Number of units with batching

If you select some number of units (unit scope of threads, batches of 3), multiple threads
(paths) will execute at the same time:

1. 8 documents are retrieved from the disk

2. All 8 docs go to the flow control shape and the setting is 2 units, run in batches of 3
documents
3. Doc 1-3 go to the data process shape the same time doc 2-6 go to the data process
shape, each group completes its path
4. Doc 7-8 go to the data process shape whenever one of the groups above finish
processing completely
If you have more units, whenever a unit is done another batch will start until there are no
more. If you have run each document individually, it is the same as choosing to run a batch of
1. Each document will run through a path completely before the next document can. Two
documents will run at the same time in parallel.

1. 8 documents are retrieved from the disk

2. All 8 docs go to the flow control shape and the setting is 2 units, run individually (or
batch of 1)

155

Internal Use - Confidential

 3. Doc 1 goes to the data process shape the same time doc 2 goes to the data process
shape, each doc completes its path
4. Doc 3 goes to the data process shape once a thread is completed, then doc 4 goes to
the data process shape a little later or at the same time depending on when doc 1 and 2
finish
5. Doc 5 goes to the data process shape once a thread is completed, then doc 6 goes to
the data process shape a little later or at the same time depending on when doc 3 and 4
finish
6. Doc 7 goes to the data process shape once a thread is completed, then doc 8 goes to
the data process shape a little later or at the same time depending on when doc 5 and 6
finish
7. Process ends whenever doc 8 goes through its path
If no document batching is enabled, the flow control will split up the documents as evenly as
possible based on the number of units:

1. 8 documents are retrieved from the disk

2. All 8 docs go to the flow control shape and the setting is 2 units, no batching
3. Doc 1-4 go to the data process shape the same time doc 5-8 go to the data process
shape, each group completes its path
Threads are how many simultaneous executions can happen at the same time (on the same
computing environment, JVM) and the flow control breaks up how the documents get sent
down the path. If you have a molecule, you can choose the unit to be processes so that would
spin up a new JVM to execute the execution (that will be covered in a further section).

Data flow using try/catch

Figure 51: Data Flow with Try/Catch

The try/catch shape will suppress errors and if a document-level or process-level error

156

Internal Use - Confidential

 occurs, whatever document caused the error will be sent to the catch path. Error suppression
means the process will show up as completed like normal in process reporting. You will need
good logging to identify which document caused this issue, otherwise it'll be harder to find
which one. There's a base try/catch message you can add to any notification so you know
what error was picked up (when setting as a variable in notify, exception, or message,
Document Property -> Meta Data -> Base -> Try/Catch Message).

The try/catch starts wherever you put it and will not catch things before it. If something
happens before it, the process would fail and be recorded in process reporting. A try/catch
can retry the same document by sending it down the same try path again after failure (if it
still fails, it'll be sent to the catch path):

• 0 = The document will not be retried. Use this setting if you want to catch the error and
deal with it in some other way. For example, archive the inbound document, send an
email alert, etc
• 1 = The first retry occurs immediately following the first document failure
• 2 = If the document fails a second time, the system waits 10 seconds after retrying the
1st time
• 3 = If the document fails a third time, the system waits 30 seconds after retrying the
2nd time
• 4 = If the document fails a fourth time, the system waits 60 seconds after retrying the
3rd time
• 5 = If the document fails a fifth time, the system waits 120 seconds after retrying the
4th time

4.3.1.4.8.1 Scenario 1
If you set the try/catch to catch document errors and a document-level error occurs (or Stop
a single document is checked for an exception shape):

1. 8 documents are retrieved from the disk

2. All 8 docs go to the try/catch
3. 8 docs go to data processing, but 2 of them fail there because of a document related
error (or reaches the exception shape)
4. 6 docs are sent to the connector, then stop shape
5. 2 docs are sent to the catch path, 2 docs go to the exception shape

157

Internal Use - Confidential

 4.3.1.4.8.2 Scenario 2
If you set the try/catch to catch document errors and a process-level error occurs (or Stop a
single document is unchecked for an exception shape):

1. 8 documents are retrieved from the disk

2. All 8 docs go to the try/catch
3. 8 docs go to data processing, but 1 of them fails somewhere along the process because
of a process related error (or reaches the exception shape)
4. The whole process stops

4.3.1.4.8.3 Scenario 3
If you set the try/catch to catch all errors and a document-level error occurs (or Stop a single
document is checked for an exception shape), the same thing happens in scenario 1.

4.3.1.4.8.4 Scenario 4
if you set the try/catch to catch all errors and a process-level error occurs (or Stop a single
document is unchecked for an exception shape):

1. 8 documents are retrieved from the disk

2. All 8 docs go to the try/catch
3. 8 docs go to data processing, but 1 of them fails somewhere along the process because
of a process related error (or reaches the exception shape)
4. All 8 docs are sent to the catch path
You need good logging to identify which document caused the issue. If it is a process-level
error, then you have to fix that and it wouldn't be caused by a the document specifically.
Some additional notes:

• Avoid using too many try/catch shapes, use it where you need it e.g. right before
sending to a connector if you know there's issues at the connector. This is useful for
retrying something if the data just needs to be resent again
• If you are going to have a try/catch before another try/catch, be mindful of the retries
since the number of retries will multiply
• Rename the try/catch shapes so it is easier to identify which one(s) was triggered

158

Internal Use - Confidential

 Property Types
The platform has various property types to help with metadata (separate from the actual
data). The set properties shape is used mainly in defining the values of these properties.
Their values are used in many other places of the process.

Document property
Document properties (also called standard document properties) contain connector specific
information like file name, size, and whatever connector specific information is
available. Document properties can be retrieved as a value in parameters in most places, the
Get Document Property map function, and custom scripting. Document properties can be set
in the set properties shape, the Set Document Property map function, and custom scripting.

If you try to view the document property value after setting it, you might not get a result
since the get of the document property is tied to the connector used. For example, if you are
using a Box connector and get some documents and pass it through the process. Each
document will have document property related to the Box connector. If you view the Box
connector related document properties, you'll see the corresponding values. Let's say you
then want to send the data to a disk shape, you can set the document properties of the Disk
connector such as file name and directory and the disk connector will use that. If you try to
view the document property right after you set the file name, you will not get anything
because there was nothing retrieved from the disk connector (you got something from Box,
remember?). Some general rules:

• Document properties are tied to the document, so it will flow with a document into a
process call, data passthrough and return documents shape
• Document properties cannot exist between paths (branches)
• When you retrieve a document property in a map, the document property will be that
of document that entered the map shape

Dynamic document property

Dynamic document properties (DDP) are temporary properties tied to a document (as opposed
to being tied to the connector). It can have any name and it follows the document through

159

Internal Use - Confidential

 the process. DDPs are set and retrieved per document when a batch of documents go through
a shape such as a set properties or map. Some general rules:

• DDPs are tied to the document so it will flow with a document into a process call, data
passthrough and return documents shape
• DDPs cannot exist between paths (branches)

Dynamic process property

Dynamic process properties (DPP) are process properties that live as long as the process lives.
They can be named anything and can be used as long as you use the same name. Some
general rules:

• DPPs are a process property and are not tied to a document, the value can be retrieved
from a document or an element in the document
• DPPs can exist between branches and processes so subprocesses and master processes
can use and set DPPs (just remember the order in which you do that for it to make
sense). You cannot pull a value of a DPP you defined in the master process if you set it
in the subprocess, the value would be nothing
• DPPs can be persisted and extended (we'll go over persistence and extension later)
DPPs take on the value of the last document that reached it. If you have a batch of 5
documents go to a set properties where you set the DPP to be an element of the document,
the DPP will be the last document's value. Let's go through the flow:

1. There's a set properties shape with a DPP called element1, where the value is an
element in the document
2. 5 documents reach the set properties shape
3. Doc 1 has value1 so the DPP element1 = value 1, there are 4 more docs
4. Doc 2 has value2 so the DPP element1 = value 2, there are 3 more docs
5. Doc 3 has value3 so the DPP element1 = value 3, there are 2 more docs
6. Doc 4 has value4 so the DPP element1 = value 4, there are 1 more docs
7. Doc 5 has value5 so the DPP element1 = value 5, there are no more docs
8. If you were to pull the DPP element1 to view its value, it will be value 5 now because it
is a process property, not document property

160

Internal Use - Confidential

 Process Property
A process property is a collection of key/value pairs stored in a table like structure. Usually,
if you have global variables or want to restrict certain process properties you can use a
process property [table]. A process property can be restricted to certain allowed values,
otherwise it can be basically anything. If you give them a value, you can retrieve them
anywhere later. If you set the value, you can store it for that execution until the execution
finishes and it'll go back to the default value.

They can also be persisted and extended.

Persistence
Process properties and dynamic process properties can be persisted. The property name can
survive into subsequent executions and the last value set will be remembered. After you
execute a process the first time, the persisted value will be stored for later use (to access
Atom Management -> Click on the atom where you deployed it -> Deployed Processes -> find
the process and click "Edit Process Properties" -> all persisted values should be there).

• Persistence should be used if needed since persisted process properties are saved to
disk each time they are stored
• The value is of the last time it was set for cases of simultaneous executions (not
recommended to be used together) and flow control
• Persisted values are per process per runtime
Extensions and persistence behavior:

• If the property is persisted, you extended the same property, and there is no stored
value yet, the value in the extensions will be used
• If the property is persisted, you extended the same property, and there is stored value,
the value in the stored persisted property will be used

161

Internal Use - Confidential

 Setting Properties
Set Properties Shape
The set properties shape is the most common way to set property values. The left half of the
set properties shape is which properties to set (view 2.3). The right half (parameters) is
setting the actual value of the property. The properties are limited by their natures as
described in 2.3 but the values are not limited by anything except function and creativity. All
values can be set the same way for any property.

For testing purposes, the easiest way of setting a value is via the Static parameter value type.
You can type in any value or use a value from the client for testing. If you have multiple
values for the parameters, they will be concatenated together:

Figure 52: Set Properties

In the case of the above, the document property File Name will now be
"test4112019element1". If you wanted to add in a ".txt" to the file name, what do you do? If
you want to add spaces to the file name, what things can you do?

Set Properties as a Map Function

There is a map function for each of the properties in the map. You can set and get values
there. You can also get and set those properties as part of the user defined function for more
steps and complexity.

162

Internal Use - Confidential

 Custom Scripting
Custom scripting can set and get properties as well. See com.boomi.execution.ExecutionUtil
class more information on the class.

Figure 53: Using Groovy to Get and Set Dynamic Process Properties

DPP is a process property so you do not have to set it within the document loop. The DPP
value is a string in the script. If you are trying to store a number, you can convert the number
variable to a string via the .toString() groovy function (or equivalent).

Figure 54: Groovy Script

Figure 55: Groovy Script

163

Internal Use - Confidential

 DDPs are prefixed with "document.dynamic.userdefined." before the property's name. It's a
document property so that's why it is within the document loop.

Figure 56: Groovy Script

Using groovy to get and set the document properties

For document properties, there are certain ones you can retrieve and only certain ones you
can set.

Figure 57: Groovy Script

The document properties through custom scripting are limited. What's available is in the table
below. Pick out the connector type and that will go after "connector.track." or
"connector.dynamic.". After that, pick the corresponding connector property. For example, if
you want to use set the mail connector properties, specifically the to address when sending,
then you would pick the mail connector type -> "connector.dynamic.mail", then add
".toaddress" -> "connector.dynamic.mail.toaddress". If you want get the mail from address,
pick the mail connector type -> "connector.track.mail" -> use fromaddress ->

164

Internal Use - Confidential

 "connector.track.mail.fromaddress". Other examples:

Get disk filename:

connector.track.<Connector type>.<Connector property>
connector.track.disk.filename to get the filename of a doc coming the disk connector

Set ftp move to directory:

connector.dynamic.<Connector type>.<Connector property>
connector.dynamic.ftp.movetodirectory to set the move to directory of the FTP connector

Table 14: Connectors, Get/Set Properties

Connector type Get doc connector property Set doc connector properties

as2 file
name
subject

as2sharedserver as2fromid
as2toid
filename
messageid
subject
mimetype

disk filename directory

filename

ftp filename filename

host movetodirectory
movetodirectory remotedirectory
port
remotedirectory

jms correlationid correlationid

destination destination
messageid priority
priority replyto
redelivered
replyto

mail filename filename

fromaddress fromaddress
host subject
messageId toaddress

165

Internal Use - Confidential

 Connector type Get doc connector property Set doc connector properties

subject
toaddress
user

sap idoctype

sftp filename filename

host movetodirectory
movetodirectory remotedirectory
port
remotedirectory

Message Shape and Testing

Message Shape
The message shape allows you to add a document into the process and use that instead of
what you have. The message shape will replace documents if it is placed where documents
are going to. For example, if you use the FTP connector to get a file and put a message shape
after the FTP connector, the file from the FTP connector will be replaced by whatever
content is in the message shape.

The message shape uses a variable system (Notify and Exception use the same variable system
as well) to grab data from elsewhere to construct what's in the message. When you define a
variable, it gets assigned a number in curly brackets, {1}, {2}, {3}, etc. You can then use those
curly brackets and number to construct the message body however you want to. For example:

Figure 58: Sample Message Body

166

Internal Use - Confidential

 The resulting document will now be passed down the process path. Besides constructing data,
it is often used for testing.

You can put any text-based data into the message to send it in a process. Because of this, it is
often used for testing data in processes. Often it is easier to test with data that is already
present than have to go through the connectors to grab data. This is practical for a variety of
reasons:

• Data is deleted or modified after it is picked up

• Testing specific scenarios require the source connector to have that data which might
take a bit more work and time
• Quicker to test when you can modify data on the fly
• Connecting to different sets of data requires changing connector information
• For data coming from local on-premise sources like Disk and Databases, you do not need
the specific local atom anymore once you have a copy of the data in the message shape

To test in Integration, the message shape can be used and you can place the content directly
into the message field and link it to the next shape like a map or connector. Having multiple
message shapes labelled clearly and already linked to the rest of the process makes swapping
data easier. This way, you only need to link the No Data start shape to the message shape you
want to test.

Scenarios on How to Test with the Message Shape

4.3.1.7.2.1 Scenario 1: Testing with XML

For XML, you can copy directly into the message shape. You do not need the tabs or extra
spaces, the message shape will still accept it. You still have to follow the usual rules for XML
like escaping special characters appropriately (see table below). For example:

167

Internal Use - Confidential

 Figure 59: XML Testing

Table 15: Special Characters

Special character escaped form gets replaced by

Ampersand &amp; &

Less-than &lt; <

Greater-than &gt; >

Quotes &quot; "

Apostrophe &apos; '

Note: You have to use double quotes (") in the top line, <?xml version="1.0" encoding="UTF-8"?>, the
message shape will not take single quotes in XML

4.3.1.7.2.2 Scenario 2: Testing with JSON

Figure 60: Sample JSON

168

Internal Use - Confidential

 4.3.1.7.2.3 Scenario 3: Testing with Database

For a database (DB) profile, you need to match the component ID of the Database Profile to
the component ID present in the data (in bold), after the "DBSTART". If you copy a process for
testing or development, your profiles will be copied and they will have a new component ID.
Just replace the component ID in the DB data and the map should be able to pick it up now.
Let's say in process A my DB profile had a component ID of gy96db7f-5a6d-4agd-8509-
j5ah7b3f7058, now that I've copied it to process B, the DB profile gets a component ID of
ae83db7f-0a6d-4aed-8509-7FaF7b3f7058, to use the same data from the process A in B
without having to go through the connector,

Process A:

DBSTART|gy96db7f-5a6d-4agd-8509-j5ah7b3f7058|2|@|BEGIN|2|@|OUT_START|3|@|2078-07-77
75:49:00.0|^|20780777|^|A76593P7778BWB026|^|A76593P7778BWB026|^|04|^|ZZ|^|Z|^|WH|^|Fong
ebeF -
Falgary|^|97|^|2580|^|IL|^|7500070490|^|07|^|20780777|^|RG|^|2|^|FA|^|006736243295|^|BN|^
|6736243295|^|7778BWB026|^||^|ReFeption|^|76593|^|20780777|^||^|705057-
7|^|705057|^|FL_RG|^|2|^||^|A7|#|2078-07-77
75:49:00.0|^|20780777|^|A76593P7778BWB026|^|A76593P7778BWB026|^|04|^|ZZ|^|Z|^|WH|^|Fong
ebeF -
Falgary|^|97|^|2580|^|IL|^|7500070490|^|07|^|20780777|^|RG|^|2|^|FA|^|006736243295|^|BN|^
|6736243295|^|7778BWB026|^||^|ReFeption|^|76593|^|20780777|^||^|705057-
2|^|705057|^|FL_RG|^|2|^||^|A7|#|2078-07-77
75:49:00.0|^|20780777|^|A76593P7778BWB026|^|A76593P7778BWB026|^|04|^|ZZ|^|Z|^|WH|^|Fong
ebeF -
Falgary|^|97|^|2580|^|IL|^|7500070490|^|07|^|20780777|^|RG|^|2|^|FA|^|006736243295|^|BN|^
|6736243295|^|7778BWB026|^||^|ReFeption|^|76593|^|20780777|^||^|705057-
3|^|705057|^|FL_RG|^|2|^||^|A7|#||#|OUT_END|3|@|END|2|@|DBEND|gy96db7f-5a6d-4agd-
8509-j5ah7b3f7058|2|@|

Process B:

DBSTART|ae83db7f-0a6d-4aed-8509-7FaF7b3f7058|2|@|BEGIN|2|@|OUT_START|3|@|2078-07-77
75:49:00.0|^|20780777|^|A76593P7778BWB026|^|A76593P7778BWB026|^|04|^|ZZ|^|Z|^|WH|^|Fong
ebeF -
Falgary|^|97|^|2580|^|IL|^|7500070490|^|07|^|20780777|^|RG|^|2|^|FA|^|006736243295|^|BN|^
|6736243295|^|7778BWB026|^||^|ReFeption|^|76593|^|20780777|^||^|705057-
7|^|705057|^|FL_RG|^|2|^||^|A7|#|2078-07-77
75:49:00.0|^|20780777|^|A76593P7778BWB026|^|A76593P7778BWB026|^|04|^|ZZ|^|Z|^|WH|^|Fong
ebeF -
Falgary|^|97|^|2580|^|IL|^|7500070490|^|07|^|20780777|^|RG|^|2|^|FA|^|006736243295|^|BN|^
|6736243295|^|7778BWB026|^||^|ReFeption|^|76593|^|20780777|^||^|705057-
2|^|705057|^|FL_RG|^|2|^||^|A7|#|2078-07-77
75:49:00.0|^|20780777|^|A76593P7778BWB026|^|A76593P7778BWB026|^|04|^|ZZ|^|Z|^|WH|^|Fong
ebeF -
Falgary|^|97|^|2580|^|IL|^|7500070490|^|07|^|20780777|^|RG|^|2|^|FA|^|006736243295|^|BN|^
|6736243295|^|7778BWB026|^||^|ReFeption|^|76593|^|20780777|^||^|705057-

169

Internal Use - Confidential

 3|^|705057|^|FL_RG|^|2|^||^|A7|#||#|OUT_END|3|@|END|2|@|DBEND|ae83db7f-0a6d-4aed-
8509-7FaF7b3f7058|2|@|

If you forget to change the component ID in the message shape and it's going into a map, the
data will not be recognized. You will get a "No data produced from map, please check source
profile and make sure it matches the source data" error.

Note: |^| denotes column separation and |#| denote row separation

4.3.1.7.2.4 Scenario 4: Testing with EDI/Flat File

You can copy an EDI and Flat File profile as is directly into the message shape without any
modifications.

Note: If you are using EDIFACT which uses single quotes, you'll need to escape that single
quote with another single quote e.g.

Figure 61: Sample EDI/Flat File

Common Errors
Error: Unable to create XML files from data, the document may not be well-formed xml ;
Caused by: Unexpected character '1'

You most likely have <?xml version='1.0' encoding='UTF-8'?> or a single quote somewhere. You
need to use double quotes instead. Full error: "Unable to create XML files from data, the
document may not be well-formed xml ; Caused by: Unexpected character '1' (code 49);
expected a quote character enclosing value for 'version' at [row,col {unknown-source}]:
[##,##]"

Error: can't parse argument number (when testing with JSON)

Your JSON profile in the message is missing the single quotes wrap around the whole profile.

FAQ

4.3.1.7.4.1 How do I simulate multiple records?

For flat files, a new line denotes a new record (or set of information) so it is easy to simulate
by adding in multiple lines for the flat file. Same goes for the database, each row

170

Internal Use - Confidential

 automatically gets treated as separate record by default when going through something like
the map shape.

However, for EDI, XML and JSON, you cannot just add another profile one after the other and
expect the map shape to separate them automatically. At this point you cannot use only the
message shape for that purpose. You can use the return documents in a subprocess to
simulate getting multiple documents. You can also simulate multiple records by using the disk
connector to pick up all of your EDI, XML, and JSON files at some location.

4.3.1.7.4.2 Can I troubleshoot production data errors this way?

If you have production errors and want to know if the data is causing the issue or at what
point the data is causing the issue, you can pull that data from process reporting and then put
it in a message shape also. Make a copy of your process, put in the message shape with the
production data (using above mentioned scenarios), and remove any CREATE/SEND endpoints
(to avoid changing anything). Run the data through the process in test mode and you will have
more visibility with how the data is being transformed. Use stop shapes so you can see what
data is coming out of a shape.

4.3.1.7.4.3 Why is the source data from the message shape missing single quotes?
The message shape treats single quotes as special characters. To use single quotes as values
in your message shape, they must be escaped via another single quote.

Table 16: Message Shape and Single Quotes

In Message shape Output when running process

testing to 'see if the single quotes appear' testing to see if the single quotes
appear

testing to ''see if the single quotes appear'' (pair of testing to 'see if the single quotes
single quotes) appear'

4.3.1.7.4.4 Why are my variables in the message shape not populating?

If the data coming out of the message shape is still showing the curly brackets and number
(i.e. {1}, {2}, etc) that usually means you have a single quote somewhere. You need to escape
it. Where you escape it matters. Everything after a single quote is escaped, until there's
another single quote.

171

Internal Use - Confidential

 In Message shape, {1} is set to static value of "nice"

Table 17: Message Shape and Variables

In Message shape Output when running process

Today's weather is {1} Todays weather is {1}

Today's weather is '{1} Todays weather is nice

Today''s weather is {1} (pair of single quotes) Today's weather is nice

4.3.1.7.4.5 Why is the message shape showing unable to parse error?

Make sure everything looks right and you are following the standard rules for each format.
You can use a syntax checker online. Sometimes the message doesn't parse data correctly
even if it is in the correct profile format because of its own internal parsing rules. Easiest way
to get around this (after you are sure the syntax is correct) is to use the variables. Use {1} and
set {1} to have a Type of static and set the Static Value as the profile, you can just
copy/paste the whole profile into the static field.

4.3.1.7.4.6 How to handle apostrophes (') in the message shape?

You can escape the apostrophe according to the above methods or use a disk connector to
retrieve the file instead.

End of Data Flow

Data flow has to end at some point. Usually it can end gracefully through various shapes. If
there's a problem with the process, usually that will show up as an error in process reporting.
The process just stops without error handling and tries its best to report the error.

Stop shape
The stop shape has two options:

• Continue processing other execution paths checked, documents can continue through
the process like normal even if they reach a stop shape in an earlier branch
• Continue processing other execution paths unchecked, the whole process stops when
one document reaches the stop shape with this option

172

Internal Use - Confidential

 Exception shape
The exception shape has two options:

• Stop a single document checked, the document stops here and other documents can
continue through the process
• Stop a single document unchecked, the whole process stops when one document
reaches the stop shape with this option

Return documents
The return documents will wait for all documents being sent to it before it returns it to the
caller or whichever process called on a subprocess that uses the return documents shape.

Document cache shapes

The add to cache and remove from cache shapes will end data flow.

Connector behavior
Some connectors do end data flow because there are no response documents to return so the
data flow ends (Disk send, FTP send). This is based on legacy information and how a
connector was coded (BOOMI-26717). Although there's an arrow for the path after these
shapes, nothing actually happens. No document properties propagate either.

Some connectors end document properties too with the GET/QUERY actions, they do not
propagate properties. For example, for a Database get or Salesforce query, document
properties set prior will not propagate further after the connector. HTTP connectors using
parameters will also end document properties.

List of connectors that end data flow:

• Disk Send
• FTP Send
• LDAP Update

4.3.2 Data Manipulation

Maps
Maps are used to transform data, either in data profile and/or of the data values themselves.
The left-hand side profile, the source profile, affects what data is available to use. The right-
hand side, the destination profile, affect what the final data will look like from what
available data there is. Some general rules of maps:

• A destination element can only have one mapping

• A source element can be mapped to multiple destination elements

173

Internal Use - Confidential

 • The filter option "Show mapped fields only" will show mapped fields only
• Clicking on the triangle next to elements that are mapped will show an option to go
directly to destination element
• You can map anything to anything, including a profile to itself
• Editing a profile while there are mappings should not remove the mappings (unless you
removed the element that has a mapping)
• Choosing a new profile, even the same one, will erase the mappings
You don't actually have to map anything from the source data as long as the destination
output produces something, otherwise you'll get this error, "No data produced from map". The
can be data without source data using functions and/or default values.

Source Profile
The source profile affects what data is available to use. That does not necessarily mean it
affects the original data itself. For example, let's take this XML:

Figure 62: XML Data

The Name element will be mapped to some flat file profile element. Notice how there's
multiple Name elements. Without changing the destination profile, in the source profile, if
Max Occurs for the Name element is unbounded, there will be two lines created for the flat
file data. If the max occurs is 1, there will only one line created for the flat file data. The
original data and destination profile hasn't changed, only the source profile options, how the
data is read. How the profile is defined will determine what data is available for use. What's
available for use will then be used by the destination profile. If your data doesn't look right
after the map, make sure to check the destination profile (format you wanted) and the source
profile (available data) afterwards if there are no discrepancies in the destination profile.

Adding cached data

The source profile can pull a document from the document cache also. On one of the high
level nodes (different node for each profile) you can click on the triangle to use the option

174

Internal Use - Confidential

 "Add Cached Data", this will allow you to retrieve all of the document's elements using some
key value that comes from the incoming data. You can use the elements just like a regular
profile element.

175

Internal Use - Confidential

 Figure 63: Adding Cache Data

176

Internal Use - Confidential

 Destination profile
The destination profile affects the final look/output for the map. The data format/options is
always as you defined. This affects how many and what's in the output data. This will be
heavily dependent on destination profile options, so you have to be familiar with what's
available or do trial and error.

Destination profiles elements can have default values. Default values set the value of the
element if has no mappings or if the mapping produces no value.

If the output is producing more data than anticipated, there is most likely a max occurs of 1
where you didn't expect it to. Since there's a max occurs of 1, a new document must be made
for each instance of that element, if there are multiple.

If data is missing from the output, most likely data was not read correctly so it was not there
to be used in the first place; incorrect mapping of element(s); the looping option might be
unique, so repeats are not allowed.

The best way to determine if it is the destination or source profile, see if you can extract the
value in question. If you cannot extract the value (via DPP or simple map to a flat file
element), there is something wrong with the source profile or data since you can't get it at
all. If you can extract the value, then there is something wrong with the destination since you
were able to pull the value for another profile.

Boomi Suggest
Boomi Suggest will try to do mappings automatically for you. It uses data from all other
deployed maps to make its best guess about which map should go where. If most maps map a
Salesforce createDate element to a NetSuite createdbyDate element, it'll suggest that to you.
Of course, you can modify as desired still. For support purposes, it is often better to use the
values in question and leave unrelated/unaffected mappings out. Do not map more than you
have to in order to prove a concept or result.

177

Internal Use - Confidential

 Flat File Profile Options
Field Length Validation will see if the value of the element is below the Min Length or above
the Max Length. If it is true in either case, the map will throw an error,
"MAX_LENGTH_EXCEEDED" or "MIN_LENGTH_NOT_REACHED" for source or destination data.

Input Data
Things can get as complicated as you want them to with data and profile options, but a
couple of general rules of thumb for flat files:

• Element order matters, not element header names

• Text qualifiers and delimiters should be avoided in the data values
• If text and delimiters are necessary in the data, you need to escape them
• Escape characters only matter if the character they are escaping is after them,
otherwise they are treated as normal characters
• Escape characters must be before the delimiter or text qualifier they are escaping
• The escape character must be in the data itself and defined for escaping to work
We'll be describing input data first (left side of the map). Delimited flat files (data that uses a
character to separate data values) are much more common than data positioned (which use
position in the line to determine values). The most common file delimiters are the comma
and star characters. Let's use the simplest flat file profile option (File Type: Delimited and
File Delimiter: Star Delimited) and start there.

This has 2 lines, 5 elements per line, and is delimited by *. If we call the 2nd element address
and pull only that (how? via notify or something else?), we'll extract "123 Parkside Drive" and
"The Towers Apartments 403B".

Figure 64: Flat Files and Input Data

However, if you switch the element name order in the profile where city is 2nd and address is
3rd. We map the 3rd element "address", the map will pull "Chicago" and "Denver" now. The
flat profile does not match up names, it matches up order of elements. It doesn't matter what
you call your header elements, they have to be in the right order to work properly.

178

Internal Use - Confidential

 Now let's add a header. If you keep the same profile options, the first line will be treated the
same as the other lines and it'll just be data. Before 2 lines were mapped, now 3 lines are
mapped. If you turn on "Use Column Headers" the first line will be ignored (no matter what it
is even if you added extra characters and elements). Your header name in the data does not
have to match the header name defined in the profile, the order is what's important. Let's
pull the 2nd element again:

Figure 65: Flat Files and Input Data

Text qualifiers wrap around the value and are used to identify values as text/character based
e.g. "Sam Hill"*"123 Parkside Drive"*"Chicago". If you define text qualifiers in your data, they
will be removed e.g. if you use double quotes (") as your text qualifier then your value will
have those double quotes removed when that value is extracted. If your double quotes are
misplaced, then your data extraction will start looking strange:

Figure 66: Flat Files and Input Data 2

Because we missed the double quote after 403B, it started to include information from the
next elements until it encounters other double quotes. Because * is a delimiter, the profile is
now confused with the text qualifier so ends up like in the example, not what you wanted.

179

Internal Use - Confidential

 The same thing applies to single quotes. Text qualifiers can either be mistakes like a missing
(or extra) qualifier or it might be part of the data and is legitimately used (like " for inches in
the case of double quotes), but either way the same behavior will happen.

The text qualifier escape field is used to indicate the text qualifier character is actually part
of the data (in the case of inches), rather than being a text qualifier. This should be used in
conjunction with the "Remove Escape" option to keep or remove your escape character. The
character can be the same as the text qualifier (single or double quote). If a text qualifier
escape character is not specified, the "Delimiter Escape" character is used for both delimiters
and text qualifiers. Let's use the backslash (\) as the text qualifier escape, we have to put this
before the double quotes in the data:

Figure 67: Flat Files and Input Data 3

The delimiter escape character works the same way. Let's use carat (^) instead and turn off
the text qualifier option for now:

180

Internal Use - Confidential

 Figure 68: Flat Files and Input Data 4

181

Internal Use - Confidential

 Escape for escape characters work as excepted (when text qualifiers are not involved).
Delimiter escape is still ^ and Escape for escape is \:

Figure 69: Flat Files and Input Data 5

Let's put back the text qualifiers and leave the escape characters and delimiters inside. If you
have the text qualifier set to double quotes and use the delimiter character in the data, the
data pulled doesn't have a problem since everything inside the double quotes (text qualifier)

182

Internal Use - Confidential

 is ignored until another double quotes is found. There's no need to deal with delimiter
escapes, only text qualifier escapes:

Figure 70: Flat Files and Input Data 6

Output Data
When your destination profile is a flat file (right side of the map) aka you are generating a
flat file, the options will work slightly differently. Let's start off with File Type: Delimited and
File Delimiter: Star Delimited again but this time the flat file profile has 4 elements and I am
mapping just one element (description) to a destination element:

Figure 71: Flat Files and Output Data 1

Notice how the output has extra * in it, that's because I have 4 elements. In flat files, empty
elements have to be accounted for as well, since there are 4 elements, there should be 3 *
separating the 4 values. We'll add in the Use Column Headers option:

Figure 72: Flat Files and Output Data 2

Same input as before but my headers are what I defined in the profile now and I happened to
name them 1, 2, 3, and 4. For output data, your headers you defined are what will show up as
headers. Again, the order is still important. The first element doesn't actually get validated
for anything so it matches with the name of the first element.

183

Internal Use - Confidential

 If we add text qualifiers, the single or double quotes will wrap around the element now:

Figure 73: Flat Files and Output Data 3

Note: Remember text qualifiers wrap around values and are used to identify values as
text/character based? If your element Data type is a number, it will not get the text qualifier,
character and date/time will.

If the input data has delimiter characters in the value, you can set a delimiter escape so it
adds the escape before the delimiter character:

Figure 74: Flat Files and Output Data 4

Note: The Remove Escape option only works for reading (input) profiles. It does not affect
writing flat files.

Text qualifier escape works the same exact way. If there is a text qualifier in the value of the
element, the escape is added before the single or double quotes:

Figure 75: Flat Files and Output Data 5

184

Internal Use - Confidential

 The escape for escape character will show up before all delimiter escape characters if they
are by themselves in an element:

Figure 76: Flat Files and Output Data 6

4.3.2.2.2.1 Enforce Unique

The enforce unique option works when the flat file is the destination. It will remove rows that
have the same value as the element with enforce unique checked:

Figure 77: Enforce Unique/Flat File

It will only keep the first one that is processed. This can be used as an intermediate map to
check the uniqueness of some data, then transform it back into the original data profile to be
used again.

Data Positioned Flat Files

Data positioned flat files are very hard to read and will require a text editor like notepad++ to
help figure out the position in the line. There could be hundreds of thousands of characters in
the one line. You have to define a start column and length now to keep track of where the
data is. However, it works the same way. The data in some position for some length is one
data value and you extract or use it like a normal value between a delimiter in a delimited
flat file.

185

Internal Use - Confidential

 XML Profile Options
Field Length Validation will see if the value of the element is below the Min Length or above
the Max Length. If it is true in either case, the map will throw an error,
"MAX_LENGTH_EXCEEDED" or "MIN_LENGTH_NOT_REACHED" for source or destination data.

Input Data
XML profiles can be imported from XML data (the file) or with a schema. The element details
are a little different from the values used in the schema and they do not really import over
well. After you import, you will still need look like your profile options to make sure it is the
one you want. For XML profiles generated from a connector operation's import, they usually
come with a complex type definition so elements are predefined with how the endpoint
wanted it.

Figure 78: XML Profile Sample Data

Let's start with Min/Max Occurs, which can be 0 or 1 and 1 or unbounded, respectively. Min
Occurs has no real effect on reading the data. Min Occurs = 0 essentially makes the element
optional. Here are scenarios:

1. There is data and Min Occurs = 1, the output will have the data
2. There is no data and Min Occurs = 1, the output will not have the data; we do not throw
an error
3. There is data and Min Occurs = 0, the output will have the data

186

Internal Use - Confidential

 4. There no data is and Min Occurs = 0, the output will not have the data

Max Occurs will affect how the data is read. Max Occurs = 1 means only the first instance of
that data element is read e.g. if the Person element had a Max Occurs = 1, only the Person
element with Billy would be read and used, the other two Person elements will be ignored
(even though they are present). Max Occurs = unbounded means all instances of the element
will be read and used i.e. all three Person elements will be read and mapped.

Looping Option (which can be Unique or Occurrence) does not affect reading the data. All
instances of the element are read.

All in all, only Max Occurs has an effect on the source data.

Output Data

Figure 79: Flat File Data and Simple Mapping

We'll use the DocumentID element, if Min Occurs = 0, the element is optional so it does not
matter if there's data or not. If Min Occurs = 1, another option (under Options) is taken into
effect, Respect Min Occurs:

187

Internal Use - Confidential

 • Min Occurs = 1 and Respect Min Occurs = unchecked, no change if DocumentID has no
value
• Min Occurs = 1 and Respect Min Occurs = checked, if DocumentID has no value, the
element will be written now, <DocumentID />
Max Occurs matters if it is on the parent element (Animal) or the children element (Species or
Name). In general Max Occurs lets an element repeat. If it is on the parent element, then the
parent can repeat, if it is on a child element, the child element can repeat within the parent.

Figure 80: Output Data 1

188

Internal Use - Confidential

 Figure 81: Output Data 2

Figure 82: Output Data 3

189

Internal Use - Confidential

 Figure 83: Output Data 4

Figure 84: Output Data 5

190

Internal Use - Confidential

 Since the children elements are allowed to repeat, they all under one Animal parent element,
whichever child element has repeats will not show up. There would have been two Species
elements with the value "Rabbit" and two Name elements with the value "Nugget" otherwise.

Animal parent element has Max Occurs = unbounded and Looping Option = Occurrence,
Species child element has Max Occurs = 1 and Looping = Unique, Name child element has Max
Occurs = unbounded and Looping = Unique produces:

Figure 85: Output Data 6

191

Internal Use - Confidential

 Figure 86: Output Data 7

192

Internal Use - Confidential

 Figure 87: Output Data 8

193

Internal Use - Confidential

 Figure 88: Output Data 9

Note: the root node is not affected by Max Occurs and Looping Options because there can
only be one root node no matter what, so changing those options will not matter for the root
element.

Some reminders:

• Min Occurs affects and Respect Min Occurs work together to say whether an element
will show up or not if it's empty
• Max Occurs affects whether an element can repeat or not. It affects the parent element
whether other sibling elements of the child have a different Max Occurs or not
• Looping will remove children elements and parent elements that repeat
If you troubleshoot profile options of how data looks, make sure you know what the end data
should look like. When you know that, you can adjust your profile options to most likely make
it. Check what source data or how many of that source data is going into the destination
element.

194

Internal Use - Confidential

 Types
You'll start to see types more with imported profiles (because they use schemas) or if
someone defines it in the types tab in the XML profile. Types define parts of the XML and can
be reused so that way someone doesn't have to define the same thing repeatedly. When using
types, you cannot modify the element options directly anymore. You must modify the type's
options in the type tab and then those changes will reflect in the Data Elements tab. In fact,
you'll see the elements are grayed out.

When elements are defined by types, you'll see they have a Type Name:

Figure 89: Elements Defined by Types

Notice the standard element is greyed out and it has a Type Name of "License". If you go to
the Types tab, you'll see how License is defined and it only has the attributes purchased and
used. The element smallBusiness and enterprise also look the same as licensing because they
also have the same type. Types let you reuse element definitions.

195

Internal Use - Confidential

 Identifier Instances and Qualifiers
Identifier instances allow you to pull specific subsets of data based on occurrence of a
specified data value (a qualifier). You can think of identifier instances as a way to isolate
data or reallocate data in a profile.

Flat file and database profiles do not use instance identifiers and qualifiers but JSON, XML,
and EDI profiles do.

Generally, you define a parent element or loop via a qualifier value in a child element or
child segment. It is much more important for reading data. If you write to a profile that has
identifiers, some elements have predefined data since it's an identifier instance so some
element must have a certain value already. Some general rules of identifier instances:

• Qualifiers are case sensitive

• Instances are exclusive, if you already defined it, you will not be able to get results if
you define it twice, more information below
• Order of defining instances matter, occurrence or qualifier first
• You can define an instance using multiple qualifiers or qualifier and occurrence
• To modify options in an instance, you have to modify the original element
Currently, nested instances are not supported yet (or it doesn't work as expected and will not
produce good data). A parent element that has an identifier instance with a child element
that has an identifier instance will not produce data in child element or isn't allowed (for XML
profiles).

JSON
JSON profiles only allow an identifier instance on repeating arrays. Identifier instances cannot
be nested. That is, if a parent array has an identifier instance, then its children arrays cannot
have one. Conversely, if a child element has an identifier instance, then the parent cannot
have one. Removing an identifier instance from either parent or child element makes the
data available for use. You can define it the instance however you want, but it won't produce
any data.

196

Internal Use - Confidential

 Figure 90: Sample Data

We can see addresses is a repeating array of objects, description and state have different values. If you
were to map the description and state to some flat file elements, the map would pull all occurrences
of the array so:

197

Internal Use - Confidential

 Figure 91: Output Data

If you only want to pull the information from the object with description of home, you would
need to use identifier instances and qualifiers. You would first need to add qualifiers to
identify which instance you want to pull.

Figure 92: Add Qualifiers and Instances

198

Internal Use - Confidential

 After an instance is defined, it'll look like a copy of the original element but faded and have
the qualifier in the name, in this case the array element. If you want to modify the options in
the identifier instance, you have to modify the option in the original element. In the map's
profile, you'll see the identifier instance show up and you'll be able to map the elements from
there now.

When you use identifier instances in the source profile, the instance(s) are now part of the
identifier profile. If you were to map the original array element, only the ones that didn't get
sorted into the instance would get used. If you were to map the description and state for the
original array element, you would get:

Figure 93: Data

since the description = home is now part of that identifier instance, so it's not recognized as
part of the main array anymore. Generally, if you use identifier instances you shouldn't use
the original element anymore, otherwise, why use identifier instances in the first place if you
wanted to map everything anyway.

You cannot define an instance twice. If you defined the description = home instance already
and then define by occurrence = 1, if I try to map from occurrence = 1, there's going to be
nothing because the 1st occurrence is already accounted for. You can use the instance if it is
occurrence = 2 though (why?). Everything gets sent somewhere. If you defined occurrence
first, the first object (state = GA) is accounted for so if you define description = home
afterwards, only state = NY is available for use. Order matters for defining identifier
instances.

You can define multiple identifier instances at the same time if they are not under each
other. For example, you can define an instance for the array under addresses and an instance
for the array under phoneNumbers.

You had an array inside an array (e.g. postalCode), everything above is applicable to the child
array:

199

Internal Use - Confidential

 Figure 94: Array Sample Data

However, defining an instance on two different levels at the same time will not work. If you
defined for the addresses array element and also for the postcalCode array, the children array
won't be usable. Only the parent would be. For example:

200

Internal Use - Confidential

 Figure 95: Parent Array in Profile

the identifier instance code1=a would not recognize any values since it is inside another
identifier instance even though there is data that would fit this criteria.

You can use multiple qualifiers to define a single instance. If all qualifiers match, then that
subset of data is set aside in that instance. For example, description = home, state = GA will
match but description = home, state = PA will not (why?). Qualifiers are case sensitive. For
JSON, can only use qualifiers defined in the immediate elements/object in the array, not
based on other values in another like array inside the array.

XML
For XML profiles, the behavior is almost exactly same. Qualifiers are case sensitive. Instances
are exclusive and order matters when using occurrence and qualifiers. You can use multiple
qualifiers to define a single instance. If all qualifiers match, then that subset of data is set
aside in that instance and pulls all instances of the data. You can use the below for testing:

201

Internal Use - Confidential

 Figure 96: Matching Qualifiers

Figure 97: Parent Element with Children in it

202

Internal Use - Confidential

 Figure 98: Instance Based on Qualifier

When you define an instance of a parent element, you cannot define any instances for the
children elements. The profile will not have the option. Only if you remove the parent's
instance, then you can make an instance for the child or vice versa.

EDI
For EDI profiles, the behavior is almost exactly same as above too. Qualifiers are case
sensitive. Instances are exclusive and order matters when using occurrence and qualifiers.
You can use multiple qualifiers to define a single instance. If all qualifiers match, then that
subset of data is set aside in that instance and pulls all instances of the data.

203

Internal Use - Confidential

 Figure 99: EDI Profiles

There are multiple REF segments. It has PA, DP, or OR as a value for REF01. If you wanted to
pull that particular REF segment, you would define a qualifier for the REF01 segment and add
an identifier instance for the REF segment. Here you can only add qualifiers for segment
underneath of it.

Only loops can use qualifiers defined in children segments of the loop. Trying to define an
instance of a segment (when it belongs in a loop) instead of the loop will cause issues. You
can define a nested identifier instance, but it'll do weird behavior, so don't do it.

Data Type Formats

Date/Time format
When you use date/time data type in a profile, you'll see there are preset formats like
MMyyyydd and many others. The date/time formatting is used in map functions and properties
as well. If you use more symbols than there are digits, it will be formatted appropriately:

204

Internal Use - Confidential

 • d is day in month, 1-31. If you use d and it is the 7th, the result will be 7; dd and day is
7, result will be 07
• D is day in year, 1-365 (or 366). If you use D and it is the 9th day of the year, the result
will be 9; DD and it is the 9th day of the year, result will be 09; DDD will be 009

Note: while you can use as many symbols as you want e.g. dddd, but it wouldn't make sense
since you'll produce 0031. If there's no specific purpose for it, there's no need to use more
than what's necessary.

This is the full table of date/time formatting available:

Table 18: Date/Time Formatting

Symbol Meaning Example Symbol Meaning Example

G Era AD or BC w Week in year 27
designator
y Year 1996 W Week in month 2
M Month in July & 07 a AM/PM marker PM
year
d Day in 10 k Hour in day (1~24) 24
month
h Hour in 12 K Hour in AM/PM 4
am/pm (0~11)
H Hour in day 0 z Current time zone PDT
(0~23)N
m Minute in 30 Z Time zone offset -0500
hour from UTC
s Second in 55 ZZ Time zone offset -05:00
minute from UTC
S Millisecond 978 'Z' Converts to UTC <date
time (UTC-0:00) time
value>Z

E Day in week Tuesday ' Escape for text

D Day in year 189 '' Those '

are two single

205

Internal Use - Confidential

 Symbol Meaning Example Symbol Meaning Example

quotes, it makes a
single quote
F Day of week 2 (2nd
in month Wed in
July)

Common Date formats:

Table 19: Date Formats

Format Description Example

d Day in month, from 1 through 31 1, 15, 31
dd Day in month formatted to two digits, from 01 through 31 01, 15, 31
D Day in year; if D is 7 then DD, DDD makes 07 and 007 182, 24
M The month number of the year, from 1 through 12 1, 8, 12
MM The month number of the year formatted to two digits, 01, 08, 12
from 01 through 12
MMM The month name, formatted to three letters, from Jan Jan, Aug, Dec
through Dec;
MMMM will spell out the full name for longer months,
January, August
yy The year, formatted to two digits, from 00 through 99 99, 01, 15
yyyy The year, formatted to four digits, from 0001 through 1999, 2001,
9999 2015

Note: sometimes D is mistaken for d so the date format looks very strange.

Common Time formats:

Table 20: Time Formats

Format Description Example

SSS Millisecond, formatted to three digits, from 000 through 999 005, 135,
865

206

Internal Use - Confidential

 Format Description Example
s Seconds, from 0 through 59 1, 30, 59
ss Seconds, formatted to two digits, from 00 through 59 01, 30, 59
m Minutes, from 0 through 59 0, 14, 57
mm Minutes, formatted to two digits, from 00 through 59 00, 14, 57
h Hour, from 1 through 12 1, 8, 12
hh Hour, formatted to two digits, from 01 through 12 01, 08, 12
H Hour in 24H time, from 0 through 23 0, 8, 16
HH Hour in 24H time, formatted to two digits, from 0 through 00, 08, 16
23

Common formatting attributes:

Table 21: Attributes

Format Description Example

'T' Commonly used to delimit Date and Time, ' escapes yyyy-MM-
text between it dd'T'HH:mm:ss
2015-10-01T12:15:58
Z Timezone offset from UTC-0:00 formatted without HH:mm:ssZ
colon 12:15:58-0500
ZZ Timezone offset from UTC-0:00 formatted with HH:mm:ssZZ
colon 12:15:58-05:00
'Z' converts to UTC time (UTC-0:00) HH:mm:ss'Z'
17:15:58Z
z Time Zone; zzzz gives full time zone e.g. Eastern EDT, PDT
Daylight Time

Some example formats for October 1st, 2015 at 03:15:35.775pm (UTC-5:00):

207

Internal Use - Confidential

 Table 22: Formats

Format Result

MMddyyyy 10012015

MM/dd/yyyy 10/01/2015

MM-dd-yyyy 10-01-2015

hh 'o''clock' a, zzzz 03 o'clock PM, Pacific Daylight Time

yyyyMMdd HHmmss 20151001 151535

yyyyMMdd HHmmss.SSS 20151001 151535.775

yyyy-MM-dd 2015-10-01

yyyy-MM-dd'T'HH:mm:ss 2015-10-01T15:15:35

yyyy-MM-dd'T'HH:mm:ssZ 2015-10-01T15:15:35-0500

yyyy-MM-dd'T'HH:mm:ss.SSS 2015-10-01T15:15:35.775

yyyy-MM-dd'T'HH:mm:ss.SSS'Z' 2015-10-01T20:15:35.775Z

yyyy-MM-dd'T'HH:mm:ss.SSSZZ 2015-10-01T15:15:35.775-05:00

The platform has an internal date/time format used in the map: yyyyMMdd HHmmss.SSS (e.g.
20190501 105321.941). If you want to use the current date and convert it to another date
format, your input date mask needs to be yyyyMMdd HHmmss.SSS. If you get errors related to
the date/time formatting, it is often because your input doesn't match the formatting, check
your formatting again.

208

Internal Use - Confidential

 Number formats
Elements can be numbers and formatted a certain way as well. They work pretty much like
date/time:

Table 23: Number Formats

Symbol Meaning
0 A single digit, including zero will display
# A single digit, excluding zero will display
. Decimal point
, Comma
- Minus sign
E Used in scientific notation
% Displays as a percent (divides/multiplies by
100)
' Escape character
$ (or other currency May need to use Unicode value (e.g., "\u00A4")
symbol) for some characters

Examples of number formats:

Table 24: Number Formats

Input Format Result

123456.789 ###,###.### 123,456.789

123456.789 ###.## 123456.79

6.78 000.000 006.780

12345.67 $###,###.### $12,345.67

123E5 00E0 12E6

Numbers after the decimal will be truncated unless it is explicitly defined but numbers before
the decimal will be preserved whether you defined it or not (except for scientific notation

209

Internal Use - Confidential

 which will fit it to the format since it can).

The percentage works a little differently depending on if it is reading (input) or writing

(output) the data. If reading data:

Table 25: Percentage

Input Format Reads As

1.3% 000.000% 0.013

24.56% 00.00% 0.2456

1234.56% 0.0% 12.3456

As you can see, the number format is not taken into account when reading. It does not add
leading zeros or truncates, it preserves the entirety of the number.

For writing data (this does respect the number formatting):

Table 26: Writing Data

Input Format Writes As

0.1235 00% 12%

0.1235 00.000% 12.350%

Implied decimals
When implied decimals are used in the source profile (input profile) it shifts the decimal to
the left that many places. When implied decimals are used in a destination profile, it shifts
the decimal to the right that many places.

210

Internal Use - Confidential

 Reading data (the formatting is not respected):

Table 27: Read Data

Input Format Reads As

3456 0.0, implied decimal of 2 34.56

0.1235 000.000, implied decimal of 2 0.001235

Writing data (respects the formatting):

Table 28: Write Data

Input Format Write As

1235 0000.0000, implied decimal of 2 123500.0000

3 00.00, implied decimal of 3 3000.00

1.235 00.00, implied decimal of 2 124.00

Note: The inputs shouldn't really have decimals (they should be integers) in them since
implied decimals were meant for numbers that didn't have decimal points in place. That's why
the 1.235 after moving the decimal 2 places is 123.5, which is then rounded up to 124. Then
124.00 to fit the formatting.

Document Cache
Document cache (doc cache or just cache) is a way to store documents retrieved in a process
so you can use it later. Documents are kept only during the process and they do not carry on
in the cache into later executions. There are two ways to add to the document cache:

1. Using the Add to Cache shape

2. Using the cache option in most connector's operation tab to store whatever documents
you pull directly from a connector
The doc cache is used in two distinct ways in the platform:

211

Internal Use - Confidential

 1. Storing documents to look up information inside of the document based on some
value(s) that is already present in the document but coming from somewhere else
a. A profile is used so you can pick out elements as keys and values
b. A profile element key is defined
c. If you want to use a document property, it has to make sense to use it i.e.there's
a different value associated with each document, otherwise stick to profile
elements
2. Storing documents to be used as attachments in very specific connectors like the Web
Services SOAP or HTTP connector. This covers specific use cases and will have their own
sections
. The profile type is None since it's dealing with attachments not data types
a. A document property key is used to keep track of documents/attachments
Note: You should only be using a document property key if you plan to use the doc cache for
specific cases in conjunction with the SOAP or HTTP connector where documents are used as
attachments. All other cases, you'll be using a profile element key.

Document Cache Data Flow

Figure 100: Document Cache Data Flow

In real integrations there will probably be more than a couple of documents (how do you
simulate multiple documents being returned or retrieved?) to store and there would be a

212

Internal Use - Confidential

 connector or another shape in place of the message shape providing documents for the doc
cache. The second branch would also be data from another source and could be an entirely
different data profile.

I've added the following documents to the cache:

Figure 101: Adding Docs to the Cache

It's an XML document. I've configured my cache for that XML document and the elements in it.
The doc cache is configured for one index (what way will the doc cache retrieve data) with
one key (specifically what is the doc cache going to use to determine what document to pull
out of the cache). In this case my key is the Name element. In the retrieve from cache shape,
I pick my cache and which index to use (you can have multiple indexes i.e. different ways to
pull data. Maybe I want to pull data using Age in other scenarios). Then I pick how to define
the value that is going to be used as the key, in this case I am extracting a element from a
flat file called FirstName. Here's the full flow for the example above:

1. Some number of documents are retrieved

2. They are stored in a cache (Note: there is no more data flow after this shape)
3. Later in another branch or map or somewhere else, I want to retrieve the document(s)
stored in the cache
4. I have separate data flowing through the path, e.g. a flat file with a FirstName element
and I want to pull a document out of the cache that relates to this flat file
5. In the flat file I have the FirstName element and it equals "Billy" (the cache is case
sensitive) so the retrieve from cache shape is going to use "Billy" as the value for the
key Name since that was how I defined it earlier

213

Internal Use - Confidential

 6. The retrieve from cache shape is going to look into the cache and see there are two
documents, it will look at the key (which is the XML element Name) and see if one of
these documents has "Billy" as a value for the XML Name element
7. The cache sees there is a document with the Name element equal to "Billy" retrieves
that and sends it out of the retrieve from cache shape. This document will now be sent
down the path to future shapes

Some notes about caches:

• You can retrieve more than one document from the cache based on the same key
• You should have data coming from another place that has some same value as the key
element in the documents in the doc cache
• You can mimic a key for testing by having a message shape with the key value and
setting the parameter value to be current data so the retrieve from cache picks it up
and uses it
• You can add another index to use a different key but you can only use one index at a
time
• Using multiple keys in the same index works exactly the same way as one key. The
cache will look up the document's key element and see if all the keys have such a value
• If nothing matches, the retrieve from cache returns nothing and your data flow stops
• If document needs to be split up before going into the cache do so, or it'll be treated as
one document. Namely flat file and database profile need to be split up before going
into the cache. Whatever goes into the cache, is retrieved from the cache
• The cache does not filter out duplicates, you'll need to do it beforehand. You can use
the enforce unique options in a flat file and mapping or another strategy to remove
duplicates
• Document properties stay with the document when it is cached and retrieved
This will be basically how the doc cache works everywhere else as well. You need to supply
actual data (some data must go into the retrieve from cache shape) to retrieve data or the
whole document. Other places you can use the document cache:

• Map lookup function: give key values to retrieve as many elements from the document
as you want

214

Internal Use - Confidential

 • To the left side profile itself in the map, on one of the high level nodes (different node
for each profile) you can click on the triangle to use the option "Add Cached Data", this
will allow you to retrieve all of the document's elements using some key value that
comes from the incoming profile. You can use the elements just like a regular profile
element
• Setting property values to use in naming or decision shapes

Enforce one index entry per document

If you have this option checked in the doc cache, this allows you to:

• Retrieve all documents from the doc cache

• Use the remove from cache shape to remove specific documents
The enforce one index entry per document will make sure inside a document data itself there
is only one element that corresponds with the key. If you try to add this to the cache with the
option checked (notice there's multiple Name elements):

Figure 102: Multiple Name Elements Example

the cache will throw an error: "The Document Cache setting allows only one entry per
document in a given index. Multiple document entries were produced for index: index1"
because you have multiple keys in that document and the cache doesn't know which one to
use.

If the option is unchecked, you can now add the document above into the cache and the
cache will make one index entry for "Bob" and another index entry for "Billy". If you use "Bob"
as the key value later, the cache will retrieve that document or if you use "Billy" as the key
value later, the cache will also retrieve that same document. That one document has two
(multiple) index entries for it now.

Remove from cache shape

This allows you to remove either all documents currently in the cache or specific ones. The
remove from cache shape stops the data flow. If you want to remove specific documents, you

215

Internal Use - Confidential

 need to set the key value going into the shape like how you would do it for the retrieve from
cache shape.

Cross Reference Table

Cross reference tables (CRT) are like a very simple database table. It has columns and rows
and it is predefined, you shouldn't add things to it on a whim. You use a value in some
column(s) to look up another column value(s) in a row. Your results should be unique. Let's
use the table below:

Table 29: Cross Reference Table

Abbreviation Name ID

PA Pennsylvania 51

TX Texas 23

CA California 91

Let's say I want to look a value in Abbreviation using a value in ID, here's the logic flow:

1. Go to the ID column and look for the row with your ID value (let's use 51)
2. There is a row with the ID value of 51, so now pull the value from the Abbreviation
column (PA)
If the cross reference table couldn't find anything, it'll return nothing and represent it as a
null in the logs. Some places you can look up one value (message, notify shape) and other
places you can look up multiple values (map).

If you have a lot of values to add to a cross reference table, you can choose to import a csv
file instead, so you do not have to type it out. Some limitations:

• You can only have 2 to 20 columns

• You can have up to 10,000 rows
• CRTs were not meant to be used like a database, only to look up values so you cannot
do anything else with this table, no joining, no querying, only the most basic select
statement

216

Internal Use - Confidential

 • You can extend the table to add different values in different environments
• Values are case insensitive, so if you use TEXAS or TeXAS to match "Texas" it would still
be a match
The above is based on using an Exact Match for the data, so the values have to be exactly
what you defined. If you use Matches Regular Expression, you'll gain a lot more flexibility in
how you match something. Here's a CRT with Matches Regular Expression:

Table 30: Matches Regular Expression

ProductCode BusinessLine SKUType Family Class

20005 CONSUMER PROFESSIONAL PRO ^(?!.*XXX).*

30006 BUSINESS .*(ADVANCED).* ^(?!.*EDUCATION|.*ENTERPRISE).* ^(?!.*XXX).*

You'll start to notice some more characters for the data values. Those are part of regular
expressions (regex for short). A quick run-down of some basic regex for the CRT above:

.*(ADVANCED).* will be true if the data contains "ADVANCED" (or case insensitive versions of it
"Advanced", "advanced", etc), so "bob Advanced" or "Advanced Minion" but not "bob" or
"advance the kingdom". The .* means match anything (.) unlimited number of times (*), since
it is before and after that means there any be anything before and after "ADVANCED".

(NR3M|1M) will be true if the data is exactly "1M" or "NR3M" (the pipe | means OR), but not
"1M dollars".

^(?!.*EDUCATION|.*ENTERPRISE).*$ will be true if "EDUCATION" or "ENTERPRISE" is

not anywhere in the data, so "bob" or "School" will be true, but not "Higher Education" or
"First Education1 Assembly". The ^ means start with.

Since strings (this is what the values are in CRTs) cannot be null (but other things can be like
JSON elements, etc and we allow for integration of everything), if someone is trying to match
nulls or figure out if something is empty of has nulls they can use a small Javascript script (in
a user defined map function) to help transition (we're looking at the class column
specifically):

217

Internal Use - Confidential

 Figure 103: Script

where class_input is script_input and class_output is script_output. That's saying if the input
has a value make the class output variable the same as the input, otherwise make class
output = "XXX" meaning it is empty or null after all. The "XXX" is arbitrary, you can use
another value that will not show up in normal data. In which case, ^(?!.*XXX).* will come in
handy, meaning true if "XXX" is not anywhere i.e. the value has any characters in the original
intention or finding out if it is empty or not. So true if it is anything other than "XXX" and false
if it has "XXX" in it.

Alternatively, you can use ^(?!XXX) so XXX is not in the beginning assuming your normal data
isn't using "XXX" normally also. As you get more familiar with regex, you can make your own or
use it for another purpose.

Extending cross reference tables

You can extend cross reference tables in the process extensions options. When you deploy,
whatever is defined in the table will be the default values used in the environments. From
there you can configure each environment to have different data values in the CRT though if
you choose to override the current values.

You can update the extended CRT manually or through the environment extensions API,
though you need to be careful with the extensions API so you update all extensions at the
same time. See the Extensions section for more information.

Map Functions
Map functions help transform data values. There's a variety of functions. Most have an input,
output, an option(s) related to the function.

218

Internal Use - Confidential

 String functions
String functions have to do with strings, text values. Usually you map to the input(s) and have
a default value for the options in the functions, but you could use a default value for all
fields. Inputs are also called Original String. Fix to length is often used so we'll abbreviate it
to FTL. Whitespaces are characters in strings, so you have to use it or keep it in mind about
how it affects your string values. String related functions are case sensitive.

Table 31: String Related Functions

Function Name Input Options Output

Left character trim - remove characters on the left until "Crocodile" FTL: 5 "odile"
it is number of characters specified in Fix to Length

Right character trim - remove characters on the right "Crocodile" FTL: 6 "Crocod"
until it is number of characters specified in Fix to Length

Whitespace trim - remove white spaces from beginning " Crocodile "Crocodile and
and end of input (there are spaces at the beginning and and Alligator"
end of the example input) Alligator "

String append - makes Original String + Char to Append "Baby" Char to "Babygoats"
then removes characters on the right until value in Fix to Append:"goats"
Length (if applicable)
FTL: 6 "Babygo"
Char to
Append:"goats"

String prepend - makes Char to Prepend + Original "Baby" Char to "sharkBaby"

Stringthen removes characters on the left until value in Prepend:"shark"
Fix to Length (if applicable)
FTL: 7 "arkBaby"
Char to
Prepend:"shark"

String Concat - puts a delimiter (if there's one so it "Fried" Delimiter: "FriedChicken
counts in the character count) between the inputs then "Chicken" Wings"
removes characters on the right until value in Fix to " Wings"
Length (if applicable) Delimiter: * "Fried*Chicken*
Wings"

FTL: 7 "Fried*C"
Delimiter: *

FTL: 7 "FriedCh"
Delimiter:

String replace - searches for value in String to Search "Raining String to "Raining dogs and
(can use regex) and replaces with value in String to dogs and Search: "cat" bugss"
Replace cats" String to
Replace: "bugs"

String remove -removes value in String to Remove (can "Cats eat String to "Cats birds"
use regex). Notice there are two spaces in the output birds" Remove: "eat"
between the words, why?

String to lower - turns input into lowercase "Bob" "bob"

String to upper- turns input into uppercase "eat your "EAT YOUR
vegetables" VEGETABLES"

219

Internal Use - Confidential

 String split - splits into based on field length value or "Robots and Field Length: 5 Invalid because
delimiter. There cannot be more outputs than results of bunnies" Number of there are 4 values:
the split or you'll get an error, "Invalid split, expected to outputs: 3 "Robot","s and","
have at most X outputs but found {more than X}". There bunn","ies"
can be less splits than there are outputs, if no split
happens then it is the original value. Delimiters are "Robots and Field Length: 5 "Robot","s and",
removed on split bunn" Number of and " bunn"
outputs: 3

"Rob&ots Delimiter: "$" "Rob" and "ots and

and bun" Number of bun"
outputs: 3

"Rob$ots Delimiter: "$" Invalid, why?

an$d Number of
bunn$ies" outputs: 3

Numeric functions
Numeric functions have to do with math and numbers. Final number formatting can be
controlled by number formatting syntax, i.e. the whole number 3 will become 3.00 if your
number formatting is 0.00. The outputs below are character outputs so no formatting has
occurred.

Table 32: Numeric Related Functions

Function Input Options Output

Math absolute value - absolute value of a number, i.e.

|Value|

Math add - adds; Value + Value to Add

Math subtract - subtracts; Value - Value to Subtract

Math multiply - multiplies; Value*Value to Multiply

Math divide - divides; Value/Value to Divide

Math ceil - rounds Value up to the nearest whole number 1.2 2

Math floor - rounds Value down to the nearest whole 3.546 3

number

Math set precision - truncates accordingly to the 42564 Number of Precision: 2 42564.00
number of decimals indicated in Number of Precision
24.3561 Number of Precision: 2 24.36

Number format - sets the number formatting. See Data

Type Formats section for more information. An
alternative is to set the number formatting in the
profile's element options

The next set of functions use repeating elements in profiles or more complicated than simple
single operations.

220

Internal Use - Confidential

 Table 33: Repeating Element Functions

Function Input Mapping Output

Running total - keeps a running total of the <Header> Age to <out>

values passed into it per document. For flat <Age>25</Age> Value to <total>25</total>
file data, it will sum up all of the column <Age>50</Age> Sum <total>70</total>
values since it is one document. For non-flat- <Age>500</Age> Result to <total>500</total>
files, they are separated into separate </Header> total </out>
documents so only the elements that exist in
that document are tallied up <Header> Age to <out>
<Age>25</Age> Value to <total>25</total>
</Header> Sum </out>
Result to
total

25*** Age to <out>

50*** Value to <total>25</total>
500*** Sum <total>70</total>
Result to <total>500</total>
total </out>

Sum - sums the values for an element in a <Header> Age to <out>

document. <Age>25</Age> Value to <total>575</total>
<Age>50</Age> Sum </out>
If the sum function uses Reset Value field, <Age>500</Age> Result to
the sum will start over whenever there's a </Header> total
new unique Reset Value. There will be an
output produced for each value of Reset <Header> Age to <out>
Value. The three Age elements are added for <Person> Value to <total>575</total>
each Name they are under in the example <Name>Billy</Name> Sum <total>14</total>
<Age>25</Age> Name to <total>555</total>
<Age>50</Age> Reset </out>
<Age>500</Age> Value
</Person> Result to
<Person> total
<Name>Sangeeth</Name>
<Age>2</Age>
<Age>5</Age>
<Age>7</Age>
</Person>
<Person>
<Name>Daniel</Name>
<Age>5</Age>
<Age>50</Age>
<Age>500</Age>
</Person>
</Header>

Count - counter increments by 1 every time <Header> Age to <out>

the element is present in source and outputs <Person> Field to <total>9</total>
one output value. <Name>Billy</Name> Count </out>
<Age>25</Age> Result to
Reset Value will reset the counter and start <Age>50</Age> total
over and make a new entry per value of <Age>500</Age>
Reset value. </Person> Age to <out>
<Person> Field to <total>3</total>
The count for element Billy is counted as 5 <Name>Sangeeth</Name> Count <total>3</total>
since there are 5 Age elements for the Reset <Age>2</Age> Name to <total>3</total>
Value of "Billy". Since there are 3 Name <Age>5</Age> Reset </out>
elements, there are 3 total elements even <Age>7</Age> Value
though the first two totals are the same </Person> Result to
since Billy is repeated twice <Person> total
<Name>Daniel</Name>
<Age>5</Age>
<Age>50</Age>
<Age>500</Age>

221

Internal Use - Confidential

 </Person>
</Header>

<Header> Age to <out>

<Person> Field to <total>5</total>
<Name>Billy</Name> Count <total>5</total>
<Age>25</Age> Name to <total>3</total>
<Age>50</Age> Reset </out>
<Age>500</Age> Value
</Person> Result to
<Person> total
<Name>Billy</Name>
<Age>2</Age>
<Age>5</Age>
</Person>
<Person>
<Name>Daniel</Name>
<Age>5</Age>
<Age>50</Age>
<Age>500</Age>
</Person>
</Header>

Line Item Increment - outputs an <Header> Age to <out>

incremented count for each occurrence of <Person> Increment <count>1</count>
the element linked to the Increment Basis. <Name>Billy</Name> Basis <count>2</count>
The increment is reset based on a new value <Age>25</Age> Result to <count>3</count>
of the element linked to Reset Value <Age>50</Age> count <count>4</count>
<Age>500</Age> <count>5</count>
</Person> <count>6</count>
<Person> <count>7</count>
<Name>Bob</Name> <count>8</count>
<Age>2</Age> <count>9</count>
<Age>5</Age> </out>
<Age>10</Age>
</Person> Age to <out>
<Person> Increment <count>1</count>
<Name>Daniel</Name> Basis <count>2</count>
<Age>5</Age> Name to <count>3</count>
<Age>50</Age> Reset <count>1</count>
<Age>500</Age> Value <count>2</count>
</Person> Result to <count>3</count>
</Header> count <count>1</count>
<count>2</count>
<count>3</count>
</out>

Sequential value - a counter that will <Header> Age to <out>

increment by 1 for each occurrence of <Person> Increment <count>1</count>
element mapped to the Increment Basis. <Name>Billy</Name> Basis <count>2</count>
This value is stored on the atom (Atom <Age>25</Age> <count>3</count>
Management -> click on the atom -> <Age>50</Age> <count>4</count>
Counters) so next executions will start off <Age>500</Age> <count>5</count>
from the stored value. The Key Name field </Person> <count>6</count>
has some name for this counter, "counter1". <Person> <count>7</count>
You can edit the value of the counter on the <Name>Bob</Name> <count>8</count>
Counters tab so the next time the sequential <Age>2</Age> <count>9</count>
value function is used, it'll start with that <Age>5</Age> </out>
number. <Age>10</Age>
</Person>
If you were to run the same input again, the <Person>
count would be from 10-18 and the counter <Name>Daniel</Name>
"counter1" would now be 19. You will be able <Age>5</Age>
to use this same sequential counter <Age>50</Age>
"counter1" in different maps as long as the <Age>500</Age>
same Key Name is used. </Person>
</Header>
Batch size is 1 by default. Increasing the

222

Internal Use - Confidential

 Batch Size will improve performance when
mapping a larger number of records because
future values will be cached in memory.
When Batch Size is greater than 1, this could
break the sequence of numbers since that
amount of numbers are reserved at a time. If
they are not used, the counter will not reuse
it. If you specify a Batch Size of 100 but only
process 50 records. Those records will
receive sequential numbers 1-50. When the
process runs again, the next available
sequential value would be 101 since 1-100
was already "used"

Date functions
There are a few date/time functions.

Table 34: Date/time Functions

Function Description

Date retrieves the current system date/time in the format yyyyMMdd HHmmss.SSS, e.g.
format 20190508 131742.966

Get changes the input date from it's current format (mask) to the desired output format.
current View the Data Type Formats section in Unit 2 for date/time formatting
date

Lookup Functions
Table 35: Lookup Functions

Function Description

SQL lookup Uses a database connection to execute a SELECT query or stored procedure

Cross Same as using the cross-reference table to look up a value. View the Cross Reference
reference Table section in Unit 2
lookup

Document Same as using the document cache to retrieve a file, this time elements can be
cache selected from the profile
lookup

Simple Self-contained, two column, key value pair simple table. Works same way as a cross
lookup reference table lookup

Connector function
The connector function only has the connector call option. This is the same thing as using a
connector to pull data. Here you'll be able to define what fields act as the input (defined in
filters) and what to output (defined from the operation profile). Since this is the same thing

223

Internal Use - Confidential

 as using a connector, it will make your map take longer in general (compared to storing data
in a cache and pulling from it or something similar). View the Connectors section in Unit 2 for
more general information and Unit X for more specific connector information.

Custom scripting function

The custom scripting function lets you define your own inputs and outputs and use them in
scripting. You can use Groovy 1.5, Groovy 2.4, or Javascript. There are simple examples and
more complicated ones in the Boomi community. View the Custom Scripting section in Unit 3
for specific information about Boomi related functions/code.

Properties functions
View Property Types and Setting Properties section in Unit 2 for more information on the
properties available and what they do.

User defined functions

All of the functions above transform data once based on what you chose and usually have 1
main input. User defined functions allow you to use any of the above functions in combination
with each other. You will be able to define multiple inputs and outputs of your choosing.
Outputs from functions can be fed into inputs of other functions. On the map you'll only see it
looks like other functions where you can map inputs and outputs. Once you edit it, you'll see
it might have many other functions that make it up. Here's an example of one:

Figure 104: User Defined Functions

224

Internal Use - Confidential

 All other functions still follow the same rules and do the same things as above.

Extensions
Extensions allow you to set different values from the default values for certain fields and
properties of components in the process in different environments. Those fields include most
fields of connections, dynamic process properties, trading partners, process properties, cross
reference tables, maps, and PGP certificates. Extensions allow you to avoid redeploying a
process whenever you have credential changes or want to change values in the cross
reference and process properties. You define what you want to be extended then set the
value in one of two places:

1. Test mode extensions, when you test a process in the build tab, you can expand the
test extensions options and enter different values than the default ones present. These
extended values only apply in test mode for the specific atom you choose to execute,
so you can have different sets of values for each atom
2. Environment extensions, when you deploy to an environment, you have the option of
using a different set of values per environment
Extensions will look like this:

Figure 105: Extensions Window

The most common use cases are in environments and integration packs.

225

Internal Use - Confidential

 Extensions in parent/child processes
Generally, extensions are applied to the highest level deployed process. So how extensions
work in the main process depends on how the subprocesses are called — by a Process Call
shape or by a Process Route shape.

• If a parent or main process uses a Process Call shape to call the subprocess(es), those
subprocesses will be deployed as part of the parent process (when you deploy the
parent process). Defining extensions in the parent process will apply to the parent and
subprocesses
• If you define extensions in a subprocess, the extensions are not applied during
execution of the parent process since you deployed the parent process
• If the subprocess is also deployed independently of the main process, the extensions
would apply if the subprocess executes independently of the parent process with a
Process Route shape, the subprocesses are independent of the parent process. The
parent process, the Process Route component, and the subprocesses must all be
deployed independently.

• If you define extensions in the parent process, the extensions are applied only to the
parent process; they are not applied to any subprocesses
• If you define extensions in a subprocess, the extensions are applied only to that
subprocess; they are not applied to the parent process or to any other subprocess
• Note: the process properties are not independent, they remain in effect for the
duration of the process once it is accessed or set (by the environment extensions). To
change you need to use the Set Properties shape, map function, or custom scripting in
further processes

Setting Extensions
You can only set extensions manually in test mode but for environments you can set them
manually or through the API.

4.3.2.9.2.1 Test mode extensions

For test mode extensions, you'll see a window like this. Clicking on the "Test Extensions" bar
will expand the options and allow you to set the values:

226

Internal Use - Confidential

 Figure 106: Test Extensions

When you have test mode extensions, the values are saved <<atom_directory>>/settings-
test/<<process_ID>>/overrides.xml file.

4.3.2.9.2.2 Environment extensions updated manually

For environments, go to the environment and click on "Environment Extensions" and you'll get
an interface like this to do the same as the above:

Figure 107: Connection Settings

227

Internal Use - Confidential

 When you have environment extensions, the values are saved
<<atom_directory>>/settings/atom/overrides.xml file.

4.3.2.9.2.3 Environment extensions updated through the API

Through the API, you'll be using the Environment Extensions object. It would be best to use
the Atomsphere API connector to get the current extensions then map to the update
extensions map.

4.3.2.9.2.4 Data map and object definitions

For Data maps, you have to define the object definitions (profiles) before you can define the
data maps. But once defined, you'll be able to set new mappings in the environment
extensions.

Executing with extensions

When a process executes on an atom, it will use the environment extensions if any are set.

If you re-run documents from process reporting, environment extensions are still used.

If you re-run in Test Mode, the test mode extensions that were already setup for that atom (if
any) are used.

Extension Audit Logs

There are audit logs for extensions. They are accessible in the environment extensions
window. There aren’t any for test mode extensions. The audit logs will say who made changes
and at what time. The new values made at the time can be viewed in the New Value column.
Using notepad++ or another tool to compare differences will be helpful in figuring out what
was changed.

Cleanse Shape
The Cleanse Shape's purpose is for validation. What this shape does is either repair or reject
documents before processing further down the process flow. The Cleanse itself validates
based on a Profile. Through the profile, you can add restrictions on a per element/field basis

228

Internal Use - Confidential

 (we will go over an example using this shape). There are two paths using the cleanse shape:
'Clean' or 'Rejected'.

Here is an example process using the shape itself:

Figure 108: Cleanse Shape

As shown in the screenshot below:

Figure 109: Cleanse Shape Window

229

Internal Use - Confidential

 I added validation to Profile Element 'B'.

In this case, if the field element is missing, I set a default value to 'False', which will continue
the process flow. However, I set a validation to reject the document if the maximum length is
not met.

Example documents:
Example A: field1, field2, field3, field4, field5, field6

The process will hit the Cleanse shape and will first validate based on the Mandatory field
validation. This criterion is met. It then validates against the maximum length validation, to
which it also meets. It will continue down the 'Clean' route, to the Return Documents shape.

Example B: field1, field3, field4, field5, field6

As seen, the second element is missing. This document will go to the Cleanse shape and will
not pass the Mandatory field validation. However, the result is that this value will be
defaulted to 'False', rather than going down the 'Rejected' path.
The resulting document is as follows: field1, False, field3, field4, field5, field6. It will then
complete the maximum length validation and will go down the 'Clean' path as well.

Example C: field1, overmaximum, field3, field4, field5, field6

This document will go through the Cleanse shape and will pass the mandatory field validation.
However, it will fail the maximum length restriction, as it has 11 characters instead of 10,
and will go down the 'Rejected' path.

The 'Rejected' path in the flow of the process is performed before the 'Clean' path. You can
record/store this rejected message through Document Property->Meta Information->Base-
>Cleanse Result Message.

230

Internal Use - Confidential

 Custom Scripting
Support does not actually support troubleshooting and writing groovy scripts. Support can
choose to help the client further to the best of their abilities. The below are Boomi specific
scripting methods and class(es) so you should know how that works though.

The custom scripting function lets you define your own inputs and outputs and use them in
scripting. You can use Groovy 1.5, Groovy 2.4, or Javascript. You can have custom scripting as
an inline script or as a scripting component. The inline script only exists in the process that
uses it. Scripting component lets you reuse the same script in multiple processes, so if you
change it in the scripting component, you can change it in multiple processes.

The com.boomi.execution.ExecutionUtil class is the only Boomi related scripting import

clients should use, all other scripting principles would still apply.

The getRuntimeExecutionProperty method allows you to pull certain execution properties

(you can do the same via the parameter values):

Figure 110: Pull Execution Properties

The other methods related to getting or setting process properties are covered in Setting
Properties in Unit 2 (getProcessProperty,
setProcessProperty, getDynamicProcessProperty, setDynamicProcessProperty)
The getContainerId method (ExecutionUtil.getContainerId()) is the same as using ATOM_ID for
the getRuntimeExecutionProperty("ATOM_ID") for local atoms. The getContainerId method will
return a cloud atom's cloud parent atom ID.
The getAccountId method (ExecutionUtil.getAccountId()) is the same as using ACCOUNT_ID for
the getRuntimeExecutionProperty("ACCOUNT_ID").

231

Internal Use - Confidential

 The getBaseLogger method is documented in the Process Notifications section in Unit 2.

getDirectory() will return "component" in test mode but "processes\c66b7e1b-e11a-4de8-b3ba-

84dc2518c1e7" in deployment.

Combine and Split Documents

Split
The batching works the exact same way as flow control batching. The number in batch gets
filled up before making a new document. If there are 10 documents and the batching option
is 3, then there will be 4 documents. Doc 1, 2, and 3 have three documents each and Doc 4
has the last remaining document.

4.3.2.12.1.1 Flat File

You can split flat files by line or by profile.

4.3.2.12.1.1.1 Split by Line

Splits each line (record) into a separate document or documents. This is useful for breaking
up very large files into smaller pieces. For example, you may want to split one document with
5,000 records into 10 documents with 500 records each. If you split them prior to a Map
shape, you can achieve better performance.

• Batch Count — Used to specify the number of lines per document. The default is 0, which
means that batching is turned off, and each line will be split into a separate document. (This
is the same as setting the batch count to 1.)

• Headers Option

• No Column Headers — If selected, it is assumed that the input document does not
contain column headers. The first line is considered to be a record and will be put into
the first document.

• Remove First Line as Column Headers — If selected, the first line is considered to be a
header, not a record. The header will not be put into any of the documents.

232

Internal Use - Confidential

 • Retain First Line as Column Headers — If selected, the first line is considered to be a
header, not a record. The header will be put at the top of each document.

4.3.2.12.1.1.2 Split by Profile

Splits lines (records) based on the unique value of a profile element or "link element". The
files to be split must match the profile that you select and must contain the link element that
you select. Records with the same value for the link element are put in the same document. A
common example is a sales order file with header and detail information, where the header
information is repeated on each detail line. If you select the order number as the link
element, each resulting document will contain one complete order record.

The lines to be linked do not need to be read in consecutively. The Split Documents process
type searches through the entire document and groups matching lines in the output
document(s). For example, if you split a sales order file by state used in the address, you will
get a document for each state containing sales orders for that state (one for State_AAA, one
for State_BBB, etc.). However, you cannot link lines from multiple input documents. If your
integration scenario requires this, add a Combine Documents Data Process shape before the
Split Documents Data Process shape.

• Batch Count — Used to specify the number of link element values per document. The default
is 0, which means that batching is turned off, and records will be split based on one unique
value of the link element. (This is the same as setting the batch count to 1.) For example:

• If you split a sales order file by state used in the address and Batch Count = 1, you will
get one document for each state containing sales orders for that state (one document
for State_AAA, one document for State_BBB, etc.).

• If you split a sales order file by state used in the address and Batch Count = 2, records
will be split based on two unique values of the link element. The first document will
contain all sales orders for the first two states read in (for example, State_AAA and
State_BBB). The second document will contain all sales orders for the next two states
read in (for example, State_CCC and State_DDD). The third document will contain all
sales orders for the next two states read in, and so on.

233

Internal Use - Confidential

 • Keep Headers — If cleared, a header (the element names from the profile) will not be put into
any of the documents. If selected, a header will be put at the top of each document.

• Profile — When splitting by profile, you must select a flat file profile. If you do not select
one, the process will fail at this Data Process shape.

• Link Element— When splitting by profile, you must select a profile element for linking related
records. If you do not select one, the process will fail at this Data Process shape.

4.3.2.12.1.1.3 Example: Splitting flat files by line

The Split by Line option splits the flat file’s lines (records) into a separate document or
documents.

The following sample flat file shows 14 sales orders placed by four companies. The first line
contains a header.

Company, Address, City, State, ZipCode, Country, OrderNumber, LineNumber, ItemName, Quantity, Price,
TotalPrice
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 1, Item_A, 500, 10.00, 5000.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 2, Item_B, 25, 20.00, 500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 3, Item_C, 100, 25.00, 2500.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 2, Item_B, 50, 20.00, 1000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 3, Item_C, 10, 25.00, 250.00
CCC_Company, Address_CCC, City_CCC, State_CCC, 94003, US, 1300, 1, Item_B, 50, 20.00, 1000.00
CCC_Company, Address_CCC, City_CCC, State_CCC, 94003, US, 1300, 2, Item_C, 50, 25.00, 1250.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 1, Item_A, 250, 10.00, 2500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 2, Item_B, 10, 20.00, 200.00
DDD_Company, Address_DDD, City_DDD, State_DDD, 52004, US, 1500, 1, Item_A, 200, 10.00, 2000.00
DDD_Company, Address_DDD, City_DDD, State_DDD, 52004, US, 1500, 2, Item_C, 20, 25.00, 500.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 2, Item_C, 200, 25.00, 5000.00

First let’s vary the Batch Count. We use the Batch Count option to specify the number of lines
per document. (In each case we use the Header option Retain first line as Column Headers,
so that the first line is considered to be a header and appears at the top of each document.)
Using the sales order data shown above, here are the results.

234

Internal Use - Confidential

 • If Batch Count = 0 or 1, each non-header line is split into a separate document. We get 14
documents.
• If Batch Count = 2, the non-header lines are grouped by two’s, in the order that they are
read in. We get seven documents.
• If Batch Count = 3, the non-header lines are grouped by three’s, in the order that they are
read in. We get five documents. (The fifth document contains only two records.)
Now let’s keep the Batch Count at 0 or 1 and vary the Column Header options. We use the
Column Headers options to include or exclude headers. Using the sales order data shown
above, here are the results.

• If Batch Count = 0 or 1, each line above is split into a separate document. If we select No
Column Headers, it is assumed that the input document does not contain column headers.
The first line is considered to be a record. We get 15 documents.
• If Batch Count = 0 or 1, each line above is split into a separate document. If we select
Remove first line as Column Headers, the first line is considered to be a header and is
removed. We get 14 documents, none of which contains the header.
• If Batch Count = 0 or 1, each line above is split into a separate document. If we select Retain
first line as Column Headers, the first line is considered to be a header. We get 14
documents. The header appears at the top of each document.

4.3.2.12.1.1.4 Example: Splitting flat files by profile

The Split by Profile option splits the flat file’s lines (records) based on the unique value of a
profile element or "link element". You must select a profile and a link element.

The following sample flat file shows 14 sales orders placed by four companies. The data does
not contain a header.

AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 1, Item_A, 500, 10.00, 5000.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 2, Item_B, 25, 20.00, 500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 3, Item_C, 100, 25.00, 2500.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 2, Item_B, 50, 20.00, 1000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 3, Item_C, 10, 25.00, 250.00
CCC_Company, Address_CCC, City_CCC, State_CCC, 94003, US, 1300, 1, Item_B, 50, 20.00, 1000.00
CCC_Company, Address_CCC, City_CCC, State_CCC, 94003, US, 1300, 2, Item_C, 50, 25.00, 1250.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 1, Item_A, 250, 10.00, 2500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 2, Item_B, 10, 20.00, 200.00
DDD_Company, Address_DDD, City_DDD, State_DDD, 52004, US, 1500, 1, Item_A, 200, 10.00, 2000.00
DDD_Company, Address_DDD, City_DDD, State_DDD, 52004, US, 1500, 2, Item_C, 20, 25.00, 500.00

235

Internal Use - Confidential

 BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 2, Item_C, 200, 25.00, 5000.00
The flat file profile looks like this:

Record
Elements
Company
Address
City
State
ZipCode
Country
OrderNumber
LineNumber
ItemName
Quantity
Price
TotalPrice

We select Company as the link element.

First let’s vary the Batch Count. We use the Batch Count option to specify the number of link
element values per document. (In each case, we clear the Keep Headers option. A header
[the element names from the profile] will not be put into any of the documents.) Using the
sales order data and flat file profile shown above, here are the results.

• If Batch Count = 0 or 1, lines are split based on one unique value of the link element
(Company). There are four different values for Company in the input document. We get four
documents: a document containing the AAA_Company records, a document containing the
BBB_Company records, a document containing the CCC_Company records, and a document
containing the DDD_Company records
• If Batch Count = 2, the non-header lines are grouped by two values of the link element, in
the order that they are read in. We get two documents: a document containing the
AAA_Company and BBB_Company records, and a document containing the CCC_Company and
DDD_Company records.
• If Batch Count = 3, the non-header lines are grouped by three values of the link element, in
the order that they are read in. We get two documents: a document containing the
AAA_Company, BBB_Company and CCC_Company records, and a document containing the
DDD_Company records.

236

Internal Use - Confidential

 Now let’s keep the Batch Count at 0 or 1 and vary the Keep Headers option. We use the Keep
Headers option to include or exclude headers. Using the sales order data and flat file profile
shown above, here are the results.
• If Batch Count = 0 or 1, lines are split based on one unique value of the link element
(Company). If we clear Keep Headers, we get four documents, grouped by Company. A
header (the element names from the profile) is not put into any of the documents.
• If Batch Count = 0 or 1, lines are split based on one unique value of the link element
(Company). If we select Keep Headers, we get four documents, grouped by Company. A
header (the element names from the profile) is put at the top of each document.

4.3.2.12.1.1.5 Combining flat files

The Combine Documents process type combines multiple flat file documents into a single
document. When combining flat file data, make sure the individual documents do not contain
column headers that invalidate the format.

• Free-Form Header — Used to add header text to the combined document.

• Free-Form Footer — Used to add footer text to the combined document.

• Headers Option — Used to include or exclude headers that may be in the documents that are
being read in.

o No Column Headers — If selected, it is assumed that the input documents do not contain
column headers. The first line in each document is considered to be a record and will be
added to the combined document.

o Remove First Line as Column Headers — If selected, the first line in each document is
considered to be a header. The headers will not be put into the combined document.

o Retain First Line as Column Headers — If selected, the first line in the first document is
considered to be a header. The header will appear prior to the data in the combined
document.

4.3.2.12.1.1.6 Example: Combining flat files

The following are two flat file documents containing sales orders. Each document contains a
header.

237

Internal Use - Confidential

 Document 1 Header
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 1, Item_A, 500, 10.00, 5000.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 2, Item_B, 25, 20.00, 500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 3, Item_C, 100, 25.00, 2500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 1, Item_A, 250, 10.00, 2500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 2, Item_B, 10, 20.00, 200.00
Document 2 Header
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 2, Item_B, 50, 20.00, 1000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 3, Item_C, 10, 25.00, 250.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 2, Item_C, 200, 25.00, 5000.00

We can use the Free-Form Header and Free-Form Footer fields to add header and/or footer
text to the combined document. We use the Column Headers options to include or exclude
headers that may be in the documents that are being read in.

In the three scenarios below, we type “My Free-Form Header” in the Free-Form Header
field, and “My Free-Form Footer” in the Free-Form Footer field. Using the two documents
shown above, here are the results.
If we select No Column Headers, it is assumed that the documents do not contain column
headers. The first line in each document is considered to be a record (even though it is a
header in this case) and is added to the combined document. The combined document looks
like this:

My Free-Form Header
Document 1 Header
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 1, Item_A, 500, 10.00, 5000.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 2, Item_B, 25, 20.00, 500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 3, Item_C, 100, 25.00, 2500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 1, Item_A, 250, 10.00, 2500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 2, Item_B, 10, 20.00, 200.00
Document 2 Header
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 2, Item_B, 50, 20.00, 1000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 3, Item_C, 10, 25.00, 250.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 2, Item_C, 200, 25.00, 5000.00
My Free-Form Footer

If we select Remove first line as Column Headers, the first line in each document is
considered to be a header. The headers are not put into the combined document. The
combined document looks like this:

My Free-From Header

238

Internal Use - Confidential

 AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 1, Item_A, 500, 10.00, 5000.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 2, Item_B, 25, 20.00, 500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 3, Item_C, 100, 25.00, 2500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 1, Item_A, 250, 10.00, 2500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 2, Item_B, 10, 20.00, 200.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 2, Item_B, 50, 20.00, 1000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 3, Item_C, 10, 25.00, 250.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 2, Item_C, 200, 25.00, 5000.00
My Free-From Footer

If we select Retain first line as Column Headers, the first line in the first document is
considered to be a header. The header appears prior to the data in the combined document.
The combined document looks like this:

My Free-From Header
Document 1 Header
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 1, Item_A, 500, 10.00, 5000.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 2, Item_B, 25, 20.00, 500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1100, 3, Item_C, 100, 25.00, 2500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 1, Item_A, 250, 10.00, 2500.00
AAA_Company, Address_AAA, City_AAA, State_AAA, 19001, US, 1400, 2, Item_B, 10, 20.00, 200.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 2, Item_B, 50, 20.00, 1000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1200, 3, Item_C, 10, 25.00, 250.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 1, Item_A, 300, 10.00, 3000.00
BBB_Company, Address_BBB, City_BBB, State_BBB, 83002, US, 1600, 2, Item_C, 200, 25.00, 5000.00
My Free-From Footer

4.3.2.12.1.2 JSON
Two rules of thumb:

1. If it is a repeating array and the other "parent" elements do not have any elements on
the same level (peer/sibling elements) defined in the profile, the split will honor the
batching
2. All other cases, each record will be split into a separate document, the batch value is
not used

When a JSON profile is split, the elements outside of the split element are retained. If we
use:

239

Internal Use - Confidential

 Figure 111: JSON Profile

and split on the addresses array element, we'll get three documents of (with a different
description), as you can see the elements outside of the array are kept:

240

Internal Use - Confidential

 Figure 112: JSON Profile 2

If we were to change the batch, it wouldn't do any batching (since there are "parent"
elements that have siblings i.e. the birthday and spouse element exist). If we were to split on
a non-array element, nothing would really happen since there aren't multiple elements.

This would honor batching and the profile only has address and the elements underneath:

241

Internal Use - Confidential

 Figure 113: JSON Profile 3

4.3.2.12.1.3 XML
Two rules of thumb:

1. If there are parent elements that do not have any elements on the same level
(peer/sibling elements) in the profile and the split element is not an attribute, the split
will honor the batching
2. All other cases, each record will be split into a separate document, the batch value is
not used
When an XML profile is split, the other elements are retained in each document:

242

Internal Use - Confidential

 Figure 114: Elements Retained in Each Document

243

Internal Use - Confidential

 Figure 115: Allowing for Batching

4.3.2.12.1.4 EDI
Using this 850 profile:

244

Internal Use - Confidential

 Figure 116: 850 Profile

and splitting on PO1 segment makes the below, it retains the other segments as long as you
define your profile correctly according to the data:

Figure 117: Splitting on PO1 Segment

245

Internal Use - Confidential

 Each PO1 loop is put into a new document. If it was just a repeating segment, each instance
of the segment would be split up into a document.

Combine

4.3.2.12.2.1 XML
Let's start with multiple documents of the same structure like this:

Figure 118: XML Data

Here are the resulting data depending on what you combine by:

Table 36: XML Data Combos

Combine by <Records> Combine by <RequestRecord> Combine by <Address>

<DataA> <DataA> <DataA>

<Request> <Request> <Request>
<Records> <Records>
<Records>
<RequestRecord> <RequestRecord>
<RequestRecor
<Address>123 Boomi <Address>123 Boomi d>
Drive</Address> Drive</Address>

<PostalCode>800M1</Postal <PostalCode>800M1</Postal <Address>123

Code> Code> Boomi Drive</Address>

<Country>US</Country> <Country>US</Country>
<Address>456
</RequestRecord> </RequestRecord> Boomi Lane</Address>
</Records>
<Records> <RequestRecord>
<Address>789
<RequestRecord> <Address>456 Boomi Boomi
Lane</Address> Street</Address>
<Address>456 Boomi
Lane</Address> <PostalCode>800M1</Postal ....
Code>
<PostalCode>800M1</Postal </RequestRec
Code> <Country>US</Country> ord>

<Country>US</Country> </RequestRecord> </Records>

246

Internal Use - Confidential

 Combine by <Records> Combine by <RequestRecord> Combine by <Address>

</Request>
</RequestRecord> <RequestRecord> </DataA>
</Records>
<Records> ...
....
</Records> <RequestRecord>
... ...
</Request> </Request>
</DataA> </DataA>

Combine by <Records>, it will be repeated and everything within will be present, the
<Request> and <DataA> parent tags will remain, but CustomerID and ProjectID are gone.
Combine by <RequestRecord>, the looping will start/end with <RequestRecord> instead of
<Records>. If <Records> had any additional children elements, those children elements would
no longer be present after combining.
Combine by <Address>, <Address> it will be looped and <PostalCode> and <Country> are
removed.

Combining will remove all other elements on the same level and above the combining
element

4.3.2.12.2.2 JSON
You can only combine using repeating arrays. This will also erase all other elements on the
same level and outside the combining element. Combining multiple of these:

247

Internal Use - Confidential

 Figure 119: Combining Arrays

will produce this:

248

Internal Use - Confidential

 Figure 120: Combining Arrays

If I was to combine on the postalCode array element, everything else before postalCode is
removed:

249

Internal Use - Confidential

 Figure 121: Combining Arrays

4.3.2.12.2.3 Retaining element after a combine

If you want to retain the other elements like CustomerID and ProjectID, there are two main
ways to do it, via dynamic process properties (for smaller number of data to retain) or the
document cache (for larger amount of data to retain). Before we get started, in the above
example, let's say you get all of your documents from some Connector A using profile A. If
your profile A has a lot of fields like NetSuite connector, I would recommend mapping to a
simpler profile B with just the elements you need.

View https://community.boomi.com/s/article/xmlcombiningdocumentsandretainingotherele
ments for method.

250

Internal Use - Confidential

 4.4 Process Design

4.4.1 Use Tracked Fields/Document Tracking

Overview

Document Tracking allows you to capture key business data values from the documents being
processed to help find those specific documents in Process Reporting for tracking and
troubleshooting purposes. Using Tracked Fields is an important part of your functional
monitoring strategy. You can think of them as custom fields for Process Reporting.

Document Tracking allows you to answer questions like "what happened with Order Number
123?" By configuring connector operations to capture specific values, you can trace an
individual record through a process execution or even across multiple executions. This is
extremely useful in decoupled design patterns.

For example, consider the following scenario. You have a series of individual processes that:

1. Sync new orders from your website to the ERP application

2. Send the order details from the ERP to the shipping application
3. Updates the ERP with shipping tracking information
4. Sends an invoice from the ERP to the customer

By configuring the same Tracked Fields on each inbound and outbound connector shape, you
can easily search for a specific order number and trace its history through this workflow.

How it Works

1. Tracked Fields are first defined at your AtomSphere account level. This is because
Tracked Fields can be used across processes and are not tied a specific connector
type, operation, or profile.
2. Once defined in your account, you can configure individual connector operations to
populate values for those Tracked Fields based on the data they receive or send. You

251

Internal Use - Confidential

 will typically populate the Tracked Fields from profile elements of the document data
being processed.
3. At run time, these tracked values are captured as part of the process execution
metadata and made available through Process Reporting similar to other system-
generated document information such as file name and document size. Additionally,
you can search historical execution records by custom Tracked field values to quickly
find a specific document, and then drill down into the associated process execution
for additional details and troubleshooting information.

How to Configure

In this example we will show how to configure Document Tracking to capture the "Customer
ID" value when syncing account records from Salesforce to NetSuite. The objective is to
capture the same value on both the inbound Salesforce query and the outbound NetSuite
upsert calls for trace-ability.

Define Tracked Fields

You will need the Account Administration privilege to configure Document Tracking. To
configure a new Tracked Field for your AtomSphere account:

1. Navigate to Setup > Document Tracking (at the bottom)

2. Click to add a new field
3. Enter the Field Label and Data Type. The Data Type determines the available filter
options later

252

Internal Use - Confidential

 Figure 122: Tracked Field

Configure Connection Operations

The following example process extracts account records from Salesforce and sends them to
NetSuite. We want to configure Tracked Fields to the primary connector operations
components to capture identifying record information.

Figure 123: Add Tracked Fields to Connectors

4.4.1.2.2.1 Add Tracked Fields to the inbound Salesforce operation.

1. Edit the Salesforce Get operation component and go the Tracking tab.

253

Internal Use - Confidential

 Figure 124: Edit Tracking Tab

2. Add the "Customer ID" Tracked Field from the previous step
3. Configure the value to populate the Tracked Field. In this case choose the "Name" field
from the response XML profile (the one displayed on the operation's Options tab)
4. Save the operation

4.4.1.2.2.2 Perform the similar steps for the outbound NetSuite Upsert operation.

1. Add the same "Customer ID" Track Field

2. For the value, choose the "entityId" field from the request XML profile

Figure 125: Edit Tracking Tab(2)

254

Internal Use - Confidential

 When populating the Tracked Field value, it is possible to concatenate multiple dynamic
and/or static parameter values.

Process Reporting: View Executions Details

4.4.1.2.3.1 Deploy and execute the process then go to Process Reporting to view the
execution details.

1. Go to Manage > Process Reporting and find the execution result

2. Drill down to the execution details and select the Salesforce query operation
3. In the document details table, note the new Customer ID Tracked Field with the value
captured from the document that was executed

Figure 126: Document Details Table

4. You can do the same for the NetSuite Upsert operation and note the Customer ID
Tracked Field and value as well

Process Reporting: Search Documents

4.4.1.2.4.1 Search for a specific Tracked Field value across all documents and executions.

1. Within Process Reporting, switch to the Documents view

2. Add a filter for Tracked Fields, enter a search term (use * or ? for wildcards), and
select the specific "User Defined" field(s) to search

255

Internal Use - Confidential

 Figure 127: Show Documents

3. The results show documents across connector operations and process executions
matching the Tracked Field search term

Figure 128: Show Documents that Match Tracked Fields

256

Internal Use - Confidential

 Best Practices

• Use generic names - Avoid process- or endpoint-specific names for Tracked Fields
because they will be used across processes, applications, and data formats. For
example, consider "Account Number" vs. "Salesforce Customer ID"
• Only track a few fields - Capture only the key values required to identify and
troubleshoot documents
• Do not store sensitive values - Tracked Field values are uploaded and stored in the
AtomSphere platform database as part of the execution metadata. They are not stored
in your local Atom runtime. Be aware of this and avoid tracking sensitive values such
as credit card numbers, social security numbers, or any other protected information
because they are stored unencrypted outside your local environment. Also note that
the View Data privilege (which prevents users from viewing document data) does NOT
apply to Tracked Fields values
• There is a slight performance impact - Because the connector needs to parse the data
to extract the values, there will be a slight performance decrease. Typically, this is
negligible and the ability to easily troubleshoot records outweighs the costs, but it is
good to keep in mind not to track too many fields for a given connector operation
• Do not use for batches or repeating values - Generally if the incoming data contains a
list of values, it does not make sense to capture a value from a repeating item
because it will only capture the first instance. For example, if your process reads a
single large batch file containing many customer records, it would not be appropriate
to track customer-level values. Similarly, if your process sends an invoice record with
multiple line items, it would be appropriate to track the "invoice number" and
"customer number" from the "header", but do not track "item number" from the
repeating line details
• Create unique names - Be sure to use unique names for Tracked Fields
• Use consistently - To get the most benefit from Document Tracking, be sure to
consistently configure Tracked Fields for new connector operations

257

Internal Use - Confidential

 Considerations

• Tracked Field values are only captured for connector operations used within Connector
shapes. They are not captured for connector operations used in map functions or in
other shape parameters, such as a Connector Call lookup within a Decision shape, even
if configured within the operation
• Tracked Field values are only captured for deployed process executions. Values are
not captured for Test Mode executions
• Tracked Field values are available to view in Process Reporting for 30 days, same as all
execution records
• If you have a Low Latency listener process, remember execution information is not
captured which includes Tracked Fields
• Deleting a tracked field is permanent. Because very careful if you choose to delete a
Tracked Field
• The maximum value length is 1000 characters. Do not store the entire document data
(i.e. Current Data)
• The maximum number of Tracked Fields per account is 20. If you try to add more,
you'll get this error message, "Cannot add another Tracked Field. You are only allowed
to have 20 tracked fields."
• Tracked Field values are not available through the AtomSphere platform API. In other
words, you cannot use the platform API to query executions by a given Tracked Field
value. The exception to this is the X12 Connector Record object used for EDI
processing

Are tracked profile fields taken from the request or response document?

The most common use case for Tracked Fields is to capture values from the document data
using the Profile Element parameter type. Because some types of connectors actions both
take in and return documents, it is important to know which document will be used for
Tracked Fields.

In general:

258

Internal Use - Confidential

 • For connector "get" actions (e.g. Get, Query, Listen), the response document is
available for tracking, that is, the document data returned from the connector shape
into the process
• For connector "send" actions (e.g. Send, Create, Update, Upsert, Delete, Execute), the
request document is available for tracking, that is, the document sent to the
connector shape

However, there are a few limited exceptions:

• A small number of connectors behave slightly differently for "send" actions when the
"Return Application Error Responses" or "Return HTTP Responses" setting is enabled in
the operation. In this specific situation the response document is available for tracking
instead of the request. Because typically the request fields are desired for tracking,
consider disabling "Return Application Error Responses" and instead handle errors using
a Try/Catch shape. Connectors exhibiting this behavior include Salesforce, SAP, HTTP
Client, and Web Services SOAP Client, OData Client, Red Prairie, QuickBooks,
QuickBase, and any "legacy" connector.

4.4.2 Record Synchronization

• Avoid “looping” on bi-directional syncs (go to Bi-Directional Sync Section). For

example, if accounts are synched from system A to system B in one process,
and then system B back to system A in another process, and if both processes
query for records using a “last modified date” field, a record could bounce
indefinitely between one system and the other.
• When synchronizing records from one system to another, pass internal keys
from the source system to a field in the destination system. Optionally you
may also want to write keys from the destination system back to a field in the
source system. This not only assists true-ups and troubleshooting, but the
presence of an ID also confirms a completed sync. An empty ID field is an easy
indicator of an un-synched record.

259

Internal Use - Confidential

 4.4.3 Rejected Document Processing

• In most cases, going directly to a Stop shape from a route, decision, try/catch,
or business rules is discouraged. If any documents go down this branch, it will
be difficult to trace after the execution is finished.

4.4.4 Branch End

• Terminate your branches with Stop shapes where possible, even if it will not
change the results of the process. This will communicate your intentions as the
developer to whomever may need to maintain your process in the future.

4.4.5 Sensible Process Layout

• Position shapes on the canvas to most clearly communicate document flow.

• Put common routines into sub-processes.
• Add logical processing in to sub-processes as appropriate. This makes the
parent canvas intentions clear and allows for testing of subroutines
independently as necessary.
• Limit the number of shapes on a particular canvas. Too many shapes make
review and debugging difficult. Use sub-processes to clarify intent.
• Tip: To move groups of shapes on a canvas, use the browser zoom feature to
zoom out to show the entire canvas, then select the shapes and move them.
• Tip: To count the number of shapes on part, or all of a canvas, zoom out, then
select them. The resulting prompt will show the number of shapes captured.

4.4.6 Build from Outside In

• During initial process build, first import connections and related profiles. You
will need to reference those profiles in other shapes during logic build out.

4.4.7 Query Operation Design

• Avoid relative date filters in connector calls (Example: “Current date minus 5
days”).
o Rationale: If the process fails, debug and record re-run becomes
difficult as current date continues to advance. Record can be missed by
re-running the same query at a later date.

260

Internal Use - Confidential

 • If possible, use sync flags to select source data instead of using
lastSuccessfulRunDate. If you need to use a date for source data selection,
write a custom routine to store and use the highest date from the source
system instead of using lastSuccessfulRunDate.
o Rationale: the “lastSuccessfulRunDate” value stored by Boomi is based
on the time on the server where the atom resides, not the time used by
the endpoint system. If the atom server’s time differs from the
endpoint’s time, sequential queries could either miss or duplicate
records.

4.4.8 Use Process Options to better meet client needs

• For example, do you want to allow simultaneous executions? Is your listener

process set to Low Latency?
• Process Options, Boomi User Guide

4.4.9 Process Re-Run

• Design processes to be re-runnable (for example, should handle upserts rather

than just creates). This helps when re-running a process after a partially failed
execution.

4.4.10 Date Handling in Maps

• Where possible, add the appropriate date formats to date fields in both source
and destination profiles and avoid the "date convert" function.
• Date Math (ex: Add a Day; find the last day of the month, etc) usually requires
custom scripting.

4.4.11 Process Decoupling

You can use the following features to split and decouple your processes in Boomi:

261

Internal Use - Confidential

 Process call shape => sub-processes, e.g. using data passthrough
Overview

On the surface, the Process Call step allows you to execute or pass data to another Process as
part of your integration workflow. However, upon closer look, this step can enable you to
incorporate some sophisticated design strategies into your integrations including:

1. Creating repeatable process logic

2. Sequencing process executions
3. Grouping documents from multiple sources
4. Unit testing

Note: The Process Call step is only available in Professional and Enterprise Editions as part of
Advanced Workflow.

Before diving right into the use cases, a quick refresher on three important concepts to
understand when working with Process Call steps and sub-Processes:

• “Wait for process to complete?” - This option on the Process Call step determines
whether the sub-Process is executed “synchronously” or “asynchronously”. In other
words, should the calling Process wait for the sub-Process to complete before
continuing on to its next step or just kick off the sub-Process and immediately
continue to the next step.
• The sub-Process’ Start Step - If the sub-Process’ Start Step is configured as “Data
Passthrough”, the set of Documents from the calling Process will be passed directly
into the sub-Process to continue executing. In this case, the sub-Process essentially
functions as a continuation of the calling Process. However, if the Start Step is
configured as “Connector” or one of the other options, the sub-Process will retrieve a
new set of Documents. Also, if not configured as “Data Passthrough” the sub-Process
will execute once for each Document in the calling Process that reaches the Process
Call step. This means if ten Documents reach the Process Call step, the sub-Process
will execute ten times.
• Return Documents step - This step typically goes hand-in-hand with the Process Call
step. It passes Documents from the sub-Process back to the calling Process to continue
executing. One very important nuance to its behavior is that Documents are returned

262

Internal Use - Confidential

 to the calling Process only after the sub-Process has completed. This means that
Documents that reach a Return Documents step early in the sub-Process will wait
there until the rest of the sub-Process completes, then they will be passed back to the
calling Process.

Now armed with that understanding, let’s look at some of the clever things you can do with
the Process Call step.

Create Repeatable Process Logic

The AtomSphere development environment encourages reuse through its modular,

component-based architecture (think Connections, Profiles, Map Functions, etc.). The same
can apply to Process steps as well. If you find yourself adding the same series of steps to
Processes over and over, consider putting those common steps in a sub-Process and then use
the Process Call step to reference those steps wherever needed. Make sure the sub-Process’
Start Step is configured as “Data Passthrough” to pass Documents from the calling Process
into the sub-Process.

Looking for opportunities to create reusable sub-Processes--even if it’s only a few steps--can
facilitate maintenance, improve readability and organization, and enable more fine-grained
unit testing. Some examples of situations that often lend themselves to common sub-
Processes include:

• Connector response handling - Check for success/fail response codes and routing
accordingly
• Custom archiving, logging, or notifications - Standardize file names, log messages, and
alert formats.
• Working with a standard or “canonical” format - Individual Processes transform
disparate data to a common format then pass to a common sub-Process to continue
execution.

Here’s an example of a simple Connector response handling a sub-Process that interrogates

the application response to determine success or failure, generate a consistent alert
notification, and return the Document to the calling Process accordingly.

263

Internal Use - Confidential

 Figure 129: Example Sub-process

Tip: When possible make your sub-Processes even more reusable by configuring the steps to
inspect User Defined Document Properties instead of Profile Elements. This allows the sub-
Process to be called from Processes with different object types or even data formats.

Sequence Process Executions

If you have a series of Process that must run in a specific order, you can use a “master”
Process to execute each Process sequentially. In the Process Call step configuration, check
“Wait for process to complete” so each Process will wait until the previous Process is
complete before starting. Use the “Abort if process fails” option to control whether the next
Process should still execute if the previous Process failed or not.

Then you only need to set one schedule for the “master” Process and each sub-Process will
execute as soon as it can. This is often easier than trying to stagger schedules, especially for
Processes whose execution times may vary.

Deployment Note: Technically you only need to deploy the “master” Process however if you
want to be able to retry Documents you must deploy each sub-Process as well.

264

Internal Use - Confidential

 Figure 130: Sub-process Example

Group Documents from Multiple Sources

Sub-Processes can be used to group Documents from multiple sources together, so you can
operate upon them as a single group later in the Process to do things like combining, splitting,
and take advantage of Connector batching. This use case is probably best illustrated with an
example.

Let’s say you need to extract both direct and indirect customer records from some web
application and create a single CSV file. You need to query for each type of customer however
the application’s API does not allow you to use a compound filter such as “WHERE
customerType=‘DIRECT’ OR customerType= ‘INDIRECT’”. This means you will need to extract
each type with a separate query call. The Process might look something like this:

Figure 131: Multi-source Document Call

265

Internal Use - Confidential

 The problem here is that two different files will be created because each Branch path will
execute to completion, meaning the “DIRECT” customers retrieved on Branch 1 will be
mapped, combined (Data Process step), and sent to the FTP server before the “INDIRECT”
customers are retrieved on Branch 2. So how can you get both groups of Documents together,
so they can be combined into a single file?

Return Documents step to the rescue! By moving the Connector calls into a sub-Process, you
can rely on the fact that the Return Documents step will wait until the Process is complete
(remember the three concepts above?) to return a single group of Documents to the calling
Process. The single group of Documents will then continue down the calling Process together
and can be mapped and combined into one CSV file. The calling Process will now look like
this:

Figure 132: Calling Process

...and the new sub-Process will look like this:

Figure 133: Multi-source Document Call with Return

266

Internal Use - Confidential

 Unit Testing

Process Call steps can also help with unit testing by separating the source of inbound data
from the rest of the Process workflow. For example, you can have one Process that performs
the “real” Connector call to retrieve data from the intended source and another Process that
retrieves test data staged somewhere such as a local disk folder (or maybe even contained
directly in the Process using a Message step). Then each of these simple Processes can pass
the retrieved Documents to the same “main” Process containing the bulk of the workflow
steps.

This is especially useful for testing integrations that use event-based Connectors that cannot
be run in Test Mode such as Web Services Server and AS2.

Below is an example of two processes, one that actually listens for incoming data via the Web
Services Server and one that retrieves staged test data from a local Disk location. Both then
pass the Documents to the common “Create Order” sub-Process that performs the rest of the
integration. The “test” Process can be run in Test Mode to assist during development before
deploying the actual listening Process for production use.

Figure 134: Two-process Comparison >>

Process routing shape => dynamic routing

267

Internal Use - Confidential

 Overview

The Process Route allows you to dynamically choose a subprocess to execute at runtime based
upon some document value. The "routing rules" are configured separately which define the
mapping between a given value and the subprocess to execute, as well as the "interface" for
how the subprocess should behave such as Passthrough or not and the number of expected
Return Document paths.

The feature can greatly reduce the configuration and maintenance effort when building
processes that must handle many different types of documents and therefore many different
execution paths. However additional upfront design and coordination is required to use
effectively. If your process only needs a few routes that are relatively static, using a Process
Route may be overkill. If the process has many routes that are constantly changing, the extra
design and configuration sophistication will pay off in the long run with respect to ongoing
maintenance.

The two key aspects that make Process Route different than a regular Process Call are:

1. Ability to dynamically determine which subprocess to execute based on some runtime

criteria
2. Ability to deploy the main and subprocess(es) independently

For more information visit the User Guide Process Route Components and Process Route
Shape.

Use Case

The scenario for this tutorial is as follows:

• Inbound files for many different types of records are placed in a single disk directory.
• The format (and consequently the Profile), mapping rules, and destinations vary by
record type.
• The files follow a consistent file naming convention: <recordtype>_<date>.<file
extension>.
• For this tutorial there will be two record types:

268

Internal Use - Confidential

 o Orders - Inbound sales orders in a CSV format that should be sent to the Order
Management database
o Employees - Employee events such as new hires and terminations in an XML
format that should be sent to the HR application via web services

Motivations

Without the Process Route shape there are several alternative approaches for accommodating
this situation:

1. Create a single process that reads all the files and uses a Route shape to send each file
to a record type-specific subprocess.
2. Create a separate process for each record type that only reads those files.

However, these approaches have several drawbacks:

1. With the first option, every time a new record type is added or if one of the
subprocesses needs to be modified, the parent process needs to be modified and
redeployed which will also redeploy all the other subprocesses. This can be very
difficult to manage if changes to another subprocess are underway but not ready to be
deployed yet. Additionally, when the number of potential routes exceeds 20-30, the
Route shape can be impractical to use.
2. With the second option, there would simply be a lot of processes to create, deploy,
and moreover schedule. Any common elements would need to be maintained and
deployed repeatedly.

With the Process Route feature, the main process and individual subprocesses can be added,
maintained, and deployed independently. Additionally, it requires only a single main process
to develop, deploy, and schedule, which is more efficient with respect to the maximum
concurrent executions on the Atom and provides more control over throttling.

Approach

The general approach for using the Process Route is as follows:

269

Internal Use - Confidential

 1. Design considerations including the type of subprocess, route key value, and return
paths
2. Create subprocesses
3. Create Process Route component
4. Create main process with a Process Route shape
5. Deploy main process, subprocesses, and Process Route component

Note the Process Route paths within a given process execution are still executed sequentially,
NOT in parallel. Return paths are executed sequentially as well.

Implementation

The example processes below are used to illustrate the relevant concepts and are not
intended to be fully configured.

4.4.11.2.5.1 Design

The Process Route component essentially defines the interface or contract between the main
process and subprocesses, so it is important to think through the desired interaction upfront.

Below are the relevant design considerations for this example:

• General: There should be a single process that reads in all files and routes each file to
a specific subprocess that can handle the mapping, logic, and target destination for
the given record type.
• Process Type: The documents from the main process should be passed directly into the
subprocess so the subprocesses will need to be configured as Data Passthrough.
• Route Key: Because each subprocess will handle a single record type, it makes sense to
use the record type value as the Route Key. The record type is available as part of the
incoming file name, so the main process can parse the file name to determine the
record type. The resulting value can be stored in a dynamic document property named
"RECORD_TYPE" to use for the Route Key lookup.
o IMPORTANT: The Route Key is an exact value match. The main process is
responsible for preparing the value appropriately to match the Route Key
exactly.

270

Internal Use - Confidential

 • Return Paths: The main process should do some generic logging upon successful or
failed completion, so it needs to know the outcome of the subprocess. If separate
return paths "Success" and "Failure" are used, each subprocess can implement logic as
necessary to indicate success or failure.
• Additional Design (optional): To consistently return information about the failure--
whether due to invalid data, connectivity issue, or application response, the main
process will expect the subprocess to populate a dynamic document property named
"FAILURE_MSG" when returning documents via the "Failure" path.

4.4.11.2.5.2 Create Subprocess(es)

Orders subprocess:

Figure 135: Orders Sub-process

Employees subprocess:

Figure 136: Employees Sub-process

271

Internal Use - Confidential

 1. Create a subprocess for each record type. The Start shape should be configured as
Data Passthrough.
2. Each subprocesses should include validation, mapping, destination connector calls,
etc. specific to the respective record type.
3. Each subprocess should have two Return Document shapes, Success and Failure, and
perform routing logic to ensure documents are sent to the appropriate Return
Documents shape.
4. Additionally, the subprocess should populate the FAILURE_MSG dynamic document
property as necessary, for example using the business rules validation result, try/catch
message, application response field, etc.
5. Save the process.

4.4.11.2.5.3 Create Process Route Component

1. Create a new Process Route component.

2. On the General tab, enable "Passthrough" and add two Return Paths, "Success" and
"Failure" (names are arbitrary):

Figure 137: Import File Record Types

272

Internal Use - Confidential

 3. On the Process Routing tab, add two Keys, "ORDER" and "EMPLOYEE". For each record
type, select the respective subprocess as the Route to Process, and map the
subprocess's Return Documents shapes to the Process Route's Return Paths
appropriately:

Figure 138: Process Routing

4. Save the process.

4.4.11.2.5.4 Create Main Process

Figure 139: Main Process

273

Internal Use - Confidential

 1. Use the Disk connector to read all files.
2. Add a Business Rules shape configured with a single map function rule to parse the
record type value from the incoming file name and store in dynamic document
property named "RECORD_TYPE". This will be used for the Route Key later.
• For more details on this Business Rules technique, see How to manipulate
properties without a Map shape. Map function details:

Figure 140: Manipulate Properties

3. Add a Process Route shape configured with the Process Route component created
above. Leave the defaults for the other General tab fields. On the Route Parameters
tab, add a single parameter type for the dynamic document property "RECORD_TYPE."

Figure 141: Process Route Shape

274

Internal Use - Confidential

 Figure 142: Route Parameters

• Note the Route Key value can be a concatenation of parameter types if useful
to your scenario.
4. The Process Route's Return Paths are automatically drawn on the canvas. Perform
subsequent steps as appropriate for your scenario. In this example, the Failure path
throws an exception and references the common FAILURE_MSG dynamic document
property:

Figure 143: Exception Shape

In summary, it is important to note that none of the configuration is file- or record-type

specific.

275

Internal Use - Confidential

 4.4.11.2.5.5 Deploy Components

On the Deploy tab:

1. Deploy (and attach to the desired Environment/Atom) the main process.

2. Deploy (and attach to the desired Environment/Atom) all subprocesses referenced in
the Process Route.
• IMPORTANT This is mandatory. This is different than the regular Process Call in
which the subprocesses are automatically included in the main process's
deployment.
3. Switch the Component type list to Process Route:

Figure 144: Component Type List

4. Deploy (and attach to the desired Environment/Atom) the Process Route component.

4.4.11.2.5.6 Execute and View Logs

When the main process executes, keep in mind the subprocesses are Data Passthrough so they
will not display as separate executions records in Process Reporting. However, the individual
subprocess(es) that were executed are captured in the main execution's Process Log:

Figure 145: View Logs

276

Internal Use - Confidential

 Also remember the incoming file name will be captured on the inbound disk connector's
tracked fields which can be viewed and searched in the Document view.

Understanding Deployment Implications

One of the greatest advantages of the Process Route component is the deployment flexibility.
Because the Process Route component and subprocesses can be deployed independently, it
greatly reduces the regression impact when changes arise. This flexibility allows you to
modify and redeploy a single subprocess without having to redeploy the main process which
would typically include all other subprocesses.

4.4.11.2.6.1 Common Scenarios

• If you need to modify one of the subprocesses, you only need to redeploy that
subprocess. You do not need to modify or redeploy the main process or the Process
Route component. However, keep in mind any changes should be compatible with the
design assumptions of the main process.
• If you need to add a new route and subprocess, you only need to create and deploy
the new subprocess and then update and redeploy the Process Route component. You
do not need to modify or redeploy the main process.
• If you need to change common logic in the main process (e.g. error handling), you only
need to modify and redeploy the main process. You do not need to modify or redeploy
the subprocesses or Process Route component. However, keep in mind any changes
should be compatible with the design assumptions of the subprocesses.

4.4.11.2.6.2 Cautions

• If you need to make a change that affects the interaction/interface between the main
process and subprocesses (such as adding a new Return Path or assuming the
availability of some process property or document cache), you should do so
thoughtfully to ensure backward compatibility. Depending on the nature of the change
this could be as simple as providing default values for properties or as complex as full
coordination and redeployment of all processes and components.
• Because there is no deployment dependency between the main and subprocesses, it is
quite possible to have different versions of the same component used in different

277

Internal Use - Confidential

 processes (for example, you could have Profile revision 1 in the subprocess but Profile
revision 2 in the main process). Take care to ensure any changes to common
components are non-destructive and backward compatible, or that you coordinate the
re-deployments of both the main process and subprocesses in those situations.
• Extensions must be declared in each deployed process; in other words, in both the
main and subprocess. For example, if both the main and subprocess wish to use the
same connection or process property extension, the extensions must be declared in
both processes. This differs from the regular Process Call shape scenario in which all
the extensible components from the subprocesses are "rolled up" and declared only
once in the main process.

Considerations

4.4.11.2.7.1 Understand Process Route's execution behavior:

• Documents will only be routed to a single path. Duplicate and wildcard keys are not
allowed so documents cannot match to multiple paths.
• Documents are first grouped together by Route Key (similar to the regular Route
shape) and then each group of documents is sent to the respective subprocess in the
sequence defined within the Process Route component.
• The Return Paths are executed in the sequence defined within the Process Route
shape, not the component.
• The Default path for documents that did not match a Route Key is executed last.
• Again, Process Route subprocess executions behave the same as regular Process Call
executions with respect to:
o Process Reporting - "Passthrough" subprocesses are logged as a continuation of
the main process; there is not a separate execution record. Non-passthrough
subprocesses will create a separate execution record.
o Execution "context" is shared so things like process properties and document
caches are available and modifiable between the main and subprocesses.

4.4.11.2.7.2 Other considerations

• When performing a "deep" copy (Include Dependents=true) of a process with a Process

Route shape, the Process Route component is NOT copied.

278

Internal Use - Confidential

 Message queueing, either Atom message queueing or JMS based
brokers => best practice

Atoms natively support message queuing. Messages are managed by a shared queue server
embedded in the Atom Java process. This approach enables simple, reliable asynchronous
process execution whereby a process invokes another process in a separate invocation.

Implementation
• Reusable queue components are configured at the account level. Each queue
component specifies the configuration of a message queue, including its name and the
messaging model with which the message queue can be used, either Point-to-Point or
Publish/Subscribe.
• An Atom’s shared queue server creates a message queue upon invocation of an Atom
Queue connector Get, Send or Listen operation that specifies a queue component.
• Messages consist of references to documents rather than actual document content.
Each message references one or more documents. The document metadata in
messages includes dynamic document properties.
• Messages persist on a message queue until one of the following occurs:
• The message is consumed.
o The documents that the message references are purged.

The messaging system supports all Atom types including Molecules, which enable clustering,
and Atom Clouds, which enable multi-tenancy.

How to Use
1. Create queue components.
2. Configure the Atom’s shared queue server, if you are the Atom’s owner. The default
configuration is likely to be suitable for your purposes, at least initially.
3. Build and deploy processes that use Atom Queue connector operations that specify the queue
components you created.
4. Perform message queue management actions as needed.

279

Internal Use - Confidential

 While Atom message queuing is simple, robust and reliable, it lacks many of the features of
an enterprise queuing system, such as message priorities and variable persistence guarantees.
If Atom message queuing does not meet your requirements, consider installing your own
messaging system and using the JMS connector.

Note: Atom message queuing is an optional feature. To have this feature enabled in your
account, contact your Boomi sales representative.

Benefits
The benefits of having a native message queuing implementation can be categorized as
follows:

• Asynchronous communication — Processes writing and reading data execute independently of

each other. Requests are batched in real-time. Enqueuing requests is more efficient than
spawning an individual execution for each real-time message or writing messages to disk for
later batch processing in a scheduled process.

• Decoupling — Producers and consumers of messages are independent and can evolve
separately at their own rate. Workflow is encapsulated into logical, reusable units — that is,
separate processes — which are organized, maintained and deployed independently.
• Multiple recipients — Messages can be sent to multiple recipients independently. Likewise,
their receipt can be monitored independently.

• Redundancy — Message queues can persist messages until they are fully processed.

• Resiliency — Decoupling implies failures are not linked, thereby mitigating the risk of
unreliable applications. A producer can continue to enqueue messages while consumers are
temporarily disabled.

Typical usage scenarios

Following are typical usage scenarios for Atom message queuing employing message queues:

• Consider a requirement for fully disconnected process execution. This requirement is

common in the case of services with known reliability issues. Having a separate processing
path enables more granular retries. In this scenario an Atom Queue connector Send
operation would send failed documents to a Point-to-Point message queue in batches. In

280

Internal Use - Confidential

 another process an Atom Queue connector operation — either Listen or scheduled Get —
would receive the batches.
• Consider a requirement for aggregate AS2 document processing — a batch process
operating upon separate incoming documents. In this scenario the AS2 Shared Server
connector would listen for incoming documents, and an Atom Queue connector Send
operation would send them to a Point-to-Point message queue. In another process a
scheduled Atom Queue connector Get operation would receive documents in batches.
• Consider a requirement for dispersed document processing — incoming documents
processed in parallel, with failed documents retried individually. In this scenario a primary
process would have an Atom Queue connector Send operation to send documents to a
Point-to-Point message queue in small batches, in some cases a single document. In
another process an Atom Queue connector Listen operation would receive the batches and
in subsequent steps route the documents for concurrent processing.
• Consider a requirement to route messages between Atoms.
o The server Atom would act much like an enterprise service bus (ESB). It would deploy
two Web Services Server processes, one using an Atom Queue connector Send
operation to send documents to a Point-to-Point message queue for consumption by
client Atoms and the other using an Atom Queue connector Get operation to receive
documents sent by client Atoms from a Point-to-Point message queue.
o Each client Atom would deploy two Web Services SOAP or HTTP client processes, one
using an Atom Queue connector Get operation to receive documents sent by the server
Atom from a Point-to-Point message queue and the other using an Atom Queue
connector Send operation to send documents to a Point-to-Point message queue for
consumption by the server Atom.
• Consider a requirement for a hub and spoke system in which documents are produced in
the hub and made available for consumption on a variable population of spokes. In this
scenario the publisher (hub) would have a scheduled process that executes an Atom
Connector Send operation to send documents to a Publish/Subscribe message queue. At
any given time, the message queue would have zero or more subscribing message queues
(spokes). Each subscriber would be executing a process in which an Atom Queue connector
Listen operation would receive published documents. Because subscribers are unknown to
the publisher, they can activate or deactivate at any time without necessitating a
modification to the publishing process.

281

Internal Use - Confidential

 Limitations
Atom message queuing is subject to the following limitations:

• Messages cannot be sent between accounts.

• Messages cannot be sent between Atoms.

• Message queues cannot be directly accessed from outside the Atom.

Listener management
• Decoupling is key for listeners in production. The listening process should
receive the data, then add the document to the decoupling mechanism
including an optional archive step. A separate scheduled or listener process
would then draw documents from that mechanism. This limits issues that
frequently occur with simultaneity and unintentionally processing documents
multiple times.
You can view the status of listener processes that are deployed to an Atom, Molecule, or
Atom Cloud to retrieve messages from a message queue by going to the Listeners panel
in Manage > Atom Management. In this panel, you can also pause, resume, and restart
listeners.

Dead letter queues

Dead letter queues contain undeliverable messages. Each ordinary message queue can have
one associated dead letter queue.

Message delivery failure commonly occurs because of

1. network failure

2. purging of referenced documents

3. general message processing failure

When a failure occurs, the shared queue server will attempt to redeliver the message up to
six more times, in intervals of 1, 2, 3, 5, 8, and 24 seconds. After six failures the message is

282

Internal Use - Confidential

 sent to the appropriate dead letter queue. If the dead letter queue does not yet exist it is
automatically created.

You should periodically analyze the contents of dead letter queues to identify the reasons for
delivery failures.

Data storage
An Atom’s shared queue server’s backing disk store is as follows:

• For a single tenant Atom or Molecule, the shared queue server’s runtime configuration
information and message queues reside in the “queue” directory within the Atom installation
directory.

• For a multi-tenant Atom Cloud, the shared queue server’s runtime configuration information
resides in the “queue” directory within the Atom installation directory. Message queues, on
the other hand, reside in the “queue” directory within each account’s directory.

Note: Documents referenced in currently enqueued messages persist until a scheduled purge
occurs. This is true even while the Atom or process is configured to enable purging of data
immediately following process execution. When documents referenced in an enqueued
message are purged, the message itself is also purged.

4.4.11.3.8.1 Monitoring the message queue service

The following metrics are available for monitoring an Atom’s message queue service:

• overall status

• message store disk usage

• temporary data store disk usage

• job scheduler store disk usage

• memory usage

To monitor these attributes, you need to use a systems management tool, such as Zabbix, and
use a JMX hook (a standardized interface for Java programs).

283

Internal Use - Confidential

 4.4.12 Business Rules Shape Considerations

• Using the Business Rules shape is preferable to chaining together decision

shapes.
• To make its decisions, the Business Rules shape can execute map functions and
interrogate properties and cache.
• Business Rules, Boomi User Guide

4.4.13 Accessing Data Changed in a Prior Branch

• At a Branch shape, Boomi sends identical copies of the documents at that

shape down each branch in sequence. As such, edits made to documents in
branch 1 are not available in branch 2. Transfer of this data must be handled
differently. For example, a Dynamic Document Property set in Branch 1 will not
be available in Branch 2.
• Common Method 1 (preferred): Use a unique identifier (single or composite)
from your data available at the branch shape to cache edited data, then used
“Add Cached Data” or “Retrieve from Cache” to retrieve that data.
• Common Method 2 (limited use only): Set Dynamic Process Property (which has
process scope) to store data in Branch 1, then retrieve that data in Branch 2. If
you process multiple records in Branch 1, you must add a flow control / Run
Each Record Individually prior to the Branch shape or risk overwriting the DPP
for each record. The flow control will slow processing, which is undesirable. If
it must be used, consider executing processing in a sub-process to limit impact
of the flow control.

4.4.14 Caching Considerations

Overview

Document Cache is a very useful concept when it comes to designing complex processes which
involve bulk data or data retrieval. Here are a few concepts as a refresher:

• Indexing - This is how documents are stored for data retrieval. A few tips to keep in mind:

284

Internal Use - Confidential

 o IMPORTANT: Documents must be cached and indexed by the granularity you wish to
retrieve them. For example, if you have a single CSV file containing all your
employees but you want to be able to retrieve/lookup individual employees by
email address, you must split the data before adding it to the cache.
o You must configure at least one index with at least one key.
o The value(s) for the keys must not be blank.
o An index can consist of one or more keys from a profile and/or document
properties. For example, you could create a "composite" key using a combination of
first name and last name.
o You can configure multiple indices, if you want to be able to retrieve the same
cached documents using different parameters at different points in your process.
For example, if you were caching a set of employee records, you could create
separate indices so that you could lookup cached records by employee number,
name, or email address.
• Document Properties - Any Document Properties associated with a given document are also
cached along with the document data itself. When a cached document is retrieved using a
Load from Cache shape, the original document properties are restored as well.

Common Scenarios
Scenario 1: Joining Data From Multiple Sources

In case you need to reference data from multiple sources and want to avoid making multiple calls,
a good approach would be to add data to cache after retrieving records from an individual source.
Once the complete data is available in multiple caches, add them as a source profile in your map.
Let's refer to the example below:

Figure 146: Joining Data from Multiple Sources

285

Internal Use - Confidential

 In this example, the complete list of doctor profiles along with hospital details where each doctor
practices are first retrieved and stored in separate caches, each indexed by "Doctor ID". In the
third branch, based on the doctor ID contained in the incoming web service request, data from
each cache is joined with the source profile based on their "Doctor ID" index and mapped to create
the final output. Note that the profile used at source matches with the input data and cached
data is added at the parent element level. This creates a 'super profile' on the source end of the
map. Adding the cached data to the source profile vs. a document cache lookup function allows
repeating/looping data elements from the cached documents to be mapped naturally, such as
Hospitals in this example.

Figure 147: Mapping Data

Scenario 2: Existence Check Lookup

Once you add data to document cache, you can use this temporary repository to lookup data.
Lookup option is available as a part of parameters for decision shape, set properties, connector as
well as map function.

One popular use case is to efficiently check for the existence of records in a destination system.
Instead of making a separate external API call for each incoming record to determine if it exists in
the destination system, first make a single call to extract and cache all the records then perform

286

Internal Use - Confidential

 the individual lookups against the "local" cached copy. If the record does not exist in the cache, an
insert can be performed, otherwise an update can be performed. Note the technique of storing
the document cache lookup result in a document property so that if the record exists the ID value
in the property can be used in the "update" map instead of having to perform a second look up to
the cache. This example assumes the destination system requires an ID value to perform an
update--a common practice.

Side note: As always, there are considerations and practical limitations of this approach, such as if
there are a very large number of records in the destination system; it may be more efficient to
query individually instead.

Figure 148: Lookup

287

Internal Use - Confidential

 Figure 149: Set Propeties > Cache Lookup

Figure 150: Parameter Values

288

Internal Use - Confidential

 Get an example of this scenario from the Process Library here

Scenario 3: Use as a Temporary "Array"

A single document cache can be used to store multiple document properties (as long as the
document is of the same structure). This can be later retrieved and used as appropriate.

The below example shows use of a single document cache for storing different types of errors.
These errors are aggregated at the end of the process and returned to the user.

Because document properties are stored as a part of document cache, we will be retrieving
try/catch message and cleanse result message from branch 1 and 2 respectively. Before writing to
cache in each branch, a dummy static dynamic document property is set with a static value (which
is the same for branch 1 and 2), which is used as an index in the cache.

Figure 151: Use Temporary Array

289

Internal Use - Confidential

 In branch 3, the same static value which was set in the dummy dynamic document property before
writing to cache is used for retrieval of the complete data (both cached document as well as
document properties) from the cache.

Figure 152: Load From Cache Shape

The document property values which we are interested in are the try-catch message and the
cleanse result message. We will use a notify shape for this purpose.

290

Internal Use - Confidential

 Figure 153: Notify shape

Scenario 4: Multiple Lookups

Documents can be added to cache and looked up multiple times in different stages of the process,
based on particular input to avoid making the external call multiple times. Let us take a case
where we have a consolidated list of all orders received from a website that must be sent to both

291

Internal Use - Confidential

 the Order and Invoice Departments. However, these departments both need additional Product
information that is not present in the source data from the website. To avoid having to make the
external queries to the product database multiple times (once for orders and once for invoices),
we can instead make one call to retrieve all the product details, add them to a document cache,
and then perform the lookup against the local cache in the maps.

Figure 154: Data Flow

In this process, Branch 1 extracts the complete information of all possible combinations of product
categories and departments. (In the Database Operation, be sure to set Batch Count=1 so each
record is returned as a separate document and cached individually. This will give you the desired
level of granularity while retrieving data from the cache.

Branch 2 and 3 pass the required data to the Order and Invoice departments. There is a need to
look up the value of Product Department based on the category of the ordered item. Document
Cache Lookup function is used within the maps in each branch for this purpose.

292

Internal Use - Confidential

 Figure 155: Document Cache Lookup in the Map

Lookup functions work perfectly well for multiple occurrences of line items (unbounded
elements). One order can have multiple items and for each item ordered, the lookup function will
be able to pick up the corresponding department value based on the item's category.

Scenario 5: Handling MIME Attachments in Certain Connectors

Some connectors support receiving and/or sending MIME attachments by leveraging the document
cache component.

• How to Receive and Send MIME SOAP Attachments for Web Services
• How to Use the Mail (IMAP) Connector

Additional Considerations
Tips for Using the Document Cache Lookup Function

Using a document cache lookup function in your map:

• Can be used only with document caches that are based on a profile type and profile (in
other words, the document cache cannot be configured with Profile = "None").
• The lookup function should only return one document, otherwise it will generate an error
message.

293

Internal Use - Confidential

 • If your requirements are to return more than one document or repeating values, add the
cached data to the map's source profile instead.
• If any key values in the source document are null or empty, the lookup will fail, and you
will receive an error.

4.4.15 Use of a Sub-Process to link Document Streams

• Scenario: You have identically formatted data being queried from 2 separate
databases. You’d like to query both databases but combine the 2 sets of
documents into a single document stream for downstream processing.
• Solution: Call a sub-process. Within the sub, branch, then execute each DB
query independently, then have both sets of results flow to the same “Return
Documents” shape. The documents from all DB queries will “wait” at the
Return Documents shape and return to the calling process in a single group.

4.4.16 Flow Control

• In general, limit use of flow control set to “run documents individually” unless
used with threading. If used without threading, use within a sub-process if
possible to limit performance impacts.
• Flow Control, Boomi User Guide

4.4.17 File Naming When Using File-based Connectors

• Whenever sending files to an endpoint, set file names and extensions as

appropriate, possibly including a date. Avoid sending files using Boomi default
“.dat” filenames.
• File-based connectors include Disk, FTP, SFTP connectors and some branded
connectors.

4.4.18 Design for Reuse

Your business sponsor and development team will thank you!

Design the integration solution with consideration to reuse as much as possible. It will lower
the implementation cost of the next solution that takes advantage of an already built process
or API.

294

Internal Use - Confidential

 The development team will also benefit in reusing common processes for logging, alerting and
error handling and hence less time is needed for implementing the integration process as they
will have access to these utilities. Also, in the future if one of these common components are
enhanced, the benefits are automatically inherited by the integration process with little
overhead (other than regression testing).

4.4.19 Test Mode Limits

When you exceed the limit in test mode in either document count or size you can deploy the
process and run it with a Return Documents shape. This will allow you to see in the process
reporting exactly how the data in your process is at that point in time. This is one way you
can get the level of detail available in test mode when you either have more than 100
documents or exceed 10MB in data.

4.4.20 Component Locking

When component locking is enabled on an account, it prevents two users from simultaneously
editing a component. This feature doesn’t allow a user to permanently lock a component. The
locks are retained for the session or until the user saves or closes the component. Users with
the Administrator role can steal a lock from other users if needed.

AtomSphere displays a warning when a user attempts to lock or save an outdated component

4.5 Process Options

You can set process options in the Options dialog when you create or edit a process.

4.5.1 Process Modes

Process Mode is set to General by default. If you are not using Services Enablement,
your process mode cannot be changed. This list box is visible only if the Services
Enablement feature is enabled in your account.

• General — The default process mode for all new processes.

• Low Latency — Use this process mode to improve performance for short-lived
processes (where total execution time is generally expected to be less than 30

295

Internal Use - Confidential

 seconds). In this process mode, the Start Shape dialog is set to use the Connector
option; the Connector field can be set to the following:

o Atom Queue

o Flow Services Server

o JMS

o MDM Listener

o MLLP Server

o Salesforce Platform Events

o Web Services Server

When the process mode is set to Low Latency:

o The other fields, where Action is set to Listen, and Atom Web Server
manages the connection settings, cannot be changed.

o Some functionality is disabled. See the Low Latency Processes topic for
more information.

A subprocess called by a low latency process always uses Low Latency mode. In other
words, the subprocess settings in the Process Options dialog are ignored and it uses
the same Process Options settings as the top-level low latency process that calls it.

Note: A process that uses the Web Services Server connector in the Start shape and
that existed prior to the 9 January 2013 release will continue to run as it did prior to
the release date. A process that uses the JMS connector in the Start shape and
that existed prior to the July 2014 release will continue to run as it did prior to the
release date. A process that uses the MLLP Server Connector in the Start Shape and
that existed prior to 3 June 2015 will continue to run as it did prior to the release
date. You will not see a process mode assigned to this process in the Start Shape

296

Internal Use - Confidential

 dialog. If you open the Process Options dialog and the process mode has never been
set, it will default to Low Latency and you will see a notification message. If you want
to set a process mode for the process, select a mode and click OK in this dialog. If you
want to leave it as is, click Cancel in this dialog.

4.5.2 Options Within Each Mode

Allow Simultaneous Executions

This check box is always available and can be turned on or off. If the Process Mode
field is set to General, this check box is off by default. If it is set to Low Latency, this
check box is on by default.

• If selected, more than one instance of this process can run at the same time on a
given Atom.

• If cleared, only one instance can run at a time. This is often beneficial when you are
processing large amounts of data (to conserve system resources) or performing time-
or state-sensitive synchronizations.

• When a process calls a subprocess, the subprocess always executes immediately

regardless of how the Allow Simultaneous Executions check box is set on the
subprocess or its parent process.

Note: This option is not recommended for use with processes that use persisted
process properties.
Auto Capture Errors/Warnings to Local Log?
If selected, logs for the process are stored in the local Atom's execution history
directory. The default location is a zip file in
the <atom_installation_directory>/execution/history/<process_execution_date> folder
. The zip file contains the log files for the process.
This check box is always available and can be selection or cleared. It is cleared by
default.

297

Internal Use - Confidential

 Capture Run Dates
Records the last run date and the last successful run date for the process. There is
significant performance degradation if this option is enabled for web server, AS2, or
event-based processes.

This check box is always available and can be turned on or off. If the Process Mode
field is set to General, this check box is on by default. If the process mode is set to
Low Latency, this check box is off by default.

Purge Data Immediately

If this setting is on, processed documents and temporary data are purged immediately
after each execution of the process.

There is a like-named setting at the Atom level.

• If the Atom-level setting is on, purging occurs for any process execution on that Atom,
regardless of the individual process option settings.

• If the Atom-level setting is off, purging occurs only for executions of processes for
which the process option setting is on.

For a process running as a subprocess, the process option setting for the parent
process takes precedence.

This check box is always available and can be turned on or off. It is off by default.

Only Generate Process Log on Error

If selected, the process log is created only if the process encounters an error. We
recommend that you turn on this check box for web server or event-based processes to
improve performance.

This check box is available only when the Process Mode field is set to Low Latency.
The check box is on by default for new processes and off by default for existing
processes. It can be turned on or off.

298

Internal Use - Confidential

 Figure 156: Process Options

299

Internal Use - Confidential

 Figure 157: Process Options

When you configure the Start shape for a process, you might see a
message containing recommendations about how to set the Allow
Simultaneous Executions and Capture Run Dates check boxes. Following
these recommendations should improve process performance. For
example:

• If you select a regular connector or No Data and Allow Simultaneous

Executions is on, it is recommended that you turn it off.

• If you select Passthrough and Capture Run Dates is on, it is

recommended that you turn it off.

• If you select Trading Partner or any type of listener (Action = Listen) and
Capture Run Dates is on, it is recommended that you turn it off.

300

Internal Use - Confidential

 • If you select Trading Partner or any type of listener (Action = Listen) and
Allow Simultaneous Executions is off, it is recommended that you turn it
on.

To follow the recommendation, click the Make Change(s) link within the
message.

4.6 Connector Guides

Basic Use Case – Writing to S3

Amazon S3 Connection

The Amazon S3 connection uses Amazon access keys, which consist of an access key ID and a
secret access key. You can create these keys using the AWS Management Console via
the security credentials page.

Figure 158: Amazon S3 Connection

301

Internal Use - Confidential

 Note that both the Amazon S3 Bucket name and AWS Region are configured at the connection
level. If you have multiple accounts or Buckets, or multiple Regions, you’ll use a separate
connection for each and configure the Amazon S3 connection accordingly.

Although there is a 1:1 relationship between the S3 Bucket and the connection (i.e., you need
a different connection for each different Bucket), you can create and write to
multiple folders within a Bucket using the same single connection. So – to read or write to
multiple Buckets you need multiple connections, but to read or write to multiple
folders within the same Bucket you only need one connection.

Process Flow

The Amazon S3 operations include Get, Create, and Delete. Here we’ll test the S3 connector
by creating a file in an S3 Bucket. Again, in this example I’m creating and uploading a CSV
data file to the S3 service, and in a later post I’ll review how I can then COPY that file into
Amazon Redshift.

Figure 159: Connection Steps

1) Get source data/file. I’ve configured a Salesforce connector to query Accounts (I limited
the operation to 100 records to run in Test Mode without hitting the Test Mode data limits).
I've then mapped the results to a flat file profile (CSV) and used a Data Process shape to
combine the documents into a single flat file document.

2) Set file properties. The S3 Operation supports three properties which can be used to set
the file name and folder name within S3:

302

Internal Use - Confidential

 Parameter Name Optional Default Value Description
File Key Yes None Full key for Amazon S3 object.
File Name Yes UUID File name for the target document.
Folder Yes None Directory path for the target document
Table 37: Parameters

Note: When you set the S3_KEY parameter the other parameters are omitted. If the S3_KEY is
blank, the Amazon Key is created by concatenating the S3_FILENAME and S3_FOLDER.

4.6.1.1.2.1 Directory structure - File Name vs Folder Names

In S3, "folders" are really just part of the file name. From the S3 docs:

In Amazon S3, buckets and objects are the primary resources, where objects are stored in
buckets. Amazon S3 has a flat structure with no hierarchy like you would see in a typical file
system. However, for the sake of organizational simplicity, the Amazon S3 console supports
the folder concept as a means of grouping objects. Amazon S3 does this by using key name
prefixes for objects.

For example, you can create a folder in the console called photos, and store an object
called myphoto.jpg in it. The object is then stored with the key name photos/myphoto.jpg,
where photos/ is the prefix.

Here are two more examples:

• If you have three objects in your bucket—logs/date1.txt, logs/date2.txt,

and logs/date3.txt—the console will show a folder named logs. If you open the
folder in the console, you will see three objects:date1.txt, date2.txt,
and date3.txt.
• If you have an object named photos/2013/example.jpg, the console will show
you a folder named photoscontaining the folder 2013 and the
object example.jpg.

For convenience, the Boomi S3 connector provides Properties for both FIle Name and Folder
Name, the S3 Connector then concatenates these values. In practice, this means that you

303

Internal Use - Confidential

 have the option to utilize both Properties, or to build up a directory structure and file name
using just the File Name property.

Here I’ve set the properties to produce a file named “S3_Connector_test.csv”:

Figure 160: Choose Property

304

Internal Use - Confidential

 Figure 161: Set Property

Figure 162: File in Bucket

By adding a second property, I can create the file within a folder in my Bucket:

305

Internal Use - Confidential

 Figure 163: Set Additional Property

Figure 164: File Path

306

Internal Use - Confidential

 However, I can also achieve the same result by setting both the folder name and file name
with the "File Name" property, either as a parameter or, in this case, using multiple
parameters:

Figure 165: Set Additional Property

Later in this post, I'll use this technique to dynamically build a directory structure with
separate folders for Year, Month, Day..

3) Configuring the S3 operation. The operation must be imported--this is an easy step to

miss. The Import action initializes the operation and generates a response XML profile. Note
there is no request profile for the connector. This means that you can send data in any format
to S3 (e.g. CSV, XML, binary, etc).

Before import:

307

Internal Use - Confidential

 Figure 166: S3 Create

After import:

Figure 167: S3 Create after Import

4) S3 response. The S3 service provides a response that includes the file name, bucket name,
and file URL, which may be of use within your integration process.

<s3UploadResult>

<key>S3_Connector_test.csv</key>

308

Internal Use - Confidential

 <bucketName>boomi-se-demo</bucketName>

<uploadedFileUrl>http://boomi-se-demo.s3.amazonaws.com/S3_Connector_test.csv
</uploadedFileUrl>

</s3UploadResult>

I’ll use the S3 response to add an email alert to be sent when the file upload is complete:

Figure 168: S3 Process

I’ve configured a Message step to create a message based on the S3 response:

309

Internal Use - Confidential

 Figure 169: Message Shape

Which produces the following email:

Figure 170: Email Produced

310

Internal Use - Confidential

 Results

Success! Checking in S3, we can see the uploaded file in the bucket folder I specified above:

Figure 171: Uploaded File in Bucket

The file is now visible and available within S3. Again, the premise here is that writing to S3
would be the first step in loading data into Amazon Redshift, which is covered in some detail
in this post.

Complex use case

So, that's all well and good. We've loaded a file into a directory. However, in practice your
use cases are bound to be more complex, and so here we'll add a few more details to our use
case, and then figure out how to address those details with Boomi.

In real life, the use cases that I'm seeing typically revolve around loading data into a data
lake, and read something like this: "we need to retrieve all recently modified Accounts from
Salesforce, and write them to S3 as json files. We need to have the process run every 5
minutes, and the files need to be written to a different directory for every year, month, and
day, and the files need to be timestamped. For example, we need to see
"[bucket]/LOAD/Accounts/2017/02/07/accounts_20170207144314.json". We need to run this
process ever five minutes, and then later we need to create a process that will retrieve all
files from a particular directory for processing"

311

Internal Use - Confidential

 Based on this use case, we'll examine the following items:

• Dynamically creating a directory structure

• Retrieving files from S3 directories
• Decoding the contents of S3 files (S3 file contents are Base64 encoded by default)

Folder or Directory Structure

Based on the use case above, we need to figure out how to create S3 folders, and how to
dynamically write to the appropriate folder.

First, I've created a basic process which will query recently modified Salesforce accounts, and
map the results to a JSON file.

Figure 172: Basic Process

Next, I need to leverage the S3 properties to build my folder structure and file name.

This is a typical Boomi task of using the Set Properties step to concatenate a series of static
and dynamic parameters. Again, with the S3 Connector you have the option of using the
Folder Name and File Name Properties, or you can simply create or specify the folder as part
of the File Name Property. Here I'll take the later approach, using the File Name Property to
build up the folder structure and file name. As you can see, I'll begin with a static value - my
top level "LOAD" folder, and then use a Current Date parameter with the appropriate mask to
create folders - being careful to add "/" between each folder. Finally, I build up a filename in
the same manner.

312

Internal Use - Confidential

 Figure 173: Set Property

Resulting in the desired folder structure and file name.

Figure 174: File Path

313

Internal Use - Confidential

 Next, we need to figure out how to retrieve files back from S3.

Retrieving files from S3

The Boomi S3 connector provides 3 operations for getting objects: Get S3 Object, Get S3
Binary Object, and Get Object Listing. Get Object and Get Binary Object expect an Object ID
as in input parameter. In other words, if you know exactly what you want to Get, you can use
these operations. However, in many cases you'll need to first get an Object Listing, then use
the results (list) to retrieve specific objects.

4.6.1.2.2.1 Working with S3 Object Listing

Per the documentation, "Get Listing retrieves a list of files and folders in the specified
bucket, but not the contents of a file that include key, folderName, filename, isDirectory,
lastModified, size, and bucketName." So, we'll get a directory listing, and then use the results
to retrieve specific files. Similar to Get Object and Get Binary Object, Get Object Listing
expects an ID as an input parameter.You can use a "*" to get a listing of everything in your
bucket, or you can provide a folder path to get a directory listing for a particular folder. For
instance, based on my example above I can provide an ID of "LOAD/2017/02/" to retrieve a
directory listing of all files in the "02" directory.

Figure 175: Retrieve Directory Listing in "02"

314

Internal Use - Confidential

 The return:

<s3ObjectListing>
<s3ObjectSummary>
<key>LOAD/2017/02/07/accounts_20170207152832.json</key>
<fileName>accounts_20170207152832.json</fileName>
<isDirectory>false</isDirectory>
<lastModified>2017-02-07T15:28:36-08:00</lastModified>
<size>71</size>
<bucketName>boomi-se-demo</bucketName>
</s3ObjectSummary>
<s3ObjectSummary>
<key>LOAD/2017/02/08/accounts_20170208085443.json</key>
<fileName>accounts_20170208085443.json</fileName>
<isDirectory>false</isDirectory>
<lastModified>2017-02-08T08:54:45-08:00</lastModified>
<size>71</size>
<bucketName>boomi-se-demo</bucketName>
</s3ObjectSummary>
<s3ObjectSummary>
<key>LOAD/2017/02/08/accounts_20170208085444.json</key>
<fileName>accounts_20170208085444.json</fileName>
<isDirectory>false</isDirectory>
<lastModified>2017-02-08T08:54:45-08:00</lastModified>
<size>70</size>
<bucketName>boomi-se-demo</bucketName>
</s3ObjectSummary>
</s3ObjectListing>

What if I need to create a scheduled process that will always retrieve today's (or yesterday')
files? Similar to creating the folder structure, one can use a Set Properties step to build up a
property consisting of static text and relative dates, and then use that property as the input
parameter for the Object Listing operation.

315

Internal Use - Confidential

 Typically, the next step is to add a Data Process/Split Documents step, splitting on
<S3ObjectSummary>, so that we can route or otherwise act on a specific file. Then, the <key>
element can be used as the input to a Get Object or Get Binary Object operation.

4.6.1.2.2.2 Working with S3 Get Object

Continuing with this use case, I've built out the following process:

Figure 176: S3 GET Process

As above, I'm returning an object listing, and then splitting the return into separate
documents. I'm then passing in the <key> element to query for specific documents using the
Get S3 Object operation

Figure 177: Connector Shape Parameters

316

Internal Use - Confidential

 However, upon executing my process, the results are not quite what I expected:

<s3Object>
<key>LOAD/2017/02/07/accounts_20170207152832.json</key>
<folderName>LOAD/2017/02/07</folderName>
<fileName>accounts_20170207152832.json</fileName>

<objectContent>Ww0KICAiTGFtIFJlc2VhcmNoIiwNCiAgIjAwMTBNMDAwMDFRT3lZV1FBMSIsDQogI
CJQcm9zcGVjdCIsDQogIG51bGwNCl0=</objectContent>
</s3Object>

The key, folderName, and fileName are returned as expected. But, the objectContent has
been Base64 encoded by S3. If you want to use the objectContent in your Boomi process this
seems to present something of a challenge: some elements are returned un-encoded while
the objectContent element is encoded. If the entire return was encoded, we could just use a
Data Process step to decode. But how can we decode a single element?

The simplest solution is to capture whichever un-encoded elements you'll need in Dynamic
Document Properties, then use a Message step to separate out the objectContent, then to
decode with a data process step. The solution looks like this:

Figure 178: Process Steps

1. As above, I'm using my Object Listing to enumerate the files in my bucket, then using
Get Object to get the specific files, and finally capturing the key and fileName, to be
used later in my process flow, as Dynamic Document Properties.

317

Internal Use - Confidential

 2. Here I'm using a Message step to capture the Base64 encoded objectContent:

Figure 179: Message Shape

3. Next, I can simply use the Data Process step's built in Base64 decode to decode the
objectContent. I now have a clear-text document which I can use within my Boomi
process - for instance, I can send this into a Map step - and I have my key and fileName
captured in DDP's in case I need those as well.

Conclusion

That wraps up the basics of working with the S3 Connector. Overall, it's pretty standard
Boomi process development, though I find a few aspect - such as the Base64 encoded
objectContent - to be a little bit less than intuitive. As always, please let me know if this
article was helpful for you, and if you have any use cases that are not addressed in this post
feel free to comment and I'll do my best to respond in a timely manner.

318

Internal Use - Confidential

 4.6.2 Database Integration

The Database Connector allows for seamless integration with a variety of on-premise and
cloud database providers. This article is a collection of best practices, configuration details,
FAQs, and common errors for database integration.

User Guide Articles

Here are some links to our User Guide and other reference material, which you may find
useful when using and configuring the Database connector.

• Database Connector
• Database Connection
• Database Profile
• Database Operation
• To Create a Custom Database Connection

Process Library Examples

• Database Basics Examples

Common Scenarios
Scenario 1: Connecting to a Database

The Database connection reference guide page lists the available fields and their usage in the
Connection component. In this example, we will be connecting to an Oracle database.

Here is a link to Oracle’s reference guide. Table 8-1 lists standard database connection
properties. Typically you will need databaseName, user, password, portNumber and
serverName at a minimum, however certain configurations require additional information.

Scenario 2: Querying Records

Setting the Grouping Options in the Database Read Operation is critical for most database
integrations. There are no database record splitting options available within the Data Process

319

Internal Use - Confidential

 step mid-process, so you must define the operation's Link Element and/or Batch Count to
group records by a matching field or record count.

Example:

Database Read Profile Statement:

SELECT ORDERNUM, LINENUMBER, PRODUCTDATA FROM ORDERS

Return Data (snippet):

12345|^|1|^|ROUTERS
12345|^|2|^|HUBS
12346|^|1|^|NETWORK CARDS

If you want to map individual logical orders to a destination system, you should batch the
orders in the Database Read Operation

Configuration:

Link Element = ORDERNUM (Element definition from Profile)

Batch Count = 1 (Meaning 1 unique ORDERNUM per batch)

Results:

Document 1:

12345|^|1|^|ROUTERS
12345|^|2|^|HUBS

Document 2:

12346|^|1|^|NETWORK CARDS

320

Internal Use - Confidential

 Setting the Max Rows option is helpful when you want to limit the amount of SQL records
entering the Process. If you are building a process that is migrating a large data set to
another system, you will ultimately want to prevent the same records from being sent
repeatedly during each Process execution.

In order to prevent this, you can try one of the following options:

• Add a final branch into a SQL Type Program Command step at the end of the Process
flow to update each record based on its record ID
• Add an SQL Update map function into one of the main Data Maps to perform the
update per record
• Add a new map and Database Write Connector that builds and executes the SQL
update

Example Program Command Script:

UPDATE ORDERS SET STATUS = 'PROCESSED' WHERE ORDERNUM = ?

Example Parameter:

Profile Element > Database > Orders Select Profile > OrderNum Element

Scenario 3: Sending Records and using the Commit Options

Setting the Commit Options in the Database Write operation is helpful when wanting to
finalize the execution of SQL statements in groups or by Data Sets for multi-table
relationships such as Header/Detail or Parent/Child.

Example:

Operation Configuration:

Commit Option: Commit by Profile

Batch Count: 1

321

Internal Use - Confidential

 The goal is to commit each unique order/orderdetail instance uniquely, in order to protect
against connection failures mid-process. By using the commit options, instead of the entire
order insert failing, the main order record may be committed, and a portion of the detail
records.

Scenario 4: Calling a Stored Procedure

The best practice method for calling a stored procedure is dependent up how the stored
procedure is being called, by using either a map function or a database connector:

1. The stored procedure can be called from within a map function by selecting the
Lookup type and then select SQL Lookup and Stored Procedure.
2. The stored procedure can also be called from a Database Connector by selecting it as
the Statement type in the Database Profile component.

The key difference between these two methods is the database connector supports stored
procedures but does not support output parameters, while the Map Function SQL Lookup
does. The database connector requires the response to be provided via a returned result set.

Therefore, if the stored procedure has both input parameters and output parameters, it will
be necessary to use the Map Function Lookup SQL stored procedure call to handle the data as
output parameters. If the stored procedure has input parameters, no output parameters and
returns a result set; you can use the database connector to execute the stored procedure
using a DB read profile and with the fields defined to match the elements returned in the
result set.

As described in this reference guide article (Database Profile), the following standard data
type elements are supported by the database profile, including for result sets:

• Character
• Number
• Date/Time
• CLOB
• BLOB

322

Internal Use - Confidential

 (Local database driver-specific data types may not supported.)

Prior to the May 2014 release, the Oracle type CURSOR was not supported.

Common Database Connection Configurations

Database Integration Guide: Common Database Connection Configurations

Frequently Asked Questions

Database Integration Guide: FAQ 1
Database Integration Guide: FAQ 2

Common Errors
Database Integration Guide: Common Errors

4.6.3 NetSuite Integration

Overview

The NetSuite connection represents a single NetSuite account, including login credentials. If
you have multiple accounts or sandbox instances, you need to use a separate connection for
each and configure the URL accordingly. You can pair a single connection with different
NetSuite operations to perform a unique action against a NetSuite account.

The connector supports the following actions (Note: Not all actions are available for every
record type):

• Search - Returns one or more records that match zero or more filters.
• Get - Returns a single record that matches a given internal ID.
• Create - Creates a new record.
• Update - Modifies an existing record.
• Delete - Removes an existing record.

323

Internal Use - Confidential

 User Guide Articles

Here are some links to our Reference Guide, which you may find useful when using and
configuring the NetSuite Connector.

• NetSuite Connection
• NetSuite Connector
• NetSuite Operation

Process Library Examples

• NetSuite Basics Examples

• NetSuite RESTLet Example

Common Scenarios
Scenario 1: Query

When querying records, each record found will be returned as a separate Document and be
processed independently. If Include List Fields is turned on- sublist records are included in the
results, otherwise only the base fields are returned. For example, when searching Customers,
enable to return AddressList, ContactList, CustomFieldList.

Scenario 2: Update

Updating a specific record in NetSuite requires that you include the internal or external
NetSuite ID for that record in the update request.

Best Practices:As a best practice, if the other application has a field that can be used to
capture an external ID, populate it with the NetSuite ID so you don't have to do a lookup to
get the ID in your Process. Alternatively, use the user-specified external ID to maintain the
"other" system's ID in NetSuite. To do this, in the Map that maps from the source Profile to the
NetSuite Update Profile, use a Map Function that performs a Connector Call to NetSuite. The
Connector Call's Operation should do a Search action against the particular object type. Add a
Filter to the Operation that you can pass in the key value(s) from the source data as an Input
Parameter to limit the results to a single record. The Map Function should return the object's
internalId attribute as an Output Parameter. Map this Output Parameter to the internalId

324

Internal Use - Confidential

 attribute in the destination Profile.If a particular object does not already exist in NetSuite,
the internalId attribute in the Update Profile will be empty after the Map. If a request
without an internalId is then sent to the NetSuite Connector, it will throw an error. If this is a
possibility in your integration scenario, you should use a Decision Step after the Map to check
that the internalId is populated in each record before sending the data to the NetSuite
Connector.

Scenario 3: Upsert

In order to do an Upsert, the externalId is a mandatory and required field, and it needs to be
non-blank. To perform the upsert the use of externalId is needed in the NetSuite system to
uniquely identify the records. Otherwise, do an insert and an update separately and add the
logic in the process to check if the record exists in NetSuite based on your uniquely
identifying field.

Scenario 4: Custom Fields and Objects

One of the great strengths of NetSuite is the ability to easily create custom fields and
objects. Because of this, the NetSuite Connector connects to your NetSuite account and
browses the available interfaces in real time. Custom objects and fields are handled no
differently than standard objects and fields. If you modify an object in NetSuite after
importing it in Boomi, you will need to edit the Operation Component and go through the
import wizard again to re-import the recent changes.

Remember to reimport operation to access new custom fields.

Scenario 5: Batching Requests

To maintain satisfactory performance for all of its clients, NetSuite governs the web service
traffic sent to its data center. There are limits on the number of records per request by the
type of request (Add vs. Update vs. Search) as well as time of day (peak vs. off peak). For
example, you cannot try to add more than 100 records at once during the middle of the day,
but at night that limits grows to 200. Refer to the NetSuite Help Center, Understanding Web
Services Governance, for more information.Please refer to our reference guide at below link,
that mentions about the Maximum No. of documents that are allowed in a batch for a
particular Operation (http://help.boomi.com/atomsphere/GUID-391FB951-69DC-484C-8B1E-

325

Internal Use - Confidential

 4248CB9368BF.html). The NetSuite Connector automatically batches records to the proper
size in accordance with NetSuite's guidelines. This means your Process can send any number
of records to the NetSuite Connector and it will chunk the records automatically for you.

• The default is 0 and will include the max number of documents per batch allowed by
SuiteTalk:
• Create — 100 ,Update — 50,Upsert — 50, Delete – 100 (cannot be changed)

However, the NetSuite Connector always uses peak time maximums even during during off-
peak times as per the current design.

Other Considerations:

• The Connector does not necessarily open and close a connection per batch. It pools up
to four connections for efficiency.
• Requests are sent synchronously.
• Request batches are sent sequentially, not concurrently.
• Create a dedicated integration user and role for traceability.
• Create dedicated integration forms to insulate from end user changes, scripts,
validations and ensure needed fields are available.
• Disable client and server SuiteScript in Web Services Preferences.
• When writing to NetSuite, cache List values either ahead of time using Document
Cache or with map function (getSelectValue) Handle application responses in process
workflow using Application Status document properties.
• Use response handling and Try/Catch to handle errors and retries instead of the built-
in connector retry.

Scenario 6: To Blank out a field in NetSuite

For fields, including customFields, you cannot pass Null values or empty tags to the NetSuite
API. The proper way of deleting a value from a certain field, is to add it to the nullFieldList
(which is a separate element within the request profile). Note, you insert the name of the

326

Internal Use - Confidential

 element you wish to blank out here. In the case of customFields, you should use the NetSuite
internal ID, not the human readable name from the profile.
Reference: https://system.na1.netsuite.com/app/help/helpcenter.nl?fid=section_N3527090h
ttps://system.na1.netsuite.com/help/helpcenter/en_US/Output/Help/section_N3438152.htm
lhttps://system.na1.netsuite.com/app/help/helpcenter.nl?fid=section_N3438152

Common Errors

NetSuite Integration Guide: Common Errors

FAQ
How do I filter the date range in NetSuite Operation?

Want to select data based on a specified date range from NetSuite. NetSuite does not permit
the same element from being used multiple times in a filter/search. ie. you cannot specify
the following in the filter:

• DateElement after (date)

• DateElement before (date)

You can use a comma separated value and select the Within operator. One way to achieve
this is to create a Dynamic Process Property that consists of the First Date in the Date Range,
a comma, "," and then the second date in the date range, and pass it to the filter as follows:

• DateElement Within (Dynamic Process Property)

What is the proper Date/Time format for NetSuite?

NetSuite Date/Time XML Profile elements should be configured with the following format:
yyyy-MM-dd'T'HH:mm:ss.SSSZZ. For example: 2010-04-01T10:33:17.837-07:00

When importing, why are not all Objects available under the basic search?

The Item Object and possible other objects, are abstract objects, and are not part of the
basic search, so they won't be available in the drop down list, when you attempt to import
the profile, to create the NetSuite Operation.Instead, for basic searches, you must search on
the type of item you wish to interact with: Assembly item, etc

327

Internal Use - Confidential

 What are the Best Practices when working with the Invoice Initialize Object in
NetSuite?

If you are trying to create an invoice initialize request(based on an existing Sales Order) then
follow the required steps.Set your NS Operation Action type to EXECUTE. Here you will find a
list of NS API actions that perform various things like initialize, attach, detach, etc.Select
Invoice Initialize depends on your requirement.You will need to populate the following fields
in the request Profile:* InitializeRecord/reference/@type (e.g. "salesOrder", "customer")*
InitializeRecord/reference/internalId (internalId of the referenced record)Sample salesOrder
Invoice Initialize Request,<ns1:InitializeRecord
xmlns:ns1="urn:core_2014_1.platform.webservices.netsuite.com">
<ns1:type>invoice</ns1:type> <ns1:reference type="salesOrder"
internalId="108508"></ns1:reference></ns1:InitializeRecord>Note: The connector will
automatically populate InitializeRecord/type.Once you will pass the request then you will get
response.

How to search by external id for customer entity in NetSuite?

The NetSuite Upsert and Update operations add or update records in NetSuite without
needing to determine first whether records already exist in NetSuite. These operations
identify records by external ID and record type. If a record of the specified type with a
matching external ID exists in NetSuite, it is updated. If it does not exist, a new record is
created.If you try to create a Customer with an externalId used by a different record, the
following error appears: Record already exists, name value must be unique. In NetSuite go to
Lists > Relationships > Customers > Search > choose 'External ID (Text)'.

Why doesn't a particular NetSuite field appear in the Filter tab pull down?

You've imported a new NetSuite operation and profile, and the field appears in the profile,
but is not available, on the filter tab, in the pull down when selecting elements/fields to
query against.Not all NetSuite fields are "searchable".For example: 'shipStatus', from 'Item
Fulfillment' isn't available as a query filter. You can see the full list of searchable fields under
the 'Search Columns' using their schema
browserexample: https://system.netsuite.com/help/helpcenter/en_US/srbrowser/Browser20
14_2/script/record/itemfulfillment.html

328

Internal Use - Confidential

 Two NetSuite Operations use the same Request Profile, but after adding a new
customField, why does only one of them work properly?

The Netsuite Operation, is closely coupled with the specified Request and Response Profile. In
some situations, you can have multiple operations configured to use the same profiles (
perhaps they have different selection criteria ).After adding a new customField, you can re-
import one of the operations, and this will update your request and response profiles. Even
though the second operation appears to use the correct new profile, you must still re-import
this second profile.

How can I see the HTTP Request and Response from the NetSuite Connector?

Request and response data for NetSuite requests is captured in NetSuite under: Setup >
Integration > Web Services Usage Log (may require admin account to view this).

How to search NetSuite transactions by status?

It is a common requirement to query or extract NetSuite transactions that have one or more
specific status values, such as all pending sales orders or all closed invoices. This is possible
however identifying the exact status value(s) is not obvious because you must use a special
API syntax and not the value you see through the NetSuite user interface.To query a NetSuite
transaction by status, first create a new NetSuite Connector Operation by importing the
specific transaction object and create a filter for the <object type>|status field.
Choose anyOf as the Operator.IMPORTANT: Be sure to select the status field for the primary
object itself and not the a status field for any of the related transactions. The fields for the
primary object are listed first in the Field picklist. Then you will need to determine the
values you will need to use. The status values are special API values and are NOT the same as
those visible through the NetSuite user interface. It is also important to note that behind the
scenes the NetSuite connector is actually performing a transacti API call.The status values are
documented in the NetSuite SuiteTalk Schema Browser. Within the NetSuite Help Center,
search for "schema browser" or go
to https://system.netsuite.com/help/helpcenter/en_US/SchemaBrowser/indexv2013_1_0.ht
ml (note the version number in the URL; modify as necessary). From within the Schema
Browser, to find the list of transacti status values navigate as follows:

• Upper left: Click transactions sales.xsd.

329

Internal Use - Confidential

 • Lower left: Click Transacti.
• Center: Within Transacti (joined search) table, find basic and click
platformCommon:TransactiBasic
• Center: Within TransactiBasic table, find status and click
tranSalesTyp:TransactionStatus
• Right: Scroll down to TransactionStatus. These are the values to use.

Example values: _invoiceOpen, _purchaseOrderClosed, _salesOrderPendingFulfillmentCopy

and paste this value into the connector step parameter value for the filter.Additional notes:

• You can specify multiple status values by including them in a comma separated list.
For example: _salesOrderPendingFulfillment,_salesOrderClosed.
• The status value returned in the response data will be the "familiar" status label value,
such as "Pending Fulfillment", not the API value.

My process references the NetSuite Legacy connector, how do I convert to the

new connector?

With Version 2014 Release 1, NetSuite retired SuiteTalk 2009.2 and Earlier Endpoints. With
this change, Boomi delivered a new NetSuite connector. If you have not done so already, you
should convert your processes to use the new connector.

• NetSuite Legacy connectors may be found in several different AtomSphere shapes

including:
o Start, Connector, Map (Connector calls and Map Functions), Decision, Process
Call and Business Rules
• NetSuite Legacy profiles may be found in several different AtomSphere shapes
including:
o Connector Operation, Map, Find Changes, Set Properties, Data Process, Add to
Cache, Load From Cache, Cleanse

Friendly reminders for converting your processes

• Configure Connection URL to match the new version of NetSuite for your upgrade. For
example, https://webservices.netsuite.com/services/NetSuitePort_2013_1. The URL
defaults to the latest available version of the service. The last few numbers in the URL

330

Internal Use - Confidential

 must match the version selected in the connection. So if you use the default URL and
the default version, then you are all set. However if you change the Version field to
“2012.1” you would have to change the last part of the URL field to
/NetSuitePort_2012_1. If you are using a sandbox, your URL might look something like
this: https://webservices.sandbox.netsuite.com/services/NetSuitePort_2012_1.
• Compare the NetSuite Legacy process to new process to be certain all selected
objects, mapping and filters are identical
• Re-import profiles
• Test the processes with new connections against currently executing NetSuite (Legacy)
process results.
• Deploy the modified process
• To see where a connector or profile is used, you can search the Component Explorer.
o Click the blue arrow ( ) next to the NetSuite (legacy) Connection or NetSuite
(legacy) XML Profile component.
o Select Show Usage.

If the component is reused in folders and/or categories, you must expand the folders and
categories to see it.

• To see which Connectors are deployed, you can use the Setup Licensing tab.
o Click the “Standard” Connector Class
o Connectors Deployed in Class, Type “netsuite” is the NetSuite (Legacy)
connector, “netsuitesdk” is the NetSuite connector

How can I import NetSuite custom fields?

When importing a NetSuite Connector operation, occasionally an expected custom field does
not display in the request or response profile under the customFieldList element.Here are the
Steps to resolve why a NetSuite custom field was not imported into the request or response
profile during a Operation browse:Generally speaking for a custom field to be available via
the API it must be accessible by the NetSuite user configured for the integration and
displayed on the NetSuite Preferred Form for the given object type.To troubleshoot this issue,
verify the following.

331

Internal Use - Confidential

 • Ensure the AtomSphere NetSuite Connection is configured with the correct NetSuite
instance (sandbox vs. production) where the custom field is defined and with the
desired user.
• Ensure that NetSuite User credentials specified in the Connection component has Web
Services privileges
o Within NetSuite, go to Setup > Users/Roles > view User > Access tab, confirm
Give Access box is checked.
o Within NetSuite, go to Setup > Integration > Web Services Preferences, confirm
User is added to table of users.
• Ensure that field is set to be displayed on the given object Type’s Preferred Form. The
field must be displayed to be available via the API and when importing the operation,
the NetSuite connector will use the Preferred Form.
o To identify the Preferred Form, within NetSuite go to Setup > Customization >
Entry/Transaction Forms. Find the object Type and confirm the Preferred box
is checked to the right. Additionally Preferred Forms can be overridden by User
Role. To confirm this, edit each form for the object type, go to the Roles tab,
and confirm the Preferred box is not checked (or only checked on the desired
Form) for the integration user's Role.
o To confirm the field is displayed, within NetSuite go to Setup > Customization >
Entry/Transaction Forms. Edit the Preferred Form and go to the Fields tab. Go
through each subtab to find where the custom field is added and confirm the
Show box is checked.
• Ensure the custom field definition does not have any Role-specific Access restrictions.
o Within NetSuite, go to Setup > Customization > Entity/Item/Transaction
Body/Transaction Column Fields. Edit the field, go to the Access tab and
confirm there are no access restrictions for the integration user's Role.
• Ensure the custom field definition is applied to the correct categories. Note it is
possible for the field to be displayed correctly while working within NetSuite but not
exposed via the API. This may require some experimentation. Applying a field to more
categories than necessary is not a problem if you do not add/apply it to forms.
o Within NetSuite, go to Setup > Customization > Entity/Item/Transaction
Body/Transaction Column Fields. Edit the field, go to the Applies To tab and
confirm the field is applied to the relevant categories.

332

Internal Use - Confidential

 How do I configure the connector to use Token-Based Authentication?

Please see How to configure the NetSuite Connection with Token Based Authentication.

4.6.4 Salesforce Integration

Content

The Salesforce connector connects seamlessly to any native Salesforce application, including
Sales Cloud and Service Cloud as well any Custom Cloud or Force.com. As a Salesforce user,
you can use the Salesforce connector to integrate with any other cloud or on-premise
application.

User Guide Articles

Here are some links to our User Guide, which you may find useful when using and configuring
the Salesforce Connector.

• Salesforce connector
• Salesforce connection
• Salesforce operation
• Web service requests from Salesforce

Process Library Examples

• Salesforce Basics Examples

• Salesforce Basics: Retrieving Attachments

Common Scenarios
Scenario 1: Querying Records

When querying records from Salesforce, each record found will be returned as a separate
Document and be processed independently. By default, the Query action ignores deleted or
archived records. If you would like to return records in these states, check the "Include

333

Internal Use - Confidential

 Deleted" option in the Operation. For users familiar with the Salesforce API, this is the
queryAll call.

Using the Like - Filter Operator:

There are cases when performing a Salesforce query where you would like to select
approximate matches for a filter parameter (fuzzy search). For example, you could have
many Account records in your organization that have the term 'ABC' in the Company Name. In
a standard query case, you would use the 'Equal To' operator to match the Name to a static
value.

Equal To - Example:

Query: Select NAME from ACCOUNT where NAME = 'ABC'

Results: ABC

This straightforward query may not yield any results, so the LIKE operator paired with your
Expression may be the best strategy. In order to implement a LIKE query, you must use the %
character on either side of the Static parameter value; similar to standard SQL syntax.

Like - Example:

Query: Select NAME from ACCOUNT where NAME LIKE '%ABC%'

Parameter Definition: %ABC%

Results: ABC, ABC Company, ABC Industries, Company ABC, ABC Corp.

Scenario 2: Updating Records

Updating a specific record in Salesforce requires that you pass in the internal Salesforce ID for
that record in the update request. This value is typically an 18-character alphanumeric value
that looks like this: 0015000000MJtnHAAT. If this value does not exist in the source data you
will need to look it up from Salesforce.

334

Internal Use - Confidential

 Note that this ID is slightly different than the 15-character ID you may see in the Salesforce UI
or in the browser address bar.

As a best practice, if the other application has a field that can be used to capture an external
ID, populate it with the Salesforce ID so you don't have to do a lookup to get the ID in your
Process.

To do this, in the Map that maps from the source Profile to the Salesforce Update Profile, use
a Map Function that performs a Connector Call to Salesforce. The Connector Call's Operation
should do a Query action against the particular object type. Add a Filter to the Operation that
you can pass in the key value(s) from the source data as an Input Parameter to limit the
results to a single record. The Map Function should return the object's Id field as an Output
Parameter. Map this Output Parameter to the Id Element in the destination Profile.

If a particular record does not already exist in Salesforce, the Id Element in the Update
Profile will be empty after the Map. If a request without an Id is then sent to the Salesforce
Connector, it will throw an error. If this is a possibility in your integration scenario, you
should use a Decision Step after the Map to check that the Id is populated in each record
before sending the data to the Salesforce Connector.

Scenario 3: Upserting Records

The Upsert capability of the Salesforce API is a convenient way to do common "insert-new-or-
update-existing" integrations. Instead of having to do a lookup against Salesforce to
determine if a given record exists and then perform separate insert or update mappings and
calls accordingly, you can simply perform one map to the Upsert request and let Salesforce
determine whether it needs to do an insert or update.

Upserts can be used with standard or custom objects that have a custom "External ID" field
configured. This External ID field should represent the primary key of the source system or
some other uniquely identifying value. This will be helpful when integrating in the opposite
direction. If you don't currently have an External ID field, it is recommended you identify one
or create a new custom field to use specifically use for the integration. The Import Wizard

335

Internal Use - Confidential

 within the Salesforce Operation retrieves the list of designated External ID fields for each
object type for you to choose. You must select an External ID for the object.

If there is not a single field in the source data to use as an External ID, see if you can uniquely
identify a record by the combination of two or more fields. If so, use a Map Function to
concatenate them together and map to the External ID field.

Depending on an object's relationships as defined in Salesforce, you may have the ability to
upsert or at least reference related records by their External IDs as well. For example, you
can upsert an Opportunity based on an Opportunity's External ID (e.g. "Order Number") and
associate it with an Account record by specifying the Account's External ID (e.g. "Legacy
Customer Number").

To enable this, in the Operation configuration select the object and then select a Reference
Field. Check to box to "Use External Id for Reference" and select the appropriate Object Type
and Ref External Id. Then in your map, you can map some value from your source data and
avoid making a Connector call to retrieve the Salesforce internal ID.

Not all objects support the Upsert action. Refer to the Salesforce API Guide for a complete
list of supported actions per object.

Using Reference Fields:

Reference fields in Salesforce refer to the object fields that can be used to attach a record to
a parent or associated object. These reference fields are generally available in the Send
Request XML by their name (Ex. AccountId, OwnerId, etc). By default, the request input
expects the object's internal ID; however, this ID may not be readily available based on the
source data. The Upsert action allows you specify other External ID fields to use in place of
the default internal ID. This can save you the need of performing a Salesforce query
(Connector Call lookup) to find the internalID based on another value such as Name from your
source data.

336

Internal Use - Confidential

 Scenario 4: Deleting Records

A few things to keep in mind when deleting records.

You must always supply the Salesforce internal Id value. External IDs can't be used.

Known Issue (BOOMI-8454: Salesforce Connector throws an exception after successful delete
when multiple IDs are used): When wishing to delete multiple records, the request document
sent to the Salesforce connector must contain only one Id per document. (This must be done
even though the "Id" element in the request profile is configured as Max Occurs=Unbounded by
default.) If multiple Ids are present in a single document, the records will actually be deleted
in Salesforce, however the connector will error with the "Number of results does not match
number of SObjects" message when processing the response. To avoid this, use a Data Process
shape to split the request data. This can be done before the Map shape to the Salesforce
delete profile, or afterward (splitting the Delete profile XML on
the OBJECT/DeleteIds/Id element.

• This applies whether the operation is configured to Use Bulk API or not.
• Note: Behind the scenes the Salesforce connector DOES batch the documents in the
actual request XML sent to Salesforce. It does not make a separate for each document.

Scenario 5: High Volume API Best Practices

When developing Processes that send/receive high volumes of data to and from Salesforce,
refer to these design considerations to ensure that you are optimizing processing efficiency
and not exceeding Salesforce API usage limits.

• Refer to the Salesforce - Web Services API Developer's Guide for API usage limits for
your Salesforce edition
o Salesforce Developers - Section: Using the API with Salesforce Features >
Implementation Considerations > Monitoring API Traffic
• Install and deploy Processes to a Local Atom and consider increasing Atom Memory
• Review AtomSphere - High Volume Troubleshooting documentation for general data
concerns

337

Internal Use - Confidential

 Get Actions:

• Use Filters - This will limit the number of inbound records based on key field criteria
• Query multiple objects in one request - In the Query Operation Import Wizard, select
associated Parent and/or Child objects (eg. Contact - Parent: Account) if you need to
reference this data in your mappings. This will prevent the need for Connector Calls
(API web service requests) later in the Process for each individual contact record.
• Query by Last Modified Date - If you are building a synchronization Process that is
continually passing updates, add a LastModifiedDate filter for the object in the
Operation so you can set a time window in which you would like to grab modified
data. This will prevent the need to run entire object data sets through for each
Process execution.
• Set a Query Limit - Maximize the number of records you would like returned in one
request by setting this number in the Operation. You can then update the retrieved
records in Salesforce (eg. Contact > Custom Retrieved_Status field) at the end of your
Process flow and prevent the same records from being re-processed in the future by
filtering on the custom field.

Send Actions:

• Configure the Batch Count - The outbound Salesforce connector operation is

automatically configured to batch the XML requests into 1 group of 200 documents.
This is the maximum amount allowed in one call by the SFDC API. This will help
considerably with API usage statistics.
• Limit the inbound data - Depending on your source for the outbound Salesforce
integration, you will want to consider minimizing the number of records processed
during a single execution. In a Database integration for example, you should consider
only querying a maximum of 1,000 - 10,000 records at a time.
• Use the Upsert Action - The Upsert action will perform the logic to either insert or
update a Salesforce record automatically. This will prevent the need for: 1) A
preliminary Salesforce lookup to see if the record exists. 2) A unique Salesforce Insert
operation.3) A unique Salesforce Update operation.

338

Internal Use - Confidential

 Scenario 6: Understanding Custom Fields and Objects

One of the great strengths of Salesforce is the ability to easily create custom fields and
objects. Because of this, the Boomi Salesforce Connector connects to your Salesforce
organization and browses the available interfaces in real time. Custom objects and fields are
handled no differently than standard objects and fields. Custom objects/fields can be
identified by the __c suffix. If you don't see your custom object or field when importing an
object, make sure the security permissions for that object/field allow at least read-only
access to the security profile assigned to the user name you're using to connect. If you modify
an object in Salesforce after importing it in Boomi, you will need to edit the Operation
Component and go through the import wizard again to re-import the recent changes.

Scenario 7: Understanding the Salesforce Date Format

Salesforce Date/Time XML Profile elements should be configured with the following format:
yyyy-MM-dd'T'HH:mm:ssZ

Scenario 8: How to use the OR logical operator between two objects in Salesforce

Question:

It appears that the AND logical operator works between two objects, ie, parent to grandchild,
but the OR logical operator does not seem to work. Is it available? How does it work? Are
there any constraints/limitations?

Answer:

The SOQL query is automatically generated based on the configuration of the filters.

Currently the filters do not allow expressions fields from different objects within the same
logical sub-group.

339

Internal Use - Confidential

 As a workaround, you may need to implement two separate connectors, one that reads from
the parent object and then another that reads from the other object. Then use a Decision
shape to filter the data.

Based on Salesforce
documentation: http://www.salesforce.com/us/developer/docs/api/Content/sforce_api_call
s_soql_relationships.htm#i1422304,

It says "Any query (including subqueries) can include a WHERE clause, which applies to the
object in the FROM clause of the current query. These clauses can filter on any object in the
current scope (reachable from the root element of the query), via the parent relationships."

Scenario 9: How to Retrieve Attachments in Salesforce (Full Example)

Please visit the following link to review this scenario: How to Retrieve Attachments in
Salesforce (Full Example).

Scenario 10: How to use the Batch Results Option on SFDC Query Operations

Please visit the following link to review this scenario: How to use the Batch Results Option on
SFDC Query Operations.

Common Errors

Salesforce Integration Guide: Common Errors

340

Internal Use - Confidential

 FAQ
How do I generate a security token in Salesforce developer account?

In order to access Salesforce via Boomi AtomSphere, you must replace your current password
with a combination of your password and a security token like this:

MypasswordMytoken

If and when you change your password, you will need to request a new security token.

To request a security token:

1. Log into your Salesforce account as the user for whom you wish to reset the token.
2. View your User Profile > Settings > My Personal Information, Reset My Security Token.
Click Reset Security Token button to triggers an email containing the new security
token.
3. Carefully copy and paste (do not copy any trailing spaces), append to your Salesforce
password, and paste into the Salesforce connection's Password field.

Is there a way to monitor the number of API calls that are made while accessing an
object from Salesforce?

Per the Salesforce API guidance (which is accessible via link from the Boomi guidance on
Salesforce connector), the Salesforce query result object contains up to 500 rows of data by
default. If the query results exceed 500 rows, then the client application uses the
queryMore() call and a server-side cursor to retrieve additional rows in 500-row chunks. You
can increase the default size up to 2,000 in the QueryOptions header, as described in
Changing the Batch Size in Queries. For more guidance, you can refer to the Salesforce API
Guide.

The Boomi guidance at the following link shows the standard Get fields which include a field
for batching results and setting the batch count: Salesforce operation.

341

Internal Use - Confidential

 So, if you set the batch count higher, you can figure out how many API calls will be made, by
dividing the number of records (e.g. 1.6 million) by the batch count setting.

See also the Salesforce developer guide for how to monitor API traffic: Developer Guide -
Section: Using the API with Salesforce Features > Implementation Considerations > Monitoring
API Traffic: https://developer.salesforce.com/docs/atlas.en-
us.api.meta/api/implementation_considerations.htm

The import did not allow for the selection of the child object. Can a SFDC upsert
can be performed on a parent and child objects in the same Salesforce connector
operation?

Per Salesforce’s API, “you can process records for one more than object type in an create() or
update() call, but all records must have the same object type in an upsert() call.”

Therefore a separate upsert operation must be implemented for each object. Therefore, it’s
not possible to do an upsert on 2 different objects (even if they are parent-child) in the same
connector operation.

https://developer.salesforce.com/docs/atlas.en-
us.api.meta/api/sforce_api_calls_upsert.htm

What is the filterValue that I should specify to fetch the records whose EndTime is
Not Null in Salesforce?

In the Query filter section of Object Event, create a filter criteria similar to:

Filter Name: EndTime

Field : EndDateTime

Operator : Not Equal To

Then in the Connector Parameter section, specified the following:

342

Internal Use - Confidential

 Input : EndTime

Type : Static

Static Value : 'null'

Why am I Unable to see Action or Object picklist after connecting to SF sandbox?

Change the connection URL from:

https://test.salesforce.com/services/Soap/u/31.0

to a previous version:

https://test.salesforce.com/services/Soap/u/30.0

Save the connection, re-import or re-create the object, until you see the Action or Object in
the pick list.

Why can I not see a particular field that I know exists in Salesforce?

Verify that the user configured in the connector has the appropriate permission to see that
field.

Also, please check what version of the Salesforce API you are using. Is there a more current
version (44.0 vs. 38.0)?

If so, modify the URL in the Salesforce connection with the new API version. For
example: https://www.salesforce.com/services/Soap/u/44.0

After saving this change and re-importing the profile, check the object for the necessary
field.

343

Internal Use - Confidential

 How do I sync the process when a new custom field is added in Salesforce?

1. Open the Salesforce operation component

2. Use the Import button to open Salesforce Import wizard.The wizard will guide you on
how to complete the import asking for connection,Salesforce object,fields etc.
3. Save the operation component and re-save the process.Finally redeploy the process for
the changes to take effect.

How do I call a Salesforce Apex webservice class?

To call a custom Apex class exposed as a web service, use the Apex Connector instead of the
Salesforce Connector.

How do I send/load data from Salesforce using the Bulk API?

There are two options that you need to look at:

1. Use Bulk API – to tell the connector to load data to Salesforce using Bulk API
2. Batch Count – the maximum number of records in a batch

To have your data upload with Bulk API, you just need to check on the “Use Bulk API”
checkbox and specify the Batch Count (the default value is 200) in the operation configuration
screen. The Boomi Salesforce connector will handle everything for you automatically at the
backend including preparing the data into batch according to the Batch Count.

As there is no option to turn on serial model, some options to address lock contention in
Salesforce could be:

• Reduce batch sizes

• Use flow control

Please note that the Salesforce connector only supports the BULK API option in parallel mode.

344

Internal Use - Confidential

 How do I Identify the Salesforce record processed within Boomi?

In order to identify the SF record processed within Boomi, you may have to manually search
for the record/data from the XML profile returned by the Salesforce connector. You will need
to use the record ID, or some other field in Salesforce that uniquely identifies the record.

Alternatively, if you configure the connector's "Tracked Fields", then you would be able to see
the recordID in the process reporting data. For reference:

http://help.boomi.com/atomsphere/GUID-103F06E6-94BF-472A-9C50-F3780CE5B497.html

http://help.boomi.com/atomsphere/GUID-C84D1FEF-BD90-46CE-BFD2-33CE720572EE.html

How do I construct complex SOQL using the Salesforce REST API?

Salesforce REST API is needed to be used in order to execute a complex SOQL.

Use HTTP Client with GET action in order to use REST API to login to Salesforce. Connection
URL should look something similar to this: https://cs18.salesforce.com/services/Soap/u/44.0

You should pass SOAPAction: login in Request Headers.

The request profile should be similar to this:

<?xml version="1.0" encoding="utf-8" ?>

<env:Envelope xmlns:xsd="
http://www.w3.org/2001/XMLSchema
" xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance
" xmlns:env="
http://schemas.xmlsoap.org/soap/envelope/
">

<env:Body>
<n1:login xmlns:n1="urn:partner.soap.sforce.com">

345

Internal Use - Confidential

 <n1:username>{1}</n1:username>
<n1:password>{2}{3}</n1:password>
</n1:login>
</env:Body>
</env:Envelope>

The Login Response profile will return a sessionID which will be used as Authorization value in
the subsequent calls.

Build your SOQL and pass it as a parameter into an HTTP Client Connector:

• Connector Action: GET

• Request/Response Profile Type: NONE
• Request Headers: pass the SessionId as the Authorization parameter
• HTTP URL: https://cs18.salesforce.com/services/data/v20.0/query/?q=SOQL

The process should look similar to this:

Figure 180: Salesforce Process

346

Internal Use - Confidential

 How do I connect to Salesforce REST API with OAuth 2.0?

Please see How to Connect to Salesforce REST API with OAuth 2.0.

4.6.5 SAP Integration

User Guide Articles

Here are some links to our user Guide, which you may find useful while using and configuring
the SAP connector.

• SAP connector
• SAP connection
• SAP operation

Glossary

• IDoc - IDoc is a popular data structure by SAP which is primarily used to exchange
business/transaction data between SAP and external systems electronically (Official
Documentation - IDoc Interface/ALE - SAP Library)
• BAPI/Remote-enabled Function Module - Interfaces developed with in SAP which can
be triggered from a external system or with in SAP
• (Official Documentation - SAP Library , RFC Interface - Connectivity - SAP Library)
• SAPRouter - SAProuter is an SAP program that acts as an intermediate station (proxy)
in a network connection between SAP systems, or between SAP systems and external
networks (Official Documentation - What is SAProuter? - SAProuter - SAP Library )
• SAP NetWeaver - SAP Netweaver refers to open technology platform that offers a
comprehensive set of technologies for running mission-critical business applications
and integrating people, processes, and information (More details
- http://scn.sap.com/community/netweaver)
• SAP Netweaver Gateway - SAP Gateway is a framework that connects business users to
SAP systems using consumer technologies, groupware, and mobile devices and is based
on open standards (such as the Atom Publishing Protocol and OData) that offer simple
services based on the REST principle (Official Documentation - SAP Gateway 2.0 – SAP
Help Portal Page )

347

Internal Use - Confidential

 Understanding SAP Integration Options

There are several options for integration with SAP system. IDoc, BAPI/RFM, file extracts, and
SOAP web services are some of the most common technologies widely used for interfacing
with SAP. A recent addition, SAP Netweaver Gateway, can make your SAP business data
accessible as RESTful resources. Based upon REST and OData principles, the SAP Netweaver
Gateway exposes a uniform, stateless interface to any software system that can communicate
using HTTP(S) and XML or JSON payloads. This approach is highly adopted in the B2C space.

While the integration use case requirements certainly factor into selecting the connectivity
approach, the customer’s preference plays a huge part. Often customers will have policies or
at least preferred approaches for integration with their SAP instance, as well as
skills/knowledge in a particular connectivity option. For example, a customer may be an
“IDoc shop” vs. a “BAPI shop”, or have decided to perform all integrations via a web services
gateway.

AtomSphere Approach and

Style Description Timing
Considerations
• “Intermediate
Document” • Use SAP Connector
• Standard or • XML profiles
extended • Submit IDoc directly to SAP
Send IDoc to
(customized) IDoc importer
SAP via IDoc Asynchronous
formats • Atom must have direct
framework
• Exchange directly connectivity to SAP (local
through SAP install)
framework

• “Intermediate
• Use SAP Connector
Document”
Receive • XML profiles
• Standard or
outbound IDocs Asynchronous • Receive IDocs from SAP real
extended
from SAP time/event based
(customized)
• Single listener process
formats

348

Internal Use - Confidential

 AtomSphere Approach and
Style Description Timing
Considerations
• Exchange directly • Atom must have direct
through SAP connectivity to SAP (local
framework install)

• Use file-based connector

such as Disk or FTP
• Same as above but
Connector
IDocs are
• User Defined EDI profiles
IDoc communicated as
(can be difficult/tedious to
import/export standalone files Asynchronous
configure by hand)
via file • Scheduled SAP jobs
• Atom may be local or
export/import files
remote depending on file
location

• Use SAP Connector

• XML profiles
• Can connect to BAPIs and
• “Business API” or public RFMs
“Remote-enabled • It is common to create
Function Module” custom functions (a.k.a. “Z
• Call SAP native API functions”) to
BAPI/RFM Synchronous
functions directly augment/wrap standard
• Function may get or functions to meet specific
send data from SAP integration requirements
• Atom must have direct
connectivity to SAP (local
install)

• HTTP Client / OData

Connector
• SAP NW Gateway
Installation and
SAP Gateway Configuration (SAP)
• Content development and
Web Services Synchronous
• REST, OData publishing (SAP ABAP
Development)
• Network considerations
applicable as for other
webserservices

• Web service
• For SOAP, use Web Services
gateway
SOAP Connector
• SOAP, REST, XML-
• For REST or XML-over-
Web Services over-HTTP Synchronous
HTTP, use HTTP Client
• Often just exposing
Connector
BAPI/RFM as web
• XML profiles
service endpoint-

349

Internal Use - Confidential

 AtomSphere Approach and
Style Description Timing
Considerations
Synchronous • Atom may be local or
request/reply remote depending on web
service accessibility
• SOAMANAGER configuration
(SAP)

• This approach should not

Direct integration to the
Database N/A be used.
SAP database.
Table 38: SAP Approach and Considerations

Selecting an Approach

There may be number of factors for e.g. Customer's integration strategy, business
requirements like security, performance, volume, ordered delivery, custom development &
SAP shipped content etc which influence or determines an integration approach. Often
customers will have policies or preferred approach matrix for integration with their SAP
instance. In some cases skills/knowledge in particular connectivity mechanism also influence
integration approach. For example, a customer may be an “IDoc shop” vs. a “BAPI shop”, or
have decided to perform all integrations via a web services gateway. In recent years there has
been inclination towards a web service (SOAP/REST) based approach for integration

Architectural Considerations
BAPI and IDocs

The SAP connector supports connectivity via BAPI/RFM and IDocs. The connector uses the SAP
JCo (Java Connectivity) and Java IDoc Class libraries, setup details are explained in depth
here How to Configure SAP R/3 for IDoc and BAPI/RFM Connectivity.

A local atom is generally required because of following reasons:

• SAP ERP is mostly locked behind customer's network and firewalls. Direct connection
from outside is not allowed.

350

Internal Use - Confidential

 • Transaction ID management for IDoc Listener which requires a local database for
acknowledgment tracking.
• There is no provision to install the prerequisite SAP library jars on the Boomi Atom
Cloud.

4.6.5.4.1.1 SAP Local Atom Reference Architecture

Figure 181: Local Atom Reference Architecture

351

Internal Use - Confidential

 4.6.5.4.1.2 Outbound IDoc SAP Reference Architecture

Figure 182: Outbound Reference Architecture

Note that some customers may use SAProuter which acts as application level gateway,
allowing connection from external systems in controlled and secured way. The SAP Connector
can route requests to SAProuter.

Web Services (SOAP/REST OData)

A SOAP enabled web service may be exposed using existing BAPI/Functional modules or
modeled and generated natively. For more information, please search for SAP Enterprise
Services and SOAManager on SAP Help Portal – The central place for SAP documentation.

352

Internal Use - Confidential

 SAP Gateway is a framework that enables and exposes REST and OData based services which
connects business users to SAP systems using consumer technologies, groupware, and mobile
devices. SAP Gateway is offered as a separate add on which can be installed on either existing
SAP business suite or separately as SAP Gateway Hub system - SAP Gateway 2.0 – SAP Help
Portal Page

Generally web service endpoints are exposed to outside world via reverse proxy or gateway.
Since this approach does not requires specific jars, there a is no technical requirements of
using a local atom. However such decisions are subjective to company's network and system
security guidelines

SAP Connector Overview

The SAP Connector connects directly to your SAP NetWeaver-based application exchange data
via BAPIs, Remote-enabled Function Modules (RFMs), and IDocs. You can browse the list of
available BAPI/RFMs and IDocs available in your SAP system and auto-generate the request
and response profiles to use in AtomSphere processes and maps.

The SAP Connector supports three types of actions:

• GET - Used for BAPI/Remote Function Module (RFC type communication)

• SEND - Used for Inbound IDoc processing
• LISTEN - Used for Outbound IDoc processing or receiving IDoc

Additional notes:

• The connector uses the SAP Java Connector (sapjco) library to communicate with SAP
via the CPI-C protocol
• The connector is SAP Certified

353

Internal Use - Confidential

 • The connector requires an enterprise connection license to deploy
• User Guide documentation: SAP Connector

Prerequisites

There are a number of environmental and SAP application configurations required to perform
the integration, including local Atom and SAP specific configurations. SAP configurations steps
will require an SAP admin.

For complete details see How to Configure SAP R/3 for IDoc and BAPI/RFM Connectivity.

Common Integration Patterns

Asynchronous

4.6.5.6.1.1 Sending an IDoc to SAP (SEND)

• Browse and Import IDoc XML definition via SAP Connector (imported as XML profile)
• From your process logic, push IDoc data (XML) to SAP Connector which would create an
IDoc at SAP end point, to be processed by a SAP application
• SAP connector uses SAP IDoc class libraries for JAVA applications. More details - IDoc
Support for Java Applications - SAP NetWeaver Composition Environment Library - SAP
Library

354

Internal Use - Confidential

 Figure 183: Send IDoc to SAP

4.6.5.6.1.2 Receiving/Listening an IDoc from SAP (LISTEN)

1. SAP configuration details are explained here: How to Configure SAP R/3 for IDoc and
BAPI/RFM Connectivity
• IMPORTANT: Because there can be only one listener for a given Program ID, this
means you must use a single “global” listener process to receive ALL IDoc types
and then route accordingly within the process flow.
2. The connector listener identifies itself to SAP using a unique “Program ID” value.
3. The connector listens for outbound IDoc messages from SAP by registering itself as a
client with the SAP IDoc framework.
4. Upon receiving an outbound IDoc message, the connector sends an acknowledgment
back to SAP.

355

Internal Use - Confidential

 • IMPORTANT: The connector immediately acknowledges the IDoc
upon successfully receiving the IDoc message, not upon successful
completion of the process execution. This means the process should be
designed to be resilient in the event of any process errors because the IDoc
event will not be resent by SAP.
5. The connector checks the status of inbound IDoc in the table. If there is an entry for
same Transaction ID (TID) with a "COMMITED" (success) status, then the IDoc is ignored
to avoid processing duplicates.
• A local database is required to maintain a table to track Transaction ID and
status.
6. The connector immediately executes the process with the IDoc document.
7. The connector automatically sets a document property called SAP Connector > IDoc
Type that you should be used to route the different types to specific maps or
subprocesses.
• Note that the Operation in the global listener process will still reference a
specific IDoc Type but this is arbitrary. The Operation’s IDoc Type is not
relevant for the listener and is only configured as such to generate the XML
profile for use with mapping.

356

Internal Use - Confidential

 Figure 184: Receive/Listen for IDoc from SAP

More details about the TID database are covered in How to Configure SAP R/3 for IDoc and
BAPI/RFM Connectivity.

Synchronous

4.6.5.6.2.1 Executing a BAPI (GET)

1. This method involves SAP connector executing a BAPI over RFC protocol.
2. For every request sent, connector expects a response back from SAP.
3. The method utilizes SAP JCo libraries which enables Java application to connect and
communicate with SAP system and invoke Remote Function Modules.
4. SAP JCo architecture is explained here - SAP JCo Architecture (Standalone Version) -
Components of SAP Communication Technology - SAP Library.

357

Internal Use - Confidential

 Figure 185: Executing a BAPI over RFC

4.6.5.6.2.2 Calling an SAP Web Service (SOAP/REST)

As explained in architectural consideration, a SOAP or REST web service could be easily

exposed or developed on SAP system. The steps involved are out of scope of this article,
however there are plenty of resources on SAP community and help.sap.com.

358

Internal Use - Confidential

 Figure 186: Calling a SAP Web Service

4.6.6 Workday Integration

Overview

The Workday application offers a number of options for integrating data with other systems.
Which option to choose will depend on your functional and nonfunctional requirements. All
data in and out of Workday goes through web services. All of Workday’s packaged integrations
as well as the integration tools available in Workday ultimately leverage the web service API
through the Object Transaction Server (OTS). However, the interaction with those web
services can be performed in a variety of ways.

Let’s look at the options for integrating with Workday.

Workday Web Services - SOAP API

Depending on the integration scenario, vendors/client applications can call the Workday APIs
to push/pull data using Workday’s SOAP API. The Workday SOAP API is well documented in the

359

Internal Use - Confidential

 Workday Community and supports a large number of transactions. To utilize the Workday Web
Services SOAP API, the Boomi Workday Connector should be used.

The Workday Connector allows a developer to connect to a target tenant by:

• Single set of user credentials to be utilized across all Workday connections. Note that
you should configure your Integration Security User (ISU) in Workday to have all
permissions needed across your Workday integrations.
• Including version in the connection, which allows this to be extended and set through
environment variables
• Configuring the Workday service in the operation component. This allows a single
Workday connection to be used against different service endpoints for the same
tenant.

Figure 187: AtomSphere Workday SOAP Model

360

Internal Use - Confidential

 Retrieving from Workday

Below are some techniques and considerations when retrieving data using the Workday
Connector.

• Transaction Log
o Use Transaction Log if available, processes more efficiently when utilized. For
example, if querying for modifications, instead of querying all data between
two date points and comparing for changes, Workday can utilize the
transaction log for the same period and only compare those records that had
transactions in the specified range. The former will query ALL records and
compare, the latter approach will only select those that had transactions and
then compare.
o Otherwise the call is run against all data records which will take longer.
o See also How to Query Workday Using the Transaction Log.
• Effective date vs Entry date
o Understand the difference when selecting records.
o For example: Getting all users since last run date (Entry date) without
considering for effective date could result in missed data from data that was
keyed in earlier but not effective until the current date range. i.e. future
dated employees not yet effective.
o Alternatively, if you want all records, regardless of when they become
effective. In this case the as of date in the request can be current and as of
effective date can be set to the distant future.
o May want to store both Last Entry Date and Last Effective Date parameters as
persisted process properties.
• SOAP API paging
o The Boomi Workday Connector will automatically retrieve all available pages.
In the Query Operation be sure to enable the option "Return All Pages". Note
each page of results is returned as a single Boomi document with multiple
records within that document. For example, assuming a page size of 100, if a
given query found 250 results, three documents would be returned to the
process: the first with 100 records, the second with 100 records, and the third
with 50 records. This is notably because this is slightly different behavior than

361

Internal Use - Confidential

 many other connectors. You will need to use a Data Process shape to split by
record if you wish process independently.
• Calculated fields and custom fields can be retrieved as well but may require additional
setup in Workday to expose them through one of the delivered web services
o Visit the Workday community for how to add additional fields to a service call
through the use of Field Overrides

Most of the considerations above refer to polling the Workday application for data. This API
can also be utilized to retrieve individual queries and lookups as well. The Workday delivered
APIs are a reliable and efficient way to retrieve data ad-hoc in integration processes as there
is little overhead compared to the other mechanisms for getting data out of Workday.

Sending to Workday

Below are some techniques and considerations when sending data (create or update records)
using the Workday Connector.

• Your reference IDs will need to match. Reference IDs are used to refer to other object
instances. Although these are static strings, they may vary across tenants during an
implementation. Consider setting up mapping in Boomi, or setup a report in Workday
that can be called within the integration, if this should be controlled in Workday.
• If the API you are using allows for batched imports, this should be taken advantage of.
However, Workday will fail the entire batch if one record in the batch fails. Be sure to
handle for this in your process. For example, you can attempt to send records as a
batched request and then if an error is encountered, break up the batch and send
each record individually.

Integrations API

The Workday SOAP API also has an Integration Service endpoint to execute a Workday Core
Connector or custom Studio or EIB integration. This API call can be paired with a file based or

362

Internal Use - Confidential

 outbound call approach to exchange data. The request is answered by a 200 OK and the
integration system spins up to execute its logic. See Recommendations below.

Custom REST-ful APIs

The Workday APIs for Reports as a Service, Message Queues, and Studio listeners can be
invoked as well. These APIs are not available out-of-the-box in Workday and require some
Workday custom development to be exposed.

Reports as a Service (RaaS)

Custom reports can be setup in Workday and then be web service enabled. This allows
AtomSphere to make a REST-ful calls to retrieve the report, in XML or JSON format. Custom
reports can often be setup by business users and then exposed through a toggle within
Workday. Reports are an easy way to retrieve custom objects and fields, leverage calculated
fields for summary data, and join multiple objects into a single data source.

To call a RaaS service within AtomSphere you will use the HTTP Client Connector, not the
Workday Connector. The HTTP connection is configured with Basic Authentication and the
service URL of the report.

Reports can also be setup with filters that can be passed in the GET request. This allows the
same Workday reports to be parameterized to retrieve for example changes between two
dates or a specific record. Filter options will vary based on the data source for the report.

Be mindful of time it takes Workday to run the report. Long running reports may need to be
optimized to prevent HTTP timeouts. Consider using indexed data sources, data type, sorting,
and reducing number of calculated fields to optimize. Visit the Workday community for more
on optimizing custom reports. Workday reports are not as quick as using the Workday SOAP
API, there is some overhead for initializing and running the report. One helpful trick is to
shorten the XML element names in the generated XML, which can greatly reduce the size of
large XML files.

363

Internal Use - Confidential

 Figure 188: RaaS Workday Model

Message Queuing

Workday has message queuing available within the Workday tenant. Message queues reside
within the tenant and must utilize the API to place messages on the queue. This means you
will need a REST-ful client to place events on the queue. Typically this can be accomplished
using Workday Studio to build the mechanism to place events transactions generated within
Workday on the queue (for outbound consumption) or use AtomSphere to place inbound data
on a queue where it can be asynchronously processed later by an internal integration.

364

Internal Use - Confidential

 Below are some considerations for using message queuing.

• Limits on the number of queues available in the tenant (10), messages expire after a
week, maximum number of messages on a queue (64K).
• Queue can be accessed through REST-ful API calls. Workday only needs to place events
on the queue. An AtomSphere process and then retrieve and process events off the
queue.
• Decouples Workday from the delivery and processing of messages to the various
destinations. Workday only needs to compile and expose the events.
• For outbound integrations where Workday Studio places events on queue consider:
o How often will it run?
o How quickly does it run?
o Leave validation logic in the client applications or downstream integration
layer.
o Separate queues per event or consolidate and route in subscriber? Consider
designing a single AtomSphere process to receive all messages and route based
on queue or payload metadata.
• Note if Workday is unavailable, no messages will be added to the queue.
• If the Boomi atom is unavailable, the process will resume picking up messages from
the queue when it becomes available again.
• Once the process has successfully executed records into destination applications, a
subsequent call is made to Workday to delete the message from the queue.

365

Internal Use - Confidential

 Figure 189: Message Queuing

Outbound Subscriptions

Using Subscriptions in Workday, you can configure outbound messages for events that occur
within the Workday tenant to achieve real-time processing. On the AtomSphere side, one
more or web services listener will be developed to receive the outbound messages from
Workday.

IMPORTANT: A disadvantage to this approach however is the subscription is automatically

disabled by Workday if the messages cannot be delivered to the endpoint after several
attempts. When evaluating this approach, carefully consider the reliability of the connection,
availability of the destination, and assured delivery design implications.

366

Internal Use - Confidential

 Below are some considerations for using outbound subscriptions.

• The standard outbound message payload only contains the event notification details,
not the full business record. Within the listening process, a subsequent query back to
Workday is usually required to retrieve the full record.
• Multiple endpoints can be configured for a subscription.
• No error handling if the event is not able to be delivered. This will have to be
managed in the integration.
• Use with environment restrictions to configure test vs. prod endpoints.

Figure 190: Outbound Subscriptions

367

Internal Use - Confidential

 File Based

Within the HCM and Financials domain, file based integration is often the only option for
vendors and legacy applications who do not expose web services. In a typical outbound
scenario from Workday, data is extracted from Workday using one of the aforementioned
methods, transformed to an agreed upon format (often a delimited/CSV or fixed length text
file), and then delivered to an external location mutually accessible to both Workday and the
target application. Because all data in and out of Workday is through web services, files
ultimately are produced from or broken down into the Workday APIs discussed above.

Using this approach requires the setup of an external shared location such as an FTP or SFTP
site that is accessible to both the source and target applications.This pattern requires
coordination from the target application process of when to retrieve the file. This
intermediary state created--although only for a short duration of time--can become outdated
once written if both applications are simultaneously updated directly from a different
mechanism during the coordination window.

Also, although having an intermediate file is convenient for integration debugging, it can also
be a liability. Files get misplaced, sent to the wrong place, left on local machines, and sent
over insecure mail during troubleshooting.

There is also the limitation of not being able to process records that “last mile” into the
endpoint. Were the records within the file actually loaded successfully into the destination?
Lack of end-to-end visibility can be challenging to manage.

Recommendations

• Reduce point-to-point interfaces by utilizing AtomSphere as your central hub for

integration processing. With this you can achieve an end-to-end view of integration
status while maintaining all business logic and error handling in one place.
• If your source and target endpoints have APIs available, utilize these direct
connections over file based approaches.
• When sending data to Workday:
o Use the Workday Connector and leverage the Workday SOAP API.

368

Internal Use - Confidential

 o AtomSphere can connect directly external applications, either on-premise or in
the cloud, and process directly into Workday.
o Depending on the external application, look to write back a record status to
the source application (e.g. synced = true/false).
o Although Workday may throttle API requests under high load, there is no set
governance limits for using the Workday API and they are robust enough to
handle high volume processing.
• When retrieving data from Workday:
o Options for real-time integration:
 Outbound Subscriptions to an AtomSphere web service listener process.
However the fragility of the connection should be carefully weighed.
 If the desired changes are associated to Workday business processes,
build a Studio integration to fire and deliver the event via HTTP callout
once the event hits a completed state in the business process.
o Options for near-real time integration:
 Schedule an AtomSphere process to run as frequently as desired (down
to every one minute) that polls the Workday API transaction log for
changes.
 Develop a Workday Studio integration (set up as described above) that
places events on a Workday message queue. Then schedule an
AtomSphere process to poll the message queue every minute to process
to downstream applications.
o Options for full record synchronization:
 If you need data from multiple objects in Workday, create a custom
report in Workday and expose it as a RaaS service then schedule an
AtomSphere process using the HTTP Client connector to periodically
extract the data in a single call.
 Otherwise retrieve data with an AtomSphere process using the Workday
connector to extract records for a single object. Additional data can be
retrieved in subsequent Workday SOAP API or RaaS calls if needed.
o Options for detecting and syncing changes only:
 If a Transaction Log is available, use the Workday connector to call the
transaction log to identify changes.

369

Internal Use - Confidential

  If the changes can be filtered using delivered fields for the target
object consider setting up a RaaS that uses time stamps in the filter
then have the AtomSphere process call the RaaS and transform and
deliver.
 If there is a Workday Core Connector integration available for the
object in Workday, take advantage of the packaged change detection.
Have the AtomSphere process call the Workday Integrations API and fire
the integration. Then either have it delivered to an AtomSphere listener
endpoint or poll the integration API to retrieve the output documents
for that event.
 If the above options are not available, use an AtomSphere process with
two queries. First query records by the saved As Of Date, then query
records by the current As Of Date. Next compare and filter. Each call
would pass in the updated As Of Date.

Additional Reference

• Workday Connector
• HTTP Client Connector

4.7 Understanding Data Profiles

4.7.1 XML (eXtensible Markup Language)

Figure 191: XML Data

370

Internal Use - Confidential

 Contents
Declaration: <?xml version="1.0" encoding="UTF-8"?> is a declaration. Most XML files begin
with the declaration to describe information about itself. If you're using XML as test data in a
process, the declaration can be omitted.

Tags: tags are always paired (opening/start and closing/end tag), <note> (opening tag) with
</note> (closing tag), <body> and </body>, etc. In the case of <salutation/>, that is called a
self closing tag and does not need a closing tag. It has no value in it. It is equivalent to
<salutation></salutation>. Element names (tags) are case sensitive.

There is data in between the tags. Tags can encapsulate other tags, making a hierarchy. The
tag on the outside is called a parent element and the tag(s) on the inside are called the child
(children) element. You can have multiple levels of parent and children. The parent can be in
another parent tag (and that can be in another parent tag). <to> is a parent element of
<name> and <address>. The element <name> is a child element of <to>. The element <to> is a
child element of <note>. Name could have more elements underneath of it. Elements that
have the same parent element are sometimes called "siblings", and the "ancestors" of an
element are the parent element, its parent (grandparent), etc. Likewise, the elements that
are the child, grandchild, etc. are sometimes called the "descendants" of an element. The
descendants of the root element include all of the elements in an XML file.

Root element: the highest element is called the root element or node. There can only be one
root element that encapsulates everything (except the declaration).

Attributes: in <name> there is an attribute called "number". Attributes are a name-value pair,
where the value is in double quotes "". It can only have a single value and it must be in the
start tag or self closing tag:

Figure 192: Single Value

371

Internal Use - Confidential

 Nesting: elements have to be nested correctly, you cannot have <a><b></a></b>, it has to be
<a><b></b></a>.

Whitespace: doesn't matter outside of the tags, it is more for human readability.

An empty element is converted to "" in a map shape. There is no NULL in XML.

Some XML documents are defined by a "schema", a document (written in XML) that defines
the type of elements, their relationships and sequencing, the format of certain elements, and
sometimes even the number of elements that can be included in the document. The schema
is, itself, an XML document, but it usually has the three letter file extension of XSD, whereas
an XML document has the file extension XML.

In many cases, it is useful to "validate" an XML document against the schema. Validation
makes sure that the XML document meets all of the requirements of the schema. There are
several shapes in Boomi processes that do this validation internally, to find and prevent
errors.

Figure 193: XSD Document

XML and Boomi Integration

An XML document profile in Boomi might be created either by importing the XML document
itself, or by importing the schema file. If the XML document is imported, the profile is

372

Internal Use - Confidential

 created so each element in the document has details that can be modified in the Data
Elements tab of the profile, and nothing will be shown in the Data Types tab. If the profile is
created by importing the XSD file, the Data Elements tab will still display the details for each
element, but most of them will be disabled. Instead, the details must be edited in the Data
Types tab by selecting the appropriate "type" for the element.

When an XML profile is created in Boomi from another system, it is created by importing the
XSD schema. When a document is generated using that profile and it is sent to the system, it
will be validated (checked to make sure it fits) against the same schema. For example,
NetSuite is a third-party system that can receive XML data to create records. In a Boomi
process that connects to NetSuite, a profile is created by connecting to NetSuite and
importing the schema for the records to be created. Boomi creates an XML profile that
matches that schema. When a document is generated in the process using that profile, it
would be sent to NetSuite. To receive the document successfully, NetSuite will validate the
document against the schema, making sure that all of the required fields are present and that
the document "fits" the fields. Validation ensures that you don't have "opportunity" records in
your "customer" list.

"Namespaces" provide a way for different types of elements to have the same name, and still
avoid conflicts. XML tags (elements) are defined by the developer of the schema. They are
free to choose the element names they like, but this often results in developers using the
same element names that other developers use.

Figure 194: Two "Note" Elements

373

Internal Use - Confidential

 In the example above, the two "note" elements would create a conflict if they were added
together. They represent different kinds of "notes”, so the tag is meaningful to both, but an
XML user or application would not understand how to handle these differences. Using a
"namespace" allows common tags to be used together. A namespace defines a "prefix" that is
placed in front of the element name to indicate which namespace it belongs to. The
namespace itself is defined by an xmlns attribute in the start of an element.

Figure 195: Different Prefixes

In the fragment above, the two "note" elements will not create a conflict because they have
different prefixes, and they belong to different namespaces. The namespace declarations can
also go in the XML root element, and this is usually what you will encounter in XML documents
processed by Boomi.

374

Internal Use - Confidential

 Figure 196: URI as an Identifier

The URI in the namespace declaration is not used to "lookup" any information, it is simply an
identifier. Because no two companies can have the same Internet domain and URI, they are a
good way to guarantee a unique name for namespace. Sometimes, though, the URI is a valid
link to the web page that contains more information about the document type that is defined
by the namespace.

XML is at the very heart of a Boomi process, a Boomi atom, and even the AtomSphere
platform itself. XML is the primary data type used to send documents, messages, even
processes from the platform to a local atom and even between nodes of a molecule or cloud.
In the revision history of any component on the Build canvas in AtomSphere, you can view the
XML of the component. The XML format can seem unnecessarily long, but the strict and
consistent formatting makes it a valuable tool for storing and sending information efficiently.

4.7.2 Flat File

Flat file data types are the simplest data types. A common example of a flat file is a .csv
(comma delimited) file. Other separator characters can be used as well, such as tab, asterisk
(star), or pipe.

Contents

A flat file can have column headers as the first line, giving the name of each field in the file.
This line is not always included.

375

Internal Use - Confidential

 If a flat file has data elements that contain the same character as the field delimiter (e.g.
comma) then a "text qualifier" character is used to wrap the value of those fields. The text
qualifier options are Double Quotes, and Single Quotes. If the data elements contain the text
qualifier character as part of the data, then a "Text Qualifier Escape" character is used to
signal that the double quotes are part of the data, and not a text qualifier.

Figure 197: Flat File Contents

The first line of this sample has the column headers. These are the names of the data
columns. If you opened this file in Microsoft Excel, you would see these names at the top of
the columns. But this line is not considered "data".

In the first line of data, the Name includes a comma followed by "P.E." the designation often
given to a professional engineer. Because the comma is part of the name string, the field
value is wrapped in double quotes, the text qualifier used in this file.

In the second line of data, the Address field includes both quotes as part of the data, around
the name of the building, and a comma. There are double quotes around the whole field, and
the quotes around the building are escaped with a backslash character to indicate that they
are part of the data.

It is also possible to set an escape character for the delimiter character, and an escape for
the escape character, though these options are not commonly used.

Flat Files and Boomi Integration

In a Boomi Flat File Profile, the escape character can be set to any character, even
"undisplayable" characters like a control character (e.g. Ctrl A, or 0x01, the "start of header"
character). This can be done by setting the "File Delimiter" character to the "Byte Character"

376

Internal Use - Confidential

 option and entering the hex value for the character (e.g. 0x01).

The Flat File profile can be used with a "data positioned" (sometimes called "fixed width") file
type too. To change the profile from "Delimited" to "Data Positioned", click the Options tab in
the profile and select "Data Positioned" in the File Type pulldown menu. Once this is done,
you will see that the options for the delimiters and escape characters are no longer visible.
You need to specify the padding character used in the data that fills out the columns to their
defined width. This is often a single space character.

Figure 198: Data Positioned

Each field in the file begins at a specific column and has a fixed width. The padding character
is what separates the data and is repeated to fill in each field. In the example above, the
"Use Column Headers" option would be enabled because the first row contains the column
headers.

When you add elements to the profile, you also specify the start column for each field. The
first column in the file is 0 (zero). You can optionally set a length of the field so that
characters beyond that length will be ignored. For the example data above, the NAME field
would start with column 0, the ADDRESS field would start with column 20, the CITY field
would start with column 50, the STATE field would start with column 70, and the POSTAL
CODE field would start with column 80. The padding character for this data would be a space.

Data Positioned files like this are less common than delimited flat files.

4.7.3 Database
The database document format is akin to the flat file, because it is a "pipe" delimited file.
The pipe character | (also sometimes called the "vertical bar") is rarely used in actual data
values, so it is used as the delimiter in this file format. However, t's a little more complicated

377

Internal Use - Confidential

 than that.

Documents of this format are created by reading data from a database using a SQL query or
stored procedure call that returns data.

The database profile can be created by importing a table from a database, or by importing a
stored procedure from Oracle, MySQL, or Microsoft SQL Server. When importing a table, all
fields of the table are imported initially. If you need only a subset of the fields, you can
remove the ones you do not need after the import is done. For a stored procedure, you must
retain all fields in the profile, otherwise the stored procedure will likely fail with an error.

The database profile can be either a "read" type for receiving information from a database, or
a "write" type for sending data to a database. When a profile is created that calls a stored
procedure with both "in" and "out" parameters, use the "read" type.

A database profile often contains the SQL statement that it works with to send or receive
information to/from the database. A SQL statement that retrieves a set of fields from a table
named "NEW_CONTACTS" might look like this:

Figure 199: SQL Statement that Retrieves a Set of Fields

This example would retrieve all rows of the NEW_CONACTS table. Often there will be a
WHERE clause that includes parameter values that limit the records returned. Sometimes the
parameter values may be passed to the statement through the Boomi process so that the
parameter changes each time the process is executed. For example, the same statement as
above might be used to only get the contact records that have the status value of "New" if
that string were passed as a value in the parameter that replaces the question mark (?) in the
statement.

378

Internal Use - Confidential

 Figure 200: SQL Statement that Retrieves a Set of Fields

The results of either of the queries above would look like this in the document returned to
the Boomi process:

DBSTART|65cf5247-62d6-44f5-a9a1-
caa11cd32d4c|2|@|BEGIN|2|@|OUT_START|3|@|false|^|0018000000Ptpg4AAB|^|003800000
0a0QbXAAU|^|Liz D'Cruz|^|(650) 450-
8810|^|LiuzAU|^|New|#|true|^|0018000000Ptpg4ACD|^|0038000000a0QbXAFT|^|Sam
Hill|^|(888) 555-1212|^|SmHill|^|New|#||#|OUT_END|3|@|END|2|@|DBEND|65cf5247-
62d6-44f5-a9a1-caa11cd32d4c|2|@|

In the example above the field values of the database records are a little hard to distinguish,
but they are there. There is more to the data than just the fields though. The document
shown begins with "DBSTART" followed by an identifier that is used for internal purposes. The
actual database field values are listed after the "OUT_START" followed by "3" and "@". The
field values are separated by "|^|" and records (each database row) are separated by "|#|".
The end of the records returned is demarcated by "|#|#|" followed by "OUT_END", then
similar fields used by Boomi and the database.

Most often it isn't necessary to look at a document in the database format, but it can be
useful to know how to recognize the fields to determine what data is sent or received.

In Boomi, the database profile (not the document format) consists of a couple of parts. The
profile determines whether you are reading information from the database or writing
information to the database. The setting for this is in the "Options" tab of the database
profile.

379

Internal Use - Confidential

 When you create a profile to write information to a database, you will need to add the
database fields to the profile manually. This is only necessary for a "standard" insert
statement though.

In the Data Elements tab of the database profile, there are three distinct parts to the tree
view shown on the left side of the window. The "Statement" is where you add the SQL
statement(s) that will be executed when the connection is made to the database. This might
be as simple as a SELECT statement that returns data from the database, or something much
more complex.

4.7.4 EDI (Electronic Data Interchange)

EDI data is used for the B2B sector (Business to Business). It is very shorthand and
abbreviated. The two most popular EDI data formats are X12, used mainly in the US and
EDIFACT, used mainly in the EU. They are similar with a few slight differences. There are
other EDI data types but less common.

380

Internal Use - Confidential

 Figure 201: X12 Document

In order to format, copy and paste into Notepad++. Use the replace command (Ctrl+H), find ~
(or whatever segment terminator is being used), replace with ~\n to add a new line after each
segment terminator. Make sure Search Mode is Extended.

Each set of data (line for easier visuals) is called a segment, the ISA segment, GS segment, ST
segment, etc. Each segment ends with a segment terminator. In this case it is a ~, but it can
be almost any character including a new line. The segment name (ISA or ST) comes from the
segment ID. The segment ID is the first entry in the segment.

Inside the segment are elements, which are separated by delimiters, in this case *. The first
element of a segment is always the segment ID, after that are other elements. Some
elements can be empty. Let's take:

ST*850*103465910~

381

Internal Use - Confidential

 the 850 position is called ST01, the 103465910 position is called ST02. Let's look at this one:

BEG*00*SA*0065-9182075-0579*123*20020624~

BEG01 = 00, first element in BEG

BEG02 = SA, second element in BEG
BEG03 = 0065-9182075-0579, third element in BEG
BEG04 = 123, fourth element in BEG
BEG05 = 20020624, fifth element in BEG

All of these values have a very specific meaning, like it's an address or a date of delivery.
Those specific meanings can be found in a specification sheet (spec sheet) for the specific X12
document. The example above is an 850 document because ST01 identifies the X12 document
type.

Loops

Figure 202: Loops

These are called loops. Loops are repeating sets of segments. In this case, the N1 loop
contains the segments N1, N3 and N4. It is repeated twice. Loops can be repeated some
number of times based on the specs, it could repeat indefinitely. You'll notice N101 is
different, some segments have elements where only certain values can be used. Those certain
values are called qualifiers. They mean something very specific and that's how loops are
differentiated.

For example, in the ‘PER’ segment, the pair of data elements PER03 and PER04 determines
the method of contacting someone:

382

Internal Use - Confidential

 BN - Beeper Number
FX - Facsimile
TE - Telephone
TL - Telex
TM - Telemail
EM - Electronic Mail
CP - Cell Phone

PER*CR*JOHN J.JOHNSON*TE*2145551212 -> (Telephone)

PER*CR*SAM SMITH*TL*0198237667 -> (Telex)
PER*CR*BILL JONES*EM*BJONES@ACMECO.COM -> (E-mail)

The Envelopes
The ST and SE segments are a pair and enclose one set of data. That set of data is called a
transaction set. It has information about that specific X12 document.

The GS/GE segments are another pair and enclose transaction sets. This is called a functional
group, it is a group of transaction sets or multiple transaction sets are in a functional group,
usually the same types of transaction sets.

The ISA/IEA segments are another pair that encloses the GS/GE pair. It is called the
Interchange envelope and carries information about the sender and receiver. There can be
multiple functional groups in a single Interchange envelope.

4.7.5 JSON (Javacript Object Notation)

JSON stands for JavaScript Object Notation and is used as a data format. It is used to transmit
data between a server and web application, as an alternative to XML.

Example of what it looks like:

383

Internal Use - Confidential

 Figure 203: JSON

You'll notice it is only made up of mainly brackets, colons, commas, double quotes, and text.
There is always the main { } around everything.

Keys and Values

JSON is made up of keys and values, or key/value pairs. A key is always a string enclosed in
quotation marks, usually describes what the value is. A value can be a string, number,
boolean expression, array, or object. Keys are case sensitive. A key value pair follows a
specific syntax, with the key followed by a colon followed by the value. Key/value pairs are
comma separated. Let's take this example:

"height" : "64"

The key is "height" and the value is "64". In Boomi, height is the element's name and 64 is the
element's value.

The value can be many things and they can be nested inside each other:

• String - plain text characters which usually form a word with quotes
• Number - some integer, without quotes
• Boolean - true or false as the value, without quotes
• Object - an array of key/value pairs, denoted by { }
• Array - an array of values, denoted by [ ]
• Null - an empty value, denoted as null without quotes

384

Internal Use - Confidential

 Objects
An object is indicated by curly brackets. Everything inside of the curly brackets is part of the
object. We already learned a value can be an object. So that means "person" and the
corresponding object are a key/value pair.

Figure 204: Object

The key/value pair "name” : "John" is nested inside the key/value pair "person" : { ... }. That's
an example of a hierarchy in JSON data. An Object can have any combination of values inside
of it, so it could have another object, string, number, etc as a value for a key/value pair.

Arrays
The "worksWith" key has a square bracket to determine it is an array. The value is multiple
strings this time. Each array element has to be the same thing. There can be an array of
numbers, strings, objects, etc.

Figure 205: Array

385

Internal Use - Confidential

 Figure 206: Array of Objects

Profiles
When building the JSON profile, there aren't many other options besides what kind of element
type to choose. Your JSON profile can be as complicated or as simple as you'd like. The only
other thing is a repeating array and absolute array. Repeating arrays repeat one kind of
element and allow you to repeat once or unlimited times while absolute arrays allow for
different kinds of elements but you have to define every instance and type of element in that
absolute array in the same order the data is coming in. You only have to define the element
once in a repeating array.

386

Internal Use - Confidential

 Figure 207: 3 Elements in an Array

You would need to define 3 array elements in the absolute array in the example above. If you
wanted another string and object element to the absolute array, you'd have to define 5
separate array elements in the absolute array. If the profile is out of sync with the way the
data has it, you will get errors related to element type.

4.7.6 Notepad++ and Plugins

Installation
It's important to be able to see data clearly. Notepad++ has some plugins to make viewing
data much easier. This was written at Notepad++ 7.6.6

Download the 64 bit Notepad++ at this link https://notepad-plus-plus.org/ and run the
installer. We will now walk through how to install the plugins, JSTool (for JSON) and XML Tool
(for XML).

1. Update your Notepad++ to the latest version, if applicable

2. Go to Plugins -> Plugins Admin...
3. Look for XML Tools and JSTool and check the box next to them
4. Click Install and restart Notepad++
5. XML Tools and JSTool should be present in the Plugins menu now

387

Internal Use - Confidential

 Using XML Tools
Paste this sample XML into Notepad++:

<connection id="99baf219-c0a7-41fe-9392-45aa5d442a7d" name="My platform"

type="boomiapi"><field id="url" label="WSDL URL" value="" encryptedValueSet="false"
usesEncryption="false" useDefault="true" atomLevelValueSet="true"/><field id="username"
label="Username" value="thanh_nguyen1@dell.com" encryptedValueSet="false"
usesEncryption="false" useDefault="false" atomLevelValueSet="true"/><field id="password"
label="Password" encryptedValueSet="true" usesEncryption="true" useDefault="false"
atomLevelValueSet="true"/></connection>

As you can see, it's not that easy to read. Go Plugins -> XML Tools -> Pretty Print (XML only -
with line breaks) to see it in a more human readable format. You can explore the other
options if desired.

Using JS Tools
Paste this sample json into Notepad++:

{"glossary": {"title": "example glossary","GlossDiv": {"title": "S","GlossList": {"GlossEntry": {"ID":

"SGML","SortAs": "SGML","GlossTerm": "Standard Generalized Markup Language","Acronym":
"SGML","Abbrev": "ISO 8879:1986","GlossDef": {"para": "A meta-markup language, used to
create markup languages such as DocBook.","GlossSeeAlso": ["GML", "XML"]},"GlossSee":
"markup"}}}}}

Go Plugins -> JSTool -> JSFormat to see it in a more human readable format. The JSON Viewer
option will give you a navigator to move around JSON data easier if there is a lot of data. You
can explore the other options if desired.

Helpful Things in Notepad++

• CRTL+H - find and replace option, search mode as "Extended" is helpful to look for
whitespace characters like newline (\n), carriage return (\r), etc
• CRTL+F - find option to look for text and unicode symbols

388

Internal Use - Confidential

 • Language menu - the Language menu option at the top lets you pick a language the
data is in and will color code it for further ease of reading
• Move to Other View - right click on a tab in notepad++ and you'll have an option called
"Move to Other View". This will move a tab to the other side of the interface so you can
see two documents side by side
• Show all Characters - there is a menu button that looks like this ¶, it shows all of the
characters currently on screen. This is a way to see special characters like newline,
tabs, and other whitespace characters
You can view the other plugins in the Plugins Admin like Compare (tool to compare tabs of
data in Notepad++) and use any that are helpful to you.

4.8 Common Integration Patterns

Integration projects involve the movement of data between application, the transformation
of data and execution of business processes that can include business rules etc.

When considering implementing integration, there are several widely known common patterns
that are typically used by many organizations and provided by middleware technology.

4.8.1 Incremental Poll (ETL Style Data Synchronization)

This integration pattern defines that the Boomi process is the initiator and is usually
scheduled to run on a job queue on a predefined period, commonly known as the polling
frequency. For example, run the process every 5 minutes between the hours of 8am to 8pm.

The process queries the source application for data that meets the query criteria and in this
case:

• Get all data that has been modified since the last time the Boomi process
ran.

Figure 208: Example >> Last Modified Date

389

Internal Use - Confidential

 For example, if the last ran date for Boomi is 2016/01/01 12:00:01, then two records will be
retrieved.

The query could also be augmented with the criteria to restrict the result set further:

• Fetch all data in block of 1000 records

• Get data that meets criteria identified by specific values, e.g. “ SELECT *
from Order WHERE Status = ‘NEW’ ”

The source application returns the results back to the Atom for processing, transformation of
the data to the specification of the target will follow as well invoking any business rules
specified in the processing logic that will affect the end results. The data is then sent to the
target system for processing and upon successful acknowledgement from the target system,
the Boomi process updates the “Last Run Date”.

Figure 209: Increment Poll (ETL Style)

390

Internal Use - Confidential

 4.8.2 Payload Driven Integration (Real-time, Synchronous, Record Payload)

This type of pattern is commonly used and is initiated by the source system. The source
system is configured to recognise data changes within its application and then retracts by
calling a listening type service on the Boomi platform. The Boomi service is configured to
receive the data, apply business rules and transformational logic to the data received, and
then send it to the target system(s).

Figure 210: Real Time (Synchronous, Record Payload)

391

Internal Use - Confidential

 4.8.3 Event Driven Integration (Real-time, Synchronous, Event Payload)

This pattern is a variation of the above pattern where the responsibility of fetching the data
is handled by the Boomi platform. The advantage of this pattern is the integration process
fetches only the data that is required to be processed. For example, an opportunity record in
a SalesForce Automation system could contain 300 fields, where the integration only requires
a subset of these fields to be processed and consumed by the target system.

Figure 211: Real Time (Synchronous, Payload[Msg=ID])

392

Internal Use - Confidential

 4.8.4 “Fire & Forget” Integration (Asynchronous)

The Fire & Forget integration pattern is similar and could be referred to as a publish and
subscribe pattern. The source system detects changes in its application and broadcasts these
changes to a queue, or calls a service that, in effect, publishes to a process that consumes
the event. The integration process status does not return to the application that published
the event.

Figure 212: Real Time (Asynchronous, "Fire & Forget")

393

Internal Use - Confidential

 4.8.5 Call-back Integration (Asynchronous)

The Call-back Integration pattern is typically used when there is a potential delay for a
response message or results to be returned to the calling application. The integration process
sends the request to the destination application and provides a mechanism for the application
to return an update or status to the caller.

Figure 213: Real Time (Asynchronous, Callback)

394

Internal Use - Confidential

 4.8.6 Accumulate-then-Process Integration

The “Accumulate-then-Process” integration pattern is typically used to optimize the

processing of source data in batches. An intermediate data store is required to hold the data
in either a database table or in a message queue until a threshold level or processing time is
triggered. For example, the intermediate data store is processed every day at 1am in the
morning or when the message queue reaches 1000 records.

Figure 214: Accumulate-then-Process

395

Internal Use - Confidential

 4.8.7 Decoupling Processing and Transmission

Process Decoupling Section

Figure 215: Decoupling Processing and Transmission

396

Internal Use - Confidential

 4.9 Know Your Integration Load
Do not be the cause of system degradation and poor performance!

Knowing your integration load will help with choosing the appropriate approach and
integration strategy as well as ensuring that you respect and do not impact the performance
and service levels of the systems that are to be integrated. For example, certain system
query 1 operations may perform a read operation with a lock on the records. The record
cannot be updated until the local is released, which prevents reading of dirty data. This could
become an issue if the lock is kept open by the requestor/read operation for longer than is
required, and where another process is waiting for the lock.

How much data are you handling?

There are two methods of data movement between one system and another:

• One-Way Synchronization – a source system (master) pushes data from its system to
one or many (secondary) systems,

• Two-Way Synchronization – the systems act as peers and data from one system is sent
to another and vice-versa 2.

The volume and frequency of data that is going to be synchronized will affect the choice of
integration strategy. The following section below details the strategies and provides
advantages and disadvantage of using them.

4.9.1 One-Way Sync Strategies

1
On certain databases, a SQL operation such as “Select * from ORDER” results in a SHARED LOCK which
may prevent other clients from updating the records until the request has finished with the operation.
2
Note: Typically, one system will be responsible for certain fields {customer name, date-of-birth,
address} and another system will be responsible for a different set of fields {demographic details, opt-
in, opt-out preferences, etc.} which are swapped between the two systems.

397

Internal Use - Confidential

 When it comes to integration, not all applications are created equally. Depending on the
availability and capabilities of an API and the configurability of the application itself, your
sync options may be limited. However, if you do have a choice, use the recommendations
below to identify the best strategy for your situation.

The three strategies we will review are:

1. BEST: Extract by “Flag” Field Value
2. GOOD: Extract by Last Modified Date
3. LAST RESORT: Change Data Capture

Extract by “Flag” Field Value

In this approach, records are extracted based on the value of some status or “flag” field.
Upon successful completion, the integration updates this field to a different value, so it will
not be extracted again.

This option provides a lot of flexibility and can enable users to easily control the sync
behavior by modifying values within the end application. Bottom line: if you have this option,
use it.

Considerations
• The flag field can take on many different forms and will depend on the application. It
could be an actual “status” field (integration-specific or business-related), a
true/false “ready to sync” field, the existence of an “external ID”, etc. It could also
be a combination of fields and values.
o Tip: If the application allows it, create a custom field specifically for
“integration status”. If not, look for a standard “status” field or even an
unused field that could be used for this purpose.
• If flag field value is not set “naturally” as part of the normal business process, you
may need to implement custom workflow logic in the end application to initialize the
value accordingly.
• Ideal for when a record should be synced once and only once, such as transactions.

398

Internal Use - Confidential

 • If you need to sync subsequent changes as well--which is common for “master data”
records like Customer or Contact--consider combining with the last modified date
approach below or with application logic to detect end-user changes using scripting or
triggers within the application.

Advantages
• Allows records to be synced indeterminate of one another. If one record in the group
fails, the others can be updated successfully and will not be reprocessed.
• Assuming the field is exposed to end users, it allows them to easily reset, retry, or
ignore individual records in a self-service fashion instead of relying on the integration
team.
• Keeps the integration stateless, meaning nothing is stored at the integration layer such
as the last run date.

Disadvantages
• If the source record does not have a field to use as the flag and you cannot create
custom fields, or the application’s API does not permit you to query by this field, this
approach is not an option.

Taking it to AtomSphere
• Configure the operation query filters in the start step to only select records with
certain values for the flag field(s).
• After successfully syncing the record, ensure the process updates the flag fields in the
source application appropriately.
o Tip: Use a User Defined Document Property to retain the source record’s
internal ID throughout the process so you can update the original record at the
end of the process.

Extract by Last Modified Date

399

Internal Use - Confidential

 Many applications automatically capture the timestamp of when records are created or
modified. This is great news when you only want to get records that have been created or
modified since the last time the integration ran.

In this approach, records are extracted based on the value of their last modified date field
being greater than some date, such as the last time the integration ran, or the most recent
record synced previously.

This is the next best option and may be the only option if there are no flag fields available in
the source application or it’s API.

Considerations
• What date value should be used when extracting records?
1. Capture the current system timestamp when the integration begins and save it
upon successful completion. However, if the integration client does not run on
the same server as the application (which is almost always the case for cloud
applications), there can be small discrepancies in the timestamps due to server
times, opening the possibility to overlook records.
2. Use a relative date, such as “extract changes made within the last 24 hours”.
However, this can be problematic a) by syncing the same records multiple
times if the integration runs more than once within the relative date range and
b) records will be missed if the integration does not run for an extended period
of time (e.g. server maintenance). This option is highly discouraged.
3. The most robust strategy is to capture the most recent last modified date from
the records themselves and persist that value upon successful completion.
Because the timestamp is from the source application itself you can be sure no
records will be missed due to server time differences.
• It is important to be vigilant in monitoring and correcting errored records to prevent
the number of records being resynced from accumulating too high.

Advantages
• Integration design usually simpler than flag field approach because you do not need to
update source application at the end of the sync.
• No end-application customizations or logic necessary.

400

Internal Use - Confidential

 Disadvantages
• May require end-application customizations or logic.
• Makes the integration “stateful”. Some troubleshooting efforts may require modifying
the last record date captured within the integration.
• Because the timestamp should only be persisted upon successful completion of the
entire batch, if one record fails, all the records will be extracted again the next time
the sync runs. This means you will need logic in the integration to determine if a
record has already been synced (for example, querying the destination application to
check if a record already exists).
o Tip: If the destination application supports an “upsert” operation and
overwriting the records with the same values is not a problem, you can skip the
existence logic to simplify the integration.
• Some applications may only provide the last modified date and not date AND TIME.
This means you must extract records modified since the beginning of that day. In this
situation, adapt your integration logic to either perform existence lookups against the
destination application or simply update the records repeatedly. Although undesirable
to sync the records unnecessarily, this latter approach is often sufficient for
infrequent (e.g. daily), lower-volume integrations.
• If mass updates are performed in the source application as part of normal business
processes, this will result in all records being synced the next time the integration
runs.

Taking it to AtomSphere
• If capturing the date from the records themselves (preferred), use a Map function to
evaluate and save the most recent date encountered as a Process Property. Then at
the very end of the process, if there were no errors or warnings, use a Set Properties
step to persist the Process Property.
• If using the process’ built-in Last Run Date or Last Successful Run Date, you don’t need
to do anything other than ensure the process status is successful/failed appropriately
(e.g. use an Exception step if it reaches a point where you do not want the Last
Successful Run Date to be updated).
• Be aware that using the Retry Documents feature within Process Monitoring to resync
failed documents will result in the persisted date being updated which will cause

401

Internal Use - Confidential

 additional records to be resynced or missed. For example, when capturing the date
from the record itself, if you retry a document from last week, upon successful
completion it will store that date and the next time the sync runs, it will extract all
records since last week. If this is a concern, avoid using Retry Documents and always
correct the records in the source application to have them to be extracted during the
next sync.

Change Data Capture (CDC)

But what if you can't use a flag field and there’s no last modified date and the application or
API only gives you the entire data set every time.

In this option, the integration must remember the entire data set from the last execution in a
“cache” to be able to compare the current data set against it. Through this comparison the
records that were added, modified, or removed can be identified.

Considerations
• Fortunately, this situation is rare for modern application-based scenarios and is more
common in working with legacy applications and file-based integrations, such as a
nightly product catalog extract for example.
• Need to ensure the entire data set is obtained from the source application every time.
• Should only be used for “master data” records, not transactional records.

Advantages
• No end-application customizations or logic necessary.

Disadvantages
• Makes the integration very stateful because the entire data set is stored in the
integration.
• Requires the extraction and processing of the entire data set, which could be a very
large number of records. This will typically take a longer time to run.
• The cache can be “corrupted” if a “bad” data set is processed, with unintentional
results. For example, if you accidentally synced a small “test” data set against your

402

Internal Use - Confidential

 full production cache, it would look like a large number of records were deleted which
could be very bad for the destination application.

Taking it to AtomSphere
• Ensure the start step operation is configured to retrieve the entire data set every
time, whether this is an application connector query or a file-based connector.
• Use the Find Changes step to determine which records should be added, updated, or
deleted in the destination application.

4.9.2 Two-Way Sync Strategies (Bi-directional Sync)

You might often hear the term bidirectional or two-way sync used casually to describe a need
for records to be exchanged between two applications. In many cases though, this translates
to some types of records need to go from Application A to Application B and other types of
records need to go from B to A. However, integration developers are keen to recognize this as
simply a collection of unrelated one-way syncs. A true two-way sync situation is when changes
to the same record type are made in either of two applications and must be synced to the
other application.
Although practically speaking a two-way sync is in fact two one-way syncs between the same
applications, the fact that both applications can update the same records back and forth
presents a few unique challenges. This situation more commonly occurs with master data
records (e.g. customers, contacts, vendors, etc.) as opposed to transactions (e.g. invoices,
payments, etc.) that are typically synced one-way accordingly to business processes.

Challenge 1: Update Conflicts

If a record can be modified in either application, what happens when the same record is
modified in both applications between syncs? Which application’s version should “win”? This
problem is addressed by choosing one of the applications as the “master”, whose version will
always overwrite that of the secondary application.

403

Internal Use - Confidential

 Again, a two-way sync is just a pair of one-way syncs working in concert, however, depending
on the capabilities of the applications themselves, the sequence in which those two syncs
should execute will vary.
If the master application has the ability to determine if the submitted update is newer than
the version it currently has (typically via an application trigger and/or requiring a previous
version number be included in the request), records should be extracted from the secondary
application first and “optimistically” synced to the master application. The master will
reject/skip any updates that are older than its version. (More on triggers later.)

Figure 216: Two Way > Secondary to Master

If the master application is unable to perform logic to reject stale updates, then records
should be extracted from the master application first. Any conflicting changes made in the
secondary application will be overwritten by the values from the master, then any other
records that were only modified in the secondary application will be synced back to the
master.

Figure 217: Two Way > Master to Secondary

Some may argue why not just query the destination record to compare its last modified date
to the record from the source application? While technically possible, one of the design
tenants you should always strive for is to minimize external API calls whenever possible for
performance (i.e. external calls are slow) and governance (i.e. some applications may limit

404

Internal Use - Confidential

 the number of calls you can make in a given period of time) reasons. In reality, doing these
sorts of lookups is only practical when dealing with very low volumes.

A Note about Field-Level vs. Record-Level Conflicts

Synchronization use case requirements often state “only the fields that have changed should
be synced”. This is a perfectly legitimate request however attempting to sync changes at a
field-by-field level is not a trivial task and typically requires the use of master data-
management software or similar logic to maintain a cached version of the records outside of
either application and perform change detection logic on each field.

More often than not, this is overkill for many integration scenarios and consequently they
operate at the record level: if any field on a record changed, all the values from that record
will be synced to the other application. That is the assumption we will make in this article.

But keep this in mind: even though changes are detected at a record level, each application
can be the master for a subset of the fields on a record. For example, between a customer
relationship management (CRM) application and an accounting application, the CRM could
“own” the customer name, shipping info, and other sales-related information, but the
accounting system could own the billing address and payment terms. This manifests in the
integration by simply not mapping certain fields when performing an update from CRM to
accounting or vice versa.

Challenge 2: Circular Updates

The second and trickier challenge when syncing the same records between the same
applications is the dilemma of circular updates. Assuming each step of the two-way sync is
extracting records incrementally (as it should be!), those records could have been created or
recently modified by integration itself and continually update one another without any “real”
changes made by an end user. Consider the following:

405

Internal Use - Confidential

 1. A user makes a change to a record in application A, updating that record’s last
modified date.
2. The integration looks for recently modified records in application A, finds this record
and syncs it to application B, updating the record’s last modified date in application B.
3. The integration then looks for recently modified records in application B, finds the
record that the previous sync just updated, and syncs it back to application A,
updating the record’s last modified date in application A.
4. The next time the integration runs, it looks for recently modified changes in
application A, finds the same record again, syncs it to application B....thus perpetually
syncing this same record back and forth until the end of time.

The theoretical conclusion is that eventually you would be syncing every record every time,
which is what we’re trying to avoid with incremental syncs in the first place!

To overcome this dilemma, we need a way to somehow identify and sync record modifications
made by a real end user and ignore modifications made by the integration itself. Below are
two methods for doing just that.

Method 1: Skip Records Last Updated By a Dedicated “Integration User”

This method relies on configuring the integration with its own user credentials and ignoring
records that were last modified by that user. When the integration creates or updates a
record, the destination application captures the integration’s user name as the last-modified-
by user value. This approach would be used in combination with the last modified date to
extract records WHERE LastModifiedDate is greater than <last record date1> AND
LastModifiedBy is not <Integration user>

1 The most recent last modified date captured and persisted from the records processed
during the previous sync.

406

Internal Use - Confidential

 Considerations
• The end application must capture the last modified by user and expose it via API
queries.

Advantages
• Simple to implement.
• No end application customizations required.

Disadvantages
• Provisioning a dedicated user for the integration typically consumes a user license and
associated cost.
• No ability to detect if an end user has more recently updated the record in the
destination application and skip the update if desired.
• Changes to specific fields in the destination application that are not mapped from the
source application will not be synced back to the source application (at least not
immediately).

Taking it to Integration
• Configure the Start Step’s connector Operation with a query filter for the appropriate
“last modified by” field and operator “not equal to”. In the Start Step Parameters tab,
enter the user name or internal ID of the dedicated integration user as a Static Value.

Method 2: Use Application Triggers to Distinguish Update Made by End

User vs. Integration

This method enhances the Sync Flag approach introduced in One-Way Sync Strategies
(whereby the integration extracts records based on the value of this flag field and then resets
the flag at the successful completion of the integration) by incorporating trigger logic in the
end applications. Triggers enable you to perform conditional logic within the destination

407

Internal Use - Confidential

 application right before the record is saved. This is much more efficient than making a series
of API calls during the integration itself. The trigger logic detects the “context” of a given
record update, whether from a real end user or from a programmatic/API user and sets value
of the Sync Flag field accordingly. Then the integration can simply extract all records where
Sync Flag=true.

This is similar to the Sync Flag approach used in one-way incremental syncs, however with the
added requirement that a record may be synced multiple times. This is often the case for
master data records like Customers, Vendors, Items, etc. vs. transactional records that are
usually synced once and only once. The feasibility and implementation of this approach can
vary widely between applications.

Considerations
• Must be able to configure custom fields and triggers in end application(s)

Advantages
• Does not require additional user licensing cost.
• Allows you to determine if an end user has updated a record more recently and
proceed differently.

Disadvantages
• More complex solution, with moving parts in both the integration and application
layers.
• Requires end application customization.
• Not all applications support user-defined triggers.

Taking it to Integration
• The Connector Operation filters are simplified to only query records where the Sync
Flag=true.
• Field mapping must be coordinated to populate custom fields that assist the
integration and trigger.

408

Internal Use - Confidential

 Sync Trigger Example

Let’s take a closer look at how triggers can be used to facilitate the sync. In this example we
will assume that we can implement triggers in both applications and that the triggers execute
on create and update requests, from both the user interface and API. We will also need to
create two custom fields on the given record type in both applications to assist with the sync:

• Sync Flag - boolean (true, false)

• Last Sync Date - date

Here’s the big idea:

• When the integration updates a record, it will always populate Last Sync Date with a
new value.
o Capture the most recent Last Modified Date and use it as the Last Sync Date
when writing back to the source application at the end of the sync. Using this
value instead of the current system time stamp will let the trigger precisely
determine if the record was updated by another user while the sync was
running.
• However, when an end user updates a record, they will not modify (or even see) the
Last Sync Date field.
• Before a record update is saved, the trigger logic compares the before and after
values of the Last Sync Date field. If they are the same (or if the record’s Last
Modified Date is greater than the new Last Sync Date), assume an end user made the
change and set the SyncFlag=true. If they are different, assume the integration made
the change and set the SyncFlag=false.

Or if you prefer pictures, here’s the flow representing the first half of the two-way sync, from
App A to App B only (the opposite sync from App B to App A would simply be the reverse of
this flow):

409

Internal Use - Confidential

 Figure 218: Flow Representing First Half of Two-way Sync

4.10 Testing Processes

4.10.1 Overview

When planning test cases, it is important to keep in mind the objective of the particular test,
which may be to verify the configuration/process within the process, connectivity to
endpoints, expected interaction with an external application, expected output, system load,
etc. Different test cases are needed to test different aspects of the integrations. Also keep in
mind the objective should be to test your specific integration requirements and configuration,
not the underlying AtomSphere platform functionality because that is verified by Boomi.

4.10.2 Designing Testable Processes

To facilitate repeatable testing, use a modular, subprocess-oriented design to encapsulate

integration process logic into isolated units of work that can be tested and verified
independently. This allows you to perform testing of the core orchestration logic without
concerning yourself with variables such connectivity and external data/application state,

410

Internal Use - Confidential

 which is especially useful for web service or any listener processes. By encapsulating the core
integration logic in a subprocess, it can be invoked directly without needing to go through the
web server or other listener framework.

A common conceptual breakdown of the logical pieces of a process might be:

Figure 219: Logic Breakdown of a Process

Below is a conceptual example of a web service listener process that invokes a subprocess to
perform the core processing logic. By encapsulating the core logic in a subprocess, it can be
invoked directly without having to submit an actual web service request through the Atom
web server.

Figure 220: Conceptual Web Service Listener Process

4.10.3 Creating Test Harness Processes

A “test harness” can be created by developing a simple integration process that invokes the
subprocesses or other components to be tested. The test harness process performs the
following tasks:

411

Internal Use - Confidential

 1. Obtain/generate known test data and setting state accordingly. This may involve
retrieving static test data files via connector or embedding within Message steps in the
test harness process itself. Any process or document properties that the subprocess
may expect will be initialized in the test harness process.
2. Invoke the unit of work to be tested, i.e., call the subprocess with the test data.
3. Confirm the results against a known set of values. This typically involves either
comparing the literal results (i.e., Return Documents) of the subprocess along with any
process or document properties that may have been set or altered by the subprocess.
The comparison or “assertion” can be performed by a Decision or Business Rules step.
“Negative” tests may result in the absence of data returned or the throwing of an
exception, so these outcomes should be incorporated in the test harness design.

Keeping with the web service example introduced in the previous section, here is an example
of simple test harness that invokes the same subprocess containing the core processing logic.

Figure 221: Simple Test Harness

The results returned from that process can then be inspected against a known set of values.
Here is a simple example:

412

Internal Use - Confidential

 Figure 222: Test Set Against Known Values

The steps used to verify the result will depend on the objective of the test case and specifics
of the integration process. A more complex verification process might use a Business Rules
step to perform multiple inspections of the result document’s values, or even perform queries
to external applications to confirm records or files were created as expected.

4.10.4 Advanced Test Harness Process Ideas

If the test intends to actually interact with an external application or relies on external data
to be available for the test case, it is often necessary to perform initialization logic before
running the test cases to generate test data. One recommendation is to perform any cleanup
from previous tests as part of the initialization logic instead of at the end of the test process.
This allows the records and application state to be preserved in the external applications for
analysis after running the test.

Another technique is to incorporate multiple test cases into a single process to verify multiple
use cases in a single process execution. With this approach the individual verification can
write to the process log and a set process property flag if failures are detected. Then after
executing the various test cases, the test harness process can inspect the process property
and throw an exception for visibility.

Below is an example of a test harness process that performs initialization logic, executes
multiple test cases, and checks for any failures.

413

Internal Use - Confidential

 Figure 223: Test Harness Process

4.10.5 Creating Mock Web Service Endpoints

When your destination is a web service endpoint, consider developing AtomSphere web
service processes to simulate the functional interaction of the real endpoint. Mock endpoint
connections are typically used during initial development and volume testing to validate the
behavior of your primary process without sending data to the real endpoint. Not actually
sending data can be especially desirable for high volume testing.

414

Internal Use - Confidential

 Ideas and Considerations

• Design the mock endpoint processes to simulate the behavior of the real endpoint with
respect to request and response payloads and HTTP response codes.
• Create different mock endpoint processes or allow parameters in your design to
achieve different outcomes, such as:
o Technical success (e.g. HTTP 200) and functional success (e.g. no SOAP
fault/application warning in response payload)
o Technical success (e.g. HTTP 200) but functional failure (e.g. SOAP
fault/application warning present in response payload)
o Technical failure (e.g. non-HTTP 200)
• If used during performance testing, keep in mind the response time of the mock
endpoints will likely be different than the real endpoint. The focus of the test should
be the execution duration of the primary process itself independent of the connector
call.
• Replace the mock endpoint connections with the real connections before deploying to
your main change control environment. Alternatively, you could design the process
with both the real and mock connections and use a decision step (perhaps evaluating
an extended Process Property) to determine which connector step execute.

Example mock endpoint web service that expects a custom HTTP header parameter to
determine what type of response to return: success, warning, error.

Figure 224: Mock Endpoint Web Service

415

Internal Use - Confidential

 4.10.6 Automating Testing

To automate testing, the test harness processes can be organized into a “test suite” and
executed sequentially by creating a simple “master” process that invokes each of the test
harness processes in succession. Test suites can be organized by interface or project to
provide granularity for testing and deployment.

After a round of development changes, the master test suite processes can be deployed to
the test environment and executed manually. Alternatively, they could be scheduled execute
automatically, but the remember the test processes will still need to be redeployed to pick
up the new configuration changes.

Another approach for automating testing especially when publishing web services is to use an
external testing tool such as SoapUI. Test cases and suites can be configured to invoke
Integration web services processes with various payloads and execute parallel calls for load
testing. With some creativity you can use test harness and utility processes to orchestrate and
verify non-web services integrations.

For example, suppose you have a process that extracts known test records from a database
and sends them to a web service application. To automate

1. Execute the process through the Execute Process API or perhaps invoke a web service
"wrapper" test harness process that invokes the process to test.
2. Invoke a second utility web service process that queries the destination application
and returns the results once available.
3. Perform the comparison with SoapUI against a known result set.

4.10.7 Using Boomi Assure for Platform Regression Testing

Automating the test cases greatly reduces the effort required for regression testing of both
process configuration changes as well as the AtomSphere platform updates. AtomSphere
updates are released monthly so reducing effort for regression testing is important. Greater
regression coverage increases confidence for each release.

The Boomi Assure framework is another means to increase regression coverage.

416

Internal Use - Confidential

 Your use of Boomi Assure provides an additional level of test case coverage for Boomi to
include in its new release regression testing. It is intended to identify regression issues
between AtomSphere platform updates. It does not identify regression issues between
different versions of YOUR process development. It should be considered an additional piece
of the regression testing toolkit, not a complete substitute for regression testing of your
integration processes.

Boomi Assure is included within the AtomSphere platform and allows customers to record and
submit process test cases to help Boomi ensure consistency across platform releases. With
Boomi Assure, a Test Mode execution is recorded and then can be replayed by Boomi as part
of its monthly testing to ensure the same results are obtained with the new AtomSphere code.

A few key concepts about Boomi Assure:

• It is a voluntary, opt-in feature.

• The record captures document data that is processed. Care should be made that no
sensitive customer data is used for the purposes of the Boomi Assure recording.
• Customers cannot run the Boomi Assure tests and are not notified of results. If a
regression is encountered, Boomi will address it internally ahead of the release or
communicate a mitigation plan in rare cases.
• Boomi Assure does not currently cover 100% of the steps or functionality within
processes, most notably connectors. Because Boomi does not have access to your
endpoints, it records the inputs/outputs of any connector calls made during the test
mode execution and uses those recordings for the regression tests.

4.11 HTTP Concepts

4.11.1 Requests
HTTP stands for HyperText Transfer Protocol and HTTPS stands for Hyper Text Transfer
Protocol Secure. HTTPS keeps data secure by using SSL or TSL encryption. HTTP is not secure.
They are protocols, sets of rules for how data is exchanged on the internet between
machines. Since HTTP/HTTPS is the how to do something, HTTP requests are the what, the
result of all the rules. Common parties involved are the client, who requests resources/data

417

Internal Use - Confidential

 (your web browser/computer); the web server, who has the information and serves it to the
client; the proxy, who is in front of the client or web server and receives the HTTP request
first before sending that request (modified or not) to the internet or before sending to the
web server.

Here is an example of a GET HTTP request (you can copy and paste
https://itunes.apple.com/search?term=Francis+Ford+Coppola&country=us&limit=10 into your
web browser to see the result):

Figure 225: GET HTTP

Modified for a POST:

Figure 226: POST HTTP

An HTTP request is made up of a couple of common parts:

1. HTTP Method - a verb like GET or POST saying what operation the client wants to do,
i.e. GET
2. Resource Path - the location of a resource displayed after the HTTP method, i.e.
/search
3. Parameters and Query strings - after the resource path you can have additional
information in the URL. The "?" indicates a query string and includes additional

418

Internal Use - Confidential

 information to be used by the web server. Query strings are always part of the URL. The
";" indicates a parameter which can be part of the URL or the HTTP request. There can
be multiple parameters and query strings, usually separated by an & and are in name,
value (ex. "country" is the name and "us" is the value) pairs i.e.
term=Francis+Ford+Coppola&country=us&limit=10
4. Request Headers - they convey more information as part of the request. It always has
the format of Header-name: Header-value, i.e. Accept, Content-Type, Authorization,
etc
5. Payload (aka the request body) - the information sent with the request, i.e. the XML
data
Whether a GET, POST, or another method or any part of the HTTP request works or not is up
to the server to define. For example, a server can choose to ignore POST request or process it
like a GET, it is defined by them.

The HTTP Response:

Figure 227: HTTP Response

Every HTTP request should get a corresponding HTTP response, even if it is a failure.

Some common components of an HTTP response:

419

Internal Use - Confidential

 1. Status Code - to indicate if the request was successful or not and why. There's a
standard list of HTTP status codes i.e. 200
2. Status Message - a short description of the status code, i.e. OK
3. Response Headers - to show more information about the response, same format as
request headers
4. Response Body - the actual data fetched from the web server. This is usually what
people want and care about the most, i.e. the JSON data present

4.11.2 Request Components

An HTTP Request/Response is made up of several common components. We'll explore them
more here.

Method
The most common HTTP methods are GET and POST. The main difference is GET requests do
not send a body and POST requests do send some data in the body. The GET usually just asks
for some information and everything it needs is in the URL/resource path and headers. A
POST sends in an additional body to be used by the web server.

Do not confuse the connector action in Boomi with the HTTP Method. The connector action is
a boomi specific concept. GET does not allow data to be sent to the HTTP connector and a
SEND connector action allows data to be sent. Both actions can use the same methods, but
because of how they work, it would be pointless to use a POST method with a GET connector
action since it would nullify the body. Using a GET method with the GET action would
suffice.

Other common methods include PUT or DELETE. A more comprehensive list can be found
online.

URL, Resource Path, and Parameters

The URL is the web address (usually just the .com/domain part) of where you want to go to
get your information, e.g. itunes.apple.com (you'll notice that's the Host header in the
request headers).

420

Internal Use - Confidential

 The resource path is the specific resource's location on the server since a server can have as
many resources as it wants, so you have to know where a particular resource is. In this case
the resource path is /search which allows you to search the iTunes database of information.
There is always a / between the URL and the first resource path. Whether there are more
slashes or not, is up to the web server to define.

The parameters include information in the URL for the web server to use. For our purposes,
they can be considered as part of the full path and it should be in some format defined by the
web server. For support, it is another part of the URL to retrieve specific information.

The full URL:

https://itunes.apple.com/search?term=Francis+Ford+Coppola&country=us&limit=50
The web server: https://itunes.apple.com
The resource path: /search?term=Francis+Ford+Coppola&country=us&limit=50
The parameters: ?term=Francis+Ford+Coppola&country=us&limit=50

Headers
Request and response headers are used to convey information to the web server. Some are
more important and necessary than others. Some common ones are:

1. Authorization - to convey who's accessing the server

2. Content-Type - to say what kind of data is being sent
3. Accept - to say what kinds of data are accepted
Some headers are not standard and are required by the web server, others will be sent
automatically by the software you are using and will not matter to the web server. You can
send any header and value in it, whether it is accepted or not is up to the web server.

Status Codes
If a request is successful, it will send back a status code of 200 (or one of the 200's). If it is
not, a different one will be sent back for a variety of reasons. Here is a list of the common
ones:

421

Internal Use - Confidential

 2XX Success

• 200 OK
• 201 Created
• 202 Accepted
4XX Client Error

• 400 Bad Request - the body or format of the HTTP request is not what the server wants
• 401 Unauthorized - the Authorization header or credentials are wrong
• 403 Forbidden - the authentication is not allowed to use or access this resource/data
• 404 Not Found - wherever you are going does not exist
• 405 Method Not Allowed - you are using the wrong HTTP method
5XX Server Error

• 500 Internal Server Error - something wrong in the server or it did not know what to do
with your request
• 503 Service Unavailable - the server is not available right now, it might be busy or
rejecting your requests due to limited resources
You can look up others as you see them.

4.11.3 Connector
HTTP requests can be sent via the HTTP connector in the platform. The HTTP connector is
made up of two components, the HTTP connection and the HTTP operation.

HTTP Connection
The HTTP connection tells the request where to go. URL is the address of the server you are
going to. For example, https://itunes.apple.com.

The Authorization determines the security information sent with the HTTP request. Selecting
"None" will not generate an Authorization header. Basic will automatically generate an
Authorization header using the user name and password to calculate a unique value. The
Authorization header value is generated by base64 encoding the username and password in
the following format, username:password. This does mean having the Authorization header is
basically the same as having the username and password and can be used without knowing the

422

Internal Use - Confidential

 actual username/password since it will be the same. This makes replicating HTTP requests
easier.

Different authentication types will send the authentication in different ways. The web server
should define how they want a client to authenticate. Something like OAuth will ask for more
information.

HTTP Operation
The HTTP operation defines the rest of the HTTP request. It comes in two varieties depending
on the connector action:

Figure 228: HTTP Client Operation > Get

423

Internal Use - Confidential

 Figure 229: HTTP Client Operation > Send

The content type can be selected from a list or you can type in your own content type for the
data you are sending. This will generate a Content-Type header automatically. If you enter
something in that field and add a Content-Type header also, the added header will override
what you set in that field.

The HTTP Method can be selected. There's only a certain set currently supported on the
platform. You might be able to get around some through an override header, but only the
server supports it.

The request and response profile can be set if you plan to use a specific profile to send
information or use specific elements returned from the HTTP connector. Otherwise, it can be
left as none for testing, and then just see what the HTTP connector retrieves.

The GET action doesn't have an option to return the HTTP response since it automatically
returns it and the POST allows you the option to return and use it in the process or not.

424

Internal Use - Confidential

 • If you check the Return HTTP Errors option, and there's an error, that error response
will be returned and move along the process (there will be no error throwing to stop the
process)
• If you do not check the Return HTTP Errors option and an error occurs, the process will
fail and stop the process so nothing else happens
• If you check the Return HTTP Responses checkbox, the response will be passed along
into the process; if off, the response will not be returned and the data flowing into the
HTTP connector will keep going along the process
Follow redirects will let the HTTP request be redirected if the URL was set to redirect you
somewhere else.

Request Headers
The Request headers can be set in this field. The headers are a name and value pair. You can
make it a static value or make it dynamic via the replacement variable option. If you have a
body, you cannot define these replacement variables like you would for other connectors, it
will erase the body. You need to use a dynamic document property with method described in
the payload section.

If you use Basic Authentication in the connection, an Authorization header will be

automatically created. You can set your own Authorization header to override that generated
one. Of course, if you are planning on overriding it, might as well set the authentication to
none.

If you choose a Content Type, a Content-Type header will be automatically generated. If you
make one yourself, it will override that.

Resource Path
Resource paths are the rest of your full URL. The connector will add a / between the
connection URL and the first resource path so do not add a / in front of the first resource
path. For example, if you used /search when the connection URL is https://itunes.apple.com,
the final URL will be https://itunes.apple.com//search instead of the one / we usually see.
Some servers might handle that normally, others might not. Resource paths can use a static
value or replacement variables to make it dynamic. You have to break up the path according

425

Internal Use - Confidential

 to what's static and dynamic then. If you have a body, you need to use a dynamic document
property with method described in the payload section.

For example, here's how you could break down

"search?term=Francis+Ford+Coppola&country=us&limit=10" if you wanted to make the
"Francis+Ford+Coppola", "us", and "10" dynamic.

Figure 230: Resource Path

The "term", "country", and "limit" are dynamic document property (DDP) variable names and
the "Is replacement variable?" checkbox is checked for each to let the process know they are
dynamic.

If you know what your full path is going to be and you're only testing that, you can skip
defining the resource path altogether and just set the connection URL to that full path. If you
were always going to go
to https://itunes.apple.com/search?term=Francis+Ford+Coppola&country=us&limit=10, then
you can set that in the connection URL, and you do not need to break out the parts that might
be dynamic in the resource path.

The Body (Payload)

The body can be mimicked or constructed by a message shape, otherwise it can be the
current data that is flowing through the process. The HTTP connector will erase the body if
you set parameters in the connector shape (different from URL parameters). To get around
that, you must use dynamic document properties.

426

Internal Use - Confidential

 After you have set up the operation with replacement variables (we'll use the same example
up there and assume we're sending a body):

1. You need a set properties shape before the connector

2. The set properties shape needs to have DDPs with the same exact name as what you
used in the operation

Figure 231: Set Properties

3. The process will end up looking like the below:

Figure 232: Data Flow

The Response
The response will continue flowing through the process if you allow it to. You can get the
response's status code via a document property.

1. On the parameters side (such as of a set properties shape), add a "Document Property"
type

427

Internal Use - Confidential

 2. Click the magnifying glass to choose and then to go to "Meta Data"
3. Expand it and expand "Base"
4. Choose "Application”

Getting the HTTP application status message.

4.11.4 Installation

Postman
Make sure you are using the desktop version (not the chrome add-on) so you have the Postman
Console.

1. Go to https://www.getpostman.com/downloads/ (or equivalent if Postman changed

links)
2. Download the installer and install to your computer
3. Once installed, open and go to the "View" menu
4. Choose "Show Postman Console"
The Postman Console will show you most raw requests, so you can see what the actual data
is.

Postman doesn't show certain things super clearly like Charles Proxy, like how the URL encode
something or Oauth related calls.

SoapUI
SoapUI's program allows for REST and SOAP testing now, before it was only SOAP requests.
You can just use it for both.

1. Go to https://www.soapui.org/downloads/soapui.html and download the SoapUI Open

Source software
2. Install to your computer with the prompts

Charles Proxy
Warning: configuring Charles Proxy to work with an atom can affect the atom's internal
processing. You should only follow these steps on a local test atom

AtomSphere gives you the ability to track data sent to/from a connector through use of the

428

Internal Use - Confidential

 Test Mode feature. It provides you a simple view of the content that you want to work with in
your Process flow.

When troubleshooting complex issues that require the analysis of the raw messages sent
to/from a web application, it is helpful to review the web traffic sent to and from the atom.
A proxy tool such as Charles Proxy offers a way for you to track this information during your
debugging. The following steps outline how to enable Charles Proxy to see traffic that is being
sent securely from a local atom.

Setting up Charles Proxy

1. Download and install Charles Proxy (not the Firefox add-on)

2. In Charles, go to the Help menu and choose "SSL Proxying" > "Save Charles Root
Certificate..."
3. Save the root certificate (as a .cer) to your desktop (e.g.
C:\Users\<<user>>\Desktop\charles-proxy-ssl-proxying-certificate.cer) or somewhere where
you can easily access it in the next steps
a. When you select "Desktop", charles might not actually put it there but into your users
folder (e.g. C:\Users\<<your_user>>), so look there if you cannot find it where you
thought you placed it
b. Shift + Right click on the certificate file and choose "Copy as Path" to copy the path to a
text editor, you'll need it later
4. Determine which JRE the Atom is using by going to:
<atom_installation_directory>\.install4j\pref_jre.cfgand seeing what's the entry in the file
a. If there is no pref_jre.cfg, then look in the inst_jre.cfg located in the same directory
b. [optional, you can skip] If you wish to change the JRE that the atom is using, make a
copy of the inst_jre.cfg file and renamed the copied file to be pref_jre.cfg
c. [optional, you can skip] In the pref_jre.cfg file, change the file path to the JRE home
folder you want to use
5. Find the cacerts file. It should be in your <<JAVA_HOME>>/jre/lib/security/cacerts, where
<<JAVA_HOME>> is your java home directory or the JRE path found in step 4
a. In the example below the java home directory is "C:\Program Files\Java" so the cacerts
will be in "C:\Program Files\Java\jre8\lib\security\cacerts"
b. Shift + Right click on the cacerts file and choose "Copy as Path" to copy the path to a
text editor, you'll need it later

429

Internal Use - Confidential

 6. Import the Charles Root Certificate
a. Open the cmd window
b. Go to the bin folder (via cd) where the keytool command exists, usually in your
JAVA_HOME/jre/bin directory
c. Run the command below but adjusted for your certificate and cacert's location. I
recommend copying the below to notepad and change the paths with the locations
copied in step 3 and, then copy and paste the final result into the command prompt:

keytool -import -alias charles -file "C:\Users\guest_user\Desktop\charles-proxy-ssl-proxying-

certificate.cer" -keystore "C:\Program Files\Java\jre8\lib\security\cacerts" -storepass changeit

1.
a. Note: The *.cer filename may be different depending on the version of Charles Proxy
installed, if that is the case, change the name/path accordingly. The value "changeit" is
the default cacerts password. If you have modified it, change it accordingly. If you get a
permission denied error when you run the above command, you need to run the
command prompt as an administrator and try the import again.
b. The command prompt should show something similar to the following, if it is successful:
c. Trust this certificate?: yes
Certificate was added to keystore

Setting up the atom

1. Go to the atom.vmoptions file located

at <<atom_installation_directory>>\bin\atom.vmoptions
2. Add the following lines to the atom.vmoptions:
3. -Dhttp.proxyHost=localhost
4. -Dhttp.proxyPort=8888
5. -Dhttps.proxyHost=localhost
-Dhttps.proxyPort=8888

6. Save the file and restart the atom

7. If your atom has a proxy set up in the container.properties (located
in <<atom_installation_directory>>\conf), you have to remove or comment out the proxy
related lines

430

Internal Use - Confidential

 8. #com.boomi.container.proxyHost=...
9. #com.boomi.container.proxyPort=...
10. #com.boomi.container.proxyUser=...
11. #com.boomi.container.proxyPassword=...
#com.boomi.container.proxyPassword.encrypted=...

Configuring Charles Proxy

1. Start Charles Proxy and go to "Proxy" > "Proxy Settings"

2. On the "Proxies" tab, set HTTP Proxy port to "8888"
3. On Windows tab, uncheck "Enable Windows Proxy" and uncheck "Enable Windows Proxy at
startup"
4. Go to the "Proxy" > "SSL Proxying"
5. Check "Enable SSL Proxying"
6. Note: You will need to do this every time you want to see traffic to a different endpoint.
Add the domains for all the SSL Locations your processes will connect to. If you do not add
the domain to this list you will not be able to view the raw request/response data. For
more information, read the changes to new Charles Proxy Version for
SSL- http://www.charlesproxy.com/documentation/proxying/ssl-proxying/
7. To add an new endpoint, click "Add" then enter the top level domain name (e.g. for boomi
site add *.boomi.com; for google sites *.google.com). You can typically leave the port
number blank
8. Here is an example of common locations as a starting point. You can always add more
later:
9.
10. *.boomi.com
*.salesforce.com

11. Make sure browser proxies are turned off

12. Execute a process on the local atom and if configured properly, the communications will
be captured as an entry to the endpoint you configured before

431

Internal Use - Confidential

 4.12 Trading Partners
A trading partner is anyone you trade with. In order for them to send or receive anything from
you (using the Trading Partner component), you have to set up a profile first. When setting up
trading partners [profiles] it is important to have X12 sample data or at least the required
information to fill in a trading partner profile. Let's take this X12 850 document:

Figure 233: X12 850 Document

ISA06 is the sender ID and ISA08 is the receiver ID. When setting up the trading partners, you
have a "My Company" trading partner which is "you" and normal trading partners which are
other people you are trading with, "them". In the example document, ISA06 is 4532345856
(let's call them TP) so TP is sending to 8038025150 (let's call them My Company). If this is
case, you would use the receiver information to fill out the My Company profile information.
GS03 is receiver's code.

For the sender TP, you'll use the sender information, ISA06 and GS02 (sender's code) to fill out
the TP trading partner profile information. Many other fields are indicated in the profile
already to help you fill it out.

Note: for testing, you can actually make up most details for your TP and My Company as long
as the X12 document conforms to it. Lp

432

Internal Use - Confidential

 4.12.1 Communication Methods

The communication method is how the trading partner sends and receives EDI data. You'll
notice there's a Communication tab and has various methods of communication. A common on
is AS2 which involves its own protocols and setup. A common method for testing will be disk.

My Company can define communication methods for all of their trading partners to use

One set is for receiving, one set is for sending. There's a mycompany defaults and there's a
specific setting.

4.12.2 Receive and Send Process

Setting up the trading partner component

1. Make a new trading partner component

2. Give the component a name, choose X12 as the standard and "This is my company."
3. Fill in the X12 Standard information based on the X12 document in 7.1
4. Save and make a new trading partner component
5. Give the component a name, choose X12 as the standard and This is a partner that I
trade with."
6. Fill in the X12 Standard information based on the X12 document in 7.1
Reference for My Company and TP setup:

Setting up disk communication

1. In the Communication tab of the trading partner

2. Add a Disk communication method
3. Change Communication Settings to be "Configure Specific Settings" (what would "Use All
Defaults" do?)
4. There will be a Get option and a Send Option (when are they used during the process?)
5. For Get options, pick a directory and file settings
6. For Send options, pick a directory and any other options you want
7. Add Disk communication for My Company's communication methods as well, leave it
blank since your trading partner is using specific information

433

Internal Use - Confidential

 Setting up the process

1. Add the sample 850 document to the directory location you picked above
2. Add a trading partner shape to the acknowledgement path so the 997s go to your send
location
This is the basis of your trading partner tests and setup now. From here, you can test with
different X12 document, test different options to see what they do, etc. This will help you
mimic what the client is trying to do. It doesn't really matter how it's picked up (as long as it's
not related to that), once the document is picked up it is treated the same whether from Disk
or AS2. So, what can this help you troubleshoot, why? What can this not help you
troubleshoot, why?

Common Errors

1. Your Trading partner shape does not have My Company and TP defined
2. For file options do not match up to the file you put in the get location
3. Your ID has extra spaces when it shouldn't (because the platform already knows to add
extra spaces)

434

Internal Use - Confidential

 5 Deployments

5.1 Performance Considerations

• Minimize connector calls.
• Maximize batching.
• Pull data all together when possible (e.g. use joins on DB selects or SAAS queries when
available).
• Caching documents where appropriate.
• Function caching where appropriate (ex: cache by map).
• Eliminate use of “map function ordering” if it is not needed.
• High Volume processes.
• How many records per unit of time is this process expected to handle?
• Has it been tested at this volume?
• If high volume, is this process set up appropriately for high volume (caching,
parallelism, low latency, etc.)?
• Parallel processing if needed and appropriate.
• Minimize or eliminate the use of “run each document individually” feature for high-
volume processes. Batching where possible.
• Try/Catch Impacts

• If greater speed is required, consider handling errors without try/catch.

Try/Catch reads entire document sets into memory. For large volume
processes, this can impact performance.
• Limit retries if possible to improve performance. Try/Catch retries have
predefined wait times per retry and can slow processing, especially for
multiple retries within a process.

435

Internal Use - Confidential

 5.2 Requirement Satisfaction and Licensing (Prior to
Deployment)
• Have you notated satisfaction against the documented requirements, including
incremental changes during development that affected the build?
• Have you completed Unit Testing to confirm that requirements have been
implemented to your satisfaction? (developer)
• Have you completed User Acceptance Testing to the client’s satisfaction?
• Boomi Licensing
• Prior to process completion and deployment, verify that the client account has
enough Boomi licenses available for each deployment in the development
effort
• Consider license type when estimating license needs (Enterprise, Standard,
Test, EDI)
• Consider atom architecture (atoms, molecules, servers) and # of deployments
when estimating license needs
• Endpoint Licensing
• If endpoints have licenses (ex: NetSuite, SFDC, …), verify the client has enough
endpoint licenses for planned processing
• Reminder: parallel processing may drive concurrent connections and consume
additional licenses
• Consider atom architecture (atoms, molecules, servers) when estimating
licenses

5.3 Deployment Considerations

5.3.1 Full Testing Checks

• Has each process been tested?

• Has each logical condition been tested for proper function?
• For sequential processes, has end-to-end testing been completed?
• Have expected results been verified within the endpoints?

436

Internal Use - Confidential

 5.3.2 Load Testing Checks

• Has each process been tested for expected and peak loads?
• Is performance acceptable for expected and peak loads?
• Reminder: Load testing critical for high-volume processes

5.4 Component Versioning

Each component has its own independent revision history. Every time you save a change to a
component, the Revision number is incremented along with the user who changed it.

Figure 234: Revision History

You can restore (roll back) to a previous revision of the component by selecting a previous
version and saving the component. The user needs to be careful when restoring a component.

• Make sure other processes/components are not relying on the most recent version.
• Restoring a previous version will not delete any other components referenced by the
newer configuration.
• The “Show Where Used” feature should be used to see all the components that
reference a given component.

437

Internal Use - Confidential

 5.5 Use of Extensions
The best practice for configuring an extension is to set fake values in the underlying
components to protect against inadvertently connecting to the wrong endpoint if an extension
is accidentally not declared on a given process or overridden in Atom Management.

• Intentionally configure a placeholder value, such as “SET BY EXTENSION,” for the

actual connection or other to-be extended components. This enforces the best
practice of always extending the connection components and relying on environment
extensions for values. With this approach, if an extension is accidently not declared in
a given process, it will result in a connectivity exception instead of inadvertently
connecting to the wrong endpoint (for example, accidentally writing to production
during testing).

Figure 235: Set Extensions

• Extend all connections across all processes. However, only extend the connection
fields that will actually change between environments. For example, in the HTTP
Client connection above, the fields URL, User, and Password would be declared as

438

Internal Use - Confidential

 extensions but Authentication Type would likely not, assuming the web service uses
the same authentication scheme across environments.

Figure 236: Extensions >> Connection Settings

• However, to configure some types of connector operations, you’ll need to reference a

connection with actual credentials. Keep a separate copy of the connection
components strictly for browsing use (and name them accordingly). Make it a practice
to never deploy them.

When executing in Test Mode, you will need to provide values for the extensions. Extensions
are configured per Atom in the Run a Test dialog. You will need to enter values for a given
process the first time you run a test, but AtomSphere saves the extension values across test
mode executions.

Figure 237: Test Extensions

• Prior to the deployment of each process, ensure all needed fields have been
extended in each process (build tab)

439

Internal Use - Confidential

 • If deploying parent/sub process, make sure to extend all needed fields within
the parent process (build tab)
• Reminder: Extensions will not appear in atom management until at least one
process is deployed with that field extended
• For deployments with newly extended fields, verify that extension values have
been set after deployment (atom management)
• For deployments for previously extended fields, verify that extensions have
been activated and checked prior to deployment since those extensions will be
used immediately upon deployment (listeners, previously scheduled processes)
• If necessary, consider pausing listeners and schedules prior to deployments, especially
if extensions need to be edited prior to next execution
• When scheduling processes, consider impact of other processes executing at the same
time. Processes that execute at the same time may compete for atom resources,
endpoint connections, and file access.
• Promoting Processes Between Environments
• Consider: Dev, Test, and Prod Boomi environments
• Process deployed to Dev. When ready to promote to test, use the “Copy
Deployment” feature on the Deploy tab rather than directly deploying to Test
from the build tab. Similar steps when promoting from Test  Prod
• Look for upcoming “unified deployment” feature
• When deploying, add notes for each deployment

5.6 Deployment Procedures

5.6.1 Standard Deployment Procedures

The standard development and deployment procedures in Integration are as follows:

1. Perform development in Build tab.

440

Internal Use - Confidential

 2. Perform unit testing with limited test data via Test Mode. Set connection and other
extension values via Test Mode Extensions. These values are saved per process per
Atom.

3. Attach (first time only) and deploy the process to the first environment (i.e.
“Development”) using the Deploy Latest Revision of Process button. This creates a new
deployment version.

Figure 238: Deploy Latest Revision

4. Configure extensions for the process in the Development environment via Atom
Management.

5. Perform end-to-end testing in the Development environment.

6. Promote to the next environment (i.e. “QA”):

a. Attach the process to the next environment (first time only). Note that
eventually the process will be attached to all environments.

b. CRITICAL: DO NOT CLICK DEPLOY LATEST REVISION OF PROCESS

c. Select the current environment (e.g. Development).

d. Click the gear icon on the latest deployment version and choose Copy
Deployment.

441

Internal Use - Confidential

 Figure 239: Processes

e. Choose the next environment (e.g. if currently in “Development”, next

environment is “QA”). Enter relevant promotion notes.

Figure 240: Deployment Confirmation

442

Internal Use - Confidential

 f. Navigate to next environment to confirm new deployment version created.
Note this will have a different deployment version number (specific to that
environment). The Notes column will have an audit trail of the promotion.

7. Configure extensions for the new environment.

8. Perform end-to-end testing.

Repeat steps 6-8 for each environment through Production.

5.6.2 Important Deployment and Extension Concepts

• In general, only the top-level process should be deployed; the sub-processes should
not be deployed. When the top process is deployed, the sub-process is attached to
that deployment. The exception is if the sub-process should be able to run alone,
such as to manually reprocess a document via the Process Reporting UI.

• Similarly, extensions must be declared on the process that will be deployed. For
example, extensions for components contained within a parent process or any of its
sub-processes must be declared on the parent process.

• Extensions for the same component must be declared on each process that will be
deployed. For example, if two processes use the same database connection, the
extensions for the database’s host, user, and password and so on must be declared on
both processes. If not declared, it will use the default value, which should be the fake
“set by extension” value configured in the connection component by convention.

• Once an environment extension value is set (such as Manage > Atom Management >
Environment Extensions) by your deployment or operations team, it doesn’t need to be
reset if a new version of the same process is deployed to that environment or if a new
process is attached and deployed to that environment and declares the same
extension.

If a new process or version introduces a new extension to the environment, that would need
to be set.

443

Internal Use - Confidential

 5.7 Automating Deployments with the AtomSphere API
5.7.1 Overview

The AtomSphere platform provides a web service API (REST or SOAP) to perform process and
other component deployments and promotions programmatically that you can incorporate
into your automated change management procedure. Use of the deploy APIs requires
orchestration logic to be performed by a client application such as an internal scripted
application or even another Integration process.

The platform API is available free-of-charge to all AtomSphere customers and partners. You
can call the platform API from an external client application but there is also an AtomSphere
API connector which can be used to easily interact with the API from within a Integration
process. Note the platform API is separate from and available without the API Management
offering.

This section will assume the use of Environments and Extensions which are not available in all
editions. However, there are Atom-only versions of the attachments APIs that may be used
instead.

5.7.2 Usage Notes and Limitations

• The AtomSphere API connector can be used to interact with the API from within an
Integration process.
• Using the API provides an opportunity to devise business logic to enforce environment-
promotion sequencing if desired. For example, you could use decision to enforce that
a deployment targeting the "production" environment may only come from a
deployment that currently resides in the "stage" environment, not "test" or "dev".
• The digest value serves as the consistent value for comparing versions of a given
deployment across environments. Note the version and Deployment id will vary across
environments. You will use this to identify and validate the same deployment version.
• The digest is only returned by the Deployment GET action. It is not returned by the
QUERY action.
• When updating environment extension values via the API, the entire set of values must
be provided in the request; partial updates are not supported. This means the request

444

Internal Use - Confidential

 must contain values for all extended fields, not only ones that are to be changed. If an
extended field is omitted in the update, the value will be removed. The only
exception is encrypted password fields.
• The client application may wish to query and maintain a local copy of the list of
process and environment names and internal IDs for convenience.
• As of the January 2017 release, new deployments for additional component types such
as API, Certificate, and Process Route can be created via the API.
• As of the January 2017 release, use of the new generic
ComponentEnvironmentAttachment and ComponentAtomAttachment APIs are
preferred over the legacy ProcessEnvironmentAttachment and ProcessAtomAttachment
APIs.
• As of the June 2017 release, use of the new generic deployComponent API to
copy/promote any type of component from one environment to another is preferred
over the legacy deployProcess API.

5.7.3 Steps

Assumptions

The steps below assume the target Environment has been created, the Atom or Cloud runtime
has been installed, and that runtime has been attached to the given Environment.

Deploy New Version to First Environment

1. Developer manually attaches (if first time deploying to the target environment) and
deploys initial deployment version to the first environment using the AtomSphere user
interface. Although the APIs exist to attach and create the new deployment version
programmatically, it is recommended to create the initial deployment manually after
successful unit testing. This allows the developer more control over all the
components involved with the given change.
a. Alternatively, client application verifies attachment and calls CREATE
Deployment with componentId=<process or other component ID to
deploy>, environmentId=<target environment ID>, and deployNotes.
2. Developer or client application sets extensions.
3. For scheduled processes, developer or client application configures schedules.

445

Internal Use - Confidential

 Verify Attachment

1. Client application calls QUERY ComponentEnvironmentAttachment

with componentId=<process or other component ID> and environmentId=<target
environment ID>.
2. If a result is returned, no further action. Otherwise client application calls CREATE
ComponentEnvironmentAttachment with componentId=<process or other component ID
to attach> and environmentId=<target environment ID>.

Set Extensions

1. After deploying to a new environment, the client application calls QUERY

EnvironmentExtensions with environmentId=<target environment ID>, overlays the
desired new values, and calls UPDATE EnvironmentExtensions.
2. Alternatively, the user can manually configure any new environment extension values
required for the new process through the user interface.

Configure Schedules

1. Client application calls QUERY EnvironmentAtomAttachment

with environmentId=<target environment ID>.
2. For each atomId returned, client application calls QUERY ProcessSchedules
with atomId=<target atom ID> and processId=<process ID to schedule> to retrieve the
ProcessSchedules object for each desired process.
3. Client application calls UPDATE ProcessSchedules with id=<process schedule
ID>, atomId=<target atom ID>, and processId=<process ID to schedule> for each desired
process.

Promote to Next Environment

As of the June 2017 release, the deployComponent API should be used to promote any type of
component from one environment to another.

1. Client application calls QUERY Deployment with componentId=<process or component

ID> AND environmentId=<source environment ID> AND current=“true” to retrieve the
current deployment ID from the source environment.

446

Internal Use - Confidential

 2. Client application then calls GET Deployment deploymentId=<source deployment
ID> to retrieve the digest.
3. Client application verifies attachment to target environment (see above).
4. Client application calls DeployComponent with deploymentId=<source deployment
ID>, digest=<source deployment digest>, and environment=<target environment ID>.
5. Client application or user sets extensions as above.

6 Manage the Process

6.1 Error Handling Framework

The following functionality has been identified as common components which should be
implemented as an error handling process.

***More detailed information is needed from the customer for a more in-depth error handling
design***

6.1.1 Process Error Handling Logic

Every integration process should adhere to the proposed development framework that would
include a try/catch shape. Upon an error or failure, the catch path would pass the failed
data along with error and process properties to a predefined map. This map will differ
depending on the source canonical XML data format. All error notification maps will utilize
the same error XML format prior to submitting the message to an Atom Queue.

447

Internal Use - Confidential

 Figure 241:Process Error Handling

6.1.2 Common Error Handling Logic

The error handling process would be responsible for mapping the canonical error data from
the failed execution and submitting a message to the IT/Business owners.

Figure 242: Error Message

The standard message should follow a canonical data format containing the following
parameters:

1. Process name

2. Tracking ID

3. To Address

4. Functional Domain (ETL, B2B)

5. Error message

448

Internal Use - Confidential

 6. Execution ID (custom code to retrieve it)

7. Environment (Test, Dev, Production, etc.)

The error handling process will utilize a listen action on the start shape. As messages are
placed on the queue, notifications will be immediately sent to correct IT/Business Owners:

1. Start Shape: Atom Queue Listen

a. As messages are placed on the queue the process will listen for and process
documents.

2. Set Properties Shape: Set mail notification properties

a. Mail Subject: Will utilize dynamic and static properties to populate the mail
subject

b. From Address: Static email address.

c. To Address: Dynamically set by the profile of the source data (Error XML
canonical profile).

3. Message Shape: The body of the error notification will be set with the use of source
profile (Error XML canonical profile).

a. Process Name

b. Time

c. Execution ID

d. Tracking ID

e. Error Messages

449

Internal Use - Confidential

 6.2 Operational Monitoring

6.2.1 Integration Process Alerts

AtomSphere has a built-in event framework that automatically generates events for situations
such as process failures and Atom offline. Additionally, developers can generate custom
events within process orchestrations using Notify and Exception steps.

Events can be communicated via several channels:

• Integration UI including Process Reporting (execution summary and process logs) and
Dashboard
• AtomSphere user email subscriptions
• AtomSphere platform API
• RSS Feeds

Email alerts are configured as subscriptions for a given Integration user. Each user can choose
to subscribe one or more alerts based on:

• Log Level – INFO, WARNING, ERROR

• Event Type – ATOM.STATUS, PROCESS.EXECUTION, USER.NOTIFICATION, ALL

To send alerts to a common service account or distribution list, simply create a new
Integration user for that email address, log in as that user, and add an email alert
subscription. To avoid sending excessive emails, multiple events are consolidated and sent
every few minutes in a single email message. Emails are not sent for Test environments.

450

Internal Use - Confidential

 Figure 243: Settings

For more flexibility over filtering, routing, and formatting of alerts, consider using the
AtomSphere platform API to periodically extract recent events programmatically. The events
can be extracted and communicated via a corporate SMTP mail server or inserted directly into
an incident management application.

The event extraction can be accomplished by designing another Integration process to call the
platform API using the AtomSphere API connector. Alternatively, this event handler process
could be performed as a script or program run outside of AtomSphere. Although unlikely, it is
possible for the notification process itself to fail and not communicate its own failure. If an
external script is not desired, this risk can be mitigated by designing the notification process
to always send a “baseline” notification even if no events are found. In this case the absence
of a baseline notification after an extended period of time would be an indication of a
problem.

Details about the Event API can be found here: Event Object, Boomi User Guide

6.2.2 Execution History

The Process Reporting console within the Integration UI displays execution results and process
logs for the past 30 days with the ability to drill into each execution to view documents in and
out of the process along with the step-by-step process log.

Note: Web service processes configured as Low Latency will NOT record execution history to
the Process Monitoring console unless they have a runtime exception. The additional logging is
forfeited in favor of decreased response times. Successful executions are aggregated and
reported via the SOA Dashboard in a summarized fashion.

451

Internal Use - Confidential

 For historical analysis, Integration provides several account dashboards for summarized
integration activity over longer periods of time. This graphical information is helpful to
analyze data volumes and trends.

6.2.3 Purge History

While the Process Reporting UI will retain the summary execution history for 30 days, the
execution artifacts such as process logs, and document data can be purged more frequently.
This can be desirable for runtimes processing a large volume of data. The purge frequency is
configured for the entire runtime, not per-process.

The runtime can also be configured to Purge Immediately so that execution artifacts (most
notably document data) are removed immediately upon completion of the execution. Again,
keep in the purge setting applies to the entire runtime, not individual processes.

Execution artifacts for processes running on the Boomi Atom Cloud are purged after 14 days.
This can be decreased but not increased beyond 14 days.

While it is technically possible to increase the purge history beyond 30 days for local
runtimes, it is not practical because the execution summary information in the Process
Reporting UI is only available for 30 days.

The purge frequency can be modified per runtime in Atom Management > Atom Properties (or
Molecule or Cloud Properties). The Basic Property sets a “global” purge frequency for all
execution artifacts. Additionally, /alternatively you can override this setting by configuring
purge frequencies for logs (process and container), documents (also applies to Atom Queues),
and temp data individually with Advanced Properties.

More information on purging: Purging, Boomi User Guide.

Procedure
1. In the Manage menu, select Atom Management.
2. Do one of the following:
a. Select the Atom, Molecule, or Cloud Molecule from the list on the left.
b. Select an account’s Atom.
3. In Settings & Configuration, click on Account Properties.
4. The Account Properties panel opens.

452

Internal Use - Confidential

 5. Set Purge History After x Days to the number of days following process execution that
purging of logs, processed documents, and temporary data occurs, and processed
document archiving, if enabled, also occurs.
6. If Purge Data Immediately is on, turn it off.
7. Click Save.

6.2.4 Local Runtime Monitoring

To ensure processing and performance for critical production integrations, it is important to

monitor the local runtime for availability and overall system health. Monitoring the Atom and
underlying infrastructure is best achieved using a combination of strategies that watch
different aspects of the runtime environment.

Table 39: Monitoring Techniques

Aspect Monitoring Techniques

Server Infrastructure - Monitor basic server and OS vitals for the infrastructure running
the Atom. Key metrics include server availability, CPU usage,
memory usage, hard disk usage, and disk I/O wait latency. These
should be monitored for point-in-time anomalies and trending over
time for capacity planning.

- When starting out monitor the servers closely while running

“normal” integration loads to establish a baseline from which to
drive threshold reporting.

- General runtime capacity management considerations -

453

Internal Use - Confidential

 Aspect Monitoring Techniques

General Atom Application - Monitor that “atom.exe” OS process is running.

Availability and Status
- Subscribe to AtomSphere platform email alerts for ATOM.STATUS
to be alerted of communication issues between the local Atom and
the platform. Alternatively, the AtomSphere platform API can be
queried for the same events, either using an Integration process
running in the Atom Cloud or from a client outside of AtomSphere.

- Use a log monitoring tool such as Splunk to monitor/scrape

the ./<atom install>/logs/container.log for SEVEREs entries to
identify problems with the Atom container outside the context of
an integration process execution. This includes problems with the
embedded web server.

Cluster Health (Molecules only) - Monitor the “Cluster Problem” JMX property, or

- Monitor the Molecule node “views” files for node health and
inter-node communication issues -

454

Internal Use - Confidential

 Aspect Monitoring Techniques

Atom Application Internal - Use a JMX-compliant monitoring tool to connect into the Atom’s
Health Java JVM and gain real time insight into various application
metrics. This includes both generic Java metrics and Atom-specific
metrics. These should be monitored for point-in-time anomalies
and trending over time.

- Key metrics include:

* JVM heap used

* JVM thread count

* OS system load average

* Container status

* Has the Atom entered into the "low memory" status

* Number of times scheduler has missed the time at which

it should run schedules

* Running executions

* Cluster problems

* Complete list:
http://help.boomi.com/atomsphere/GUID-48F37A93-5BF9-
488D-82C3-38E4E9D45A22.html.

- When starting out, monitor the JMX metrics closely while running
“normal” integration loads to establish a baseline from which to
drive threshold reporting.

6.2.5 Cloud Runtime Monitoring

As a hosted service, the infrastructure capacity, application health, and general system
availability of the Atom Cloud are monitored by the Boomi team. Individual customers should

455

Internal Use - Confidential

 still implement some monitoring controls for account-specific configuration (e.g., web
services authentication) and automated awareness of the Atom Cloud availability.

Table 40: Monitoring Techniques

Aspect Monitoring Techniques

Atom Cloud Status - Subscribe to AtomSphere platform email alerts for ATOM.STATUS.
Alternatively, the AtomSphere platform API can be queried for the
same events, typically from a client running outside the Atom
Cloud.

6.2.6 Integration Logs for Troubleshooting

Below is a list of the Integration logs for troubleshooting.

Table 41: Logs for Troubleshooting

Log Location Description

Test Mode Integration Test Mode UI: - Process – Primary means of

(Process and investigation execution problems. Log
- Process – Click Process Name at top >
Document Logs) of given execution including the steps
View Log.
executed, number of documents per

- Document – Select step on canvas, then step, step execution duration, and

view Logs tab at bottom. step at which exception occurred.

Tip: save larger logs locally to view
more easily.

- Document – Additional document-

level information such as parameter
values and individual step evaluations
(info varies by step type).

456

Internal Use - Confidential

 Log Location Description

Process Reporting Integration UI > Manage > Process - Process – Primary means of
(Process and Reporting investigation execution problems. Log
Document Logs) of given execution including the steps
- Process – Click Log icon for execution
executed, number of documents per
record.
step, step execution duration, and

- Document – Click execution timestamp step at which exception occurred.

> select Start step > select View Logs Tip: save larger logs locally to view

from gear picklist. more easily.

- Document – Additional document-

level information such as parameter
values and individual step evaluations
(info varies by step type).

Container Log Integration UI > Manage > Atom Used for general runtime health and
Management > select Atom > Download errors that typically occur outside the
Atom Log context of a single process execution,
such web server/listener issues or
Locally, ./<runtime
problems communicating with the
root>/logs/yyyy_MM_dd.container.log
Integration platform. Contains
uncaught exceptions so can be an
additional source to check for
problems if nothing shown in process
execution.

Running daily log per node of runtime

activity. Does not include history of
all executions. SEVERE entries are of
most interest.

Purged according to Atom’s purge

schedule (30 days by default).

457

Internal Use - Confidential

 Log Location Description

HTTP Server Log Integration UI > Dashboards > HTTP Used to help troubleshoot incoming
Status Dashboard requests that do not result in an
execution from the platform’s
Locally, ../<runtime root>/logs/
perspective.
yyyy_MM_dd.shared_http_server.log
Running daily log of inbound web
service requests per node. Info
includes timestamp, target endpoint,
user, HTTP status, and response time.

UI dashboard contains summary info

only.

Purged according to Atom’s purge

schedule (30 days by default).

Molecule Node Integration UI > Manage > Atom Series of files indicating Molecule
Views Management > select Molecule > Cluster cluster health such as is there only
Status tab one head node, do all the nodes know
about one another, are all the nodes
Locally, ../<runtime root>/bin/views
responsive. UI displays status
graphically.

Typically monitored by operations

group.

Install4j Log Locally, ../<OS temp Used to troubleshoot failed

dir>/install4jError<generated installations of new runtimes.
number>.log

458

Internal Use - Confidential

 6.3 Functional Monitoring

6.3.1 Overview

Knowing when integration processes fail or do not complete as expected is a critical part of
managing the integration platform. Integration provides a number of options to suit your
needs depending upon the volume of executions, desired communication method, and
sophistication of notification routing and delivery requirements.

For monitoring the general availability and health of your Atoms and Molecules, see
Operational Monitoring Techniques for Atom, Molecule, and Atom Cloud Runtimes.

6.3.2 Functional Monitoring Options

The table below describes the various process monitoring options and best practices for use.

Option Description Best Practices

The Process Reporting console can be used to manually
monitor low volumes of executions but as activity
increases, handling Events on an exception-basis becomes
necessary. However regardless of how you are notified
The Process Reporting about an error, you will most likely use the Process
console provides Reporting console to research logs and document-level
detailed information information, troubleshoot, and rerun documents. See also
about process Overview of Integration Logs for Troubleshooting.
Process executions including
Reporting process logs and
documents.
(UI) Use Document Tracking to capture key values from
document data to be able to quickly find and trace a
given record through multiple process executions.
Manage > Process
Reporting

Important notes about Process Reporting:

459

Internal Use - Confidential

 Option Description Best Practices
• Results are not retained indefinitely. See the
section below: A Note about Execution History
Retention and Purge Policies.
• Access to execution results can be restricted by
user Role and/or Environment. Additionally, the
ability to view document data can be
permissioned separately. See Understanding User
Management and Custom Roles.
• Executions for web service processes configured
as Low Latency are NOT recorded in Process
Reporting unless they have an exception. Instead
they are summarized in the Real-time Dashboard
(see below).

There are three dashboards that can be used for trending

and historical analysis:

The Dashboards • Account Dashboard - Account-level summary of

provide historical and activity including number of executions/errors,
summary information throughput, document count, and connector
about process usage.
Dashboards executions and • Real-time Dashboard (available if Services
inbound requests. Enablement feature is enabled in your account) -
(UI) Summary of published web services processes
configured as Low Latency which do not capture
execution history in Process Reporting.
Dashboard > choose • HTTP Status Dashboard (available if Services
dashboard Enablement feature is enabled in your account) -
Summary of HTTP status code responses for
published web services processes by date.

Email alerts are best used for exception-based

notifications that will be received and acted upon by
individuals. As a general recommendation, configure the
Integration users can Log Level to WARNING to receive alerts for WARNING and
choose to receive ERROR events but not expected successful activity such
platform Events via as "process started" and "process completed". See Email
email. Users can alert management.
Email Alerts choose to subscribe to
Events by Log Level
(Event and Type.
Framework)
For higher executions volumes or if more advanced
subscription/routing rules are desired, use the Event API
to consume and handle Events as needed. See the
Account menu > Setup Platform API section below.
> Email Alerts

460

Internal Use - Confidential

 Option Description Best Practices
Important notes about email alerts:

• Alert subscriptions are configured per user, not

per account. Users with access to multiple
accounts will see subscriptions across their
accounts.
• Administrators cannot subscribe users to emails on
their behalf. This is notable when wanting to send
emails to a distribution list or shared inbox. A
separate Integration user must be created.
See Setting up email alerts for a distribution list.
• Email alerts are batched and sent from the
Integration platform periodically every few
minutes to avoid triggering spam filters. This
means one email can contain multiple Events from
different processes, Atoms, event types, log
levels, etc.
• The subject and format of the email messages
cannot be modified.
• Email alerts are NOT sent for Events occurring in
"Test" Environments (i.e. Environments classified
as "Test". However, are generated and may be
consumed through another channel such as API or
RSS.
• For Low Latency
executions, process.execution Events are NOT
generated, even for process errors and Exception
shapes, so no email alerts will be
delivered. user.notification Events ARE created
and therefore can be delivered via email,
however be cautious in high volume scenarios. See
the section below for considerations. See
Understanding the Event Framework for more
about Event Types.
• Email alerts do not consume a connection license.

Platform Events are

RSS are the simplest approach for receiving platform
available via RSS feed
Events. If you have an incident reporting system or
per Atom or at the
monitoring tool that can consume RSS feeds, consider
account level.
using them.
RSS Feeds

(Event
Framework) Account menu > Setup
For exception-based monitoring, use the "Alerts Only"
> Account Information
feeds to only receive Warning and Error level Events.
There are no other filtering options.
or

461

Internal Use - Confidential

 Option Description Best Practices
Manage > Atom
Management > choose
Atom > Atom See Understanding the Event Framework.
Information
The Event object provides the most flexibility for routing
rules, message formatting, and delivery options. It is the
best option for exception-based monitoring of larger
numbers of executions.

For execution-related Events, you can use the Execution

Record object and Download process log operation to
retrieve additional details about the execution. Note the
API does not provide access to document-level
Events can be information including tracked fields.
consumed
programmatically via
Platform API
the Event API.
You can create another AtomSphere process (or use an
(Event
external monitoring tool) to periodically extract new
Framework)
Events and route and deliver as required. For example,
There is no UI for the Events could be sent to your incident reporting
list of Events. application to leverage existing communication channels
and escalation rules. Note the connection to the
destination system including your company's SMTP mail
server will consume a connection license.

IMPORTANT: Event records are available for 7 days.

See Understanding the Event Framework.

If truly custom notification logic is required, you can
design this into the process flow itself by capturing errors
and validation warnings using Try/Catch and Business
Rules shapes and connector operation response handling.
Custom Custom error handling For example, runtime errors could be caught, formatted
and notification logic with a Message shape, and emailed directly to recipients
(Built into can be built into the using a Mail connector and your company's SMTP mail
Process process design/flow as server.
Flow) required.

Alternatively, the errors could be mapped to custom

alert message XML format, and written to an external

462

Internal Use - Confidential

 Option Description Best Practices
database or queue for subsequent processing. This option
may be desired when integrating with an existing
notification/auditing/monitoring framework or to provide
custom attributes in the notification message. It requires
additional upfront design, development, and testing
effort. It is still advised to use the platform Event
subscriptions to be notified of problems that cannot be
handled within the context of a process execution.
Table 42: Process Monitoring Options and Best Practices

6.3.3 A Note about Execution History Retention and Purge Policies

The Process Reporting console displays execution results for the past 30 days. This period
cannot be changed. The results include process- and document-level metadata such as
execution statistics, document counts, and document tracked fields.

The Process Reporting console will always retain the summary execution history for 30 days,
however the execution artifacts such as process logs and document data can be purged more
frequently. This may be desirable for Atoms/runtimes processing a large volume of data. Note
that purge schedules are configured for the entire runtime, not per-process.

The runtime can also be configured to "Purge Immediately" so that execution artifacts (most
notably document data) are removed immediately upon completion of the execution. Again
keep in mind the Purge Immediately setting applies to the entire runtime, not individual
processes.

Execution artifacts for processes running on the Boomi Atom Cloud are purged after 14 days.
This can be decreased but not increased beyond 14 days.

While it is technically possible to increase the purge history beyond 30 days for local

463

Internal Use - Confidential

 runtimes, it is not practical because the execution summary information in the Process
Reporting console is only available for 30 days.

The purge frequency can be modified per runtime in Atom Management > Atom Properties (or
Molecule or Cloud Properties). The Basic Property sets a “global” purge frequency for all
execution artifacts. Additionally/alternatively you can override this setting by configuring
purge frequencies for logs (process and container), documents (also applies to Atom Queues),
and temp data individually with Advanced Properties.

For more information see Purging of Atom, Molecule, or Atom Cloud logs and data.

6.3.4 Considerations for Monitoring High Volume Low Latency Processes

Processes configured to use Low Latency mode require special considerations for monitoring.
As noted above, Low Latency process executions do not generate Events for process errors
and therefore you cannot rely on email alerts, RSS feeds, or even the Event API. User
notifications (from Notify shapes) are created and therefore could be received via email
alerts, however because Low Latency mode is typically used for high volume scenarios such as
inbound web services and JMS messages, receiving an Event or email alert for each individual
error can be impractical given the sheer volume of potential messages. Instead the strategy is
to detect a general, persistent issue with the service and then use the available process logs
and dashboards to investigate further.

Below are some additional considerations and recommendations for monitoring errors for Low
Latency processes.

• Determine whether error reporting is even required. This is especially relevant for
web service requests in which some types of requests may be transient in nature. In
fact the client may have already resubmitted the request. Similarly, determine if you

464

Internal Use - Confidential

 can mitigate errors--especially those caused by "bad data" in the request--by
implementing validation logic within the process to reject the request and return an
appropriate response to the client. In those situations, there is no need to report an
error in the integration layer because the service behaved as intended and successfully
rejected an invalid request.
• Design for resiliency. For critical integrations, resiliency should be built into the
overall workflow to overcome potential connectivity and system availability issues.
This typically involves a decoupled process design, message queuing, and retry
mechanisms to ensure all messages are received, processed, and delivered.
• Implement a custom error handling framework. To capture and report issues during
a Low Latency execution, design a custom error handling framework as part of the
process flow as discussed in the table above. Using techniques such as proactive
validation (e.g. Business Rules, Decision, Cleanse shapes) and error handling (e.g.
Try/Catch shapes, connector response inspection), capture and generate your own
event messages and submit them an appropriate repository such as a message queue,
database, or custom log file to be handled asynchronously. However make sure the
repository and downstream framework are robust enough to handle your anticipated
volume of events.
• Leverage the ExecutionSummaryRecord API. The Execution Summary Record
object provides a pre-packaged summarized view of a given Low Latency process's
execution history for a given time block. This is the same information displayed in
the Real-time Dashboard. The status field indicates whether there was an issue with
one or more executions during the time block. The summary record also provides
useful statistics such as execution count, execution duration standard deviation that
can be used to detect unexpected execution patterns. Note this data is collected and
made available roughly every ~5 minutes so it is not immediate.

6.4 Event Framework

6.4.1 Overview

The Event Framework provides the technical infrastructure and plumbing for logging,
handling, and alerting from processes at run-time. It captures Events from processes and

465

Internal Use - Confidential

 Atoms within your account and records them in the platform database. These Event records
are used by Integration to generate email alerts based on user subscriptions and are available
to query directly through the Event platform API.

Note the list of Events is not visible in the Integration UI.

6.4.2 How the Event Framework Works

Figure 244: Event Framework Graphic

1. Something happens during a process execution or the Atom runtime's status changes.
2. The Atom uploads the event message to the Integration platform.
3. An Event record is created in the platform database.

466

Internal Use - Confidential

 4. The Event record is available to be sent/consumed by the various delivery channels
such as email, RSS, or the platform API.
5. The Events records are purged after 7 days.

Important Notes:

• The Atom runtime must be able to communicate with the Integration platform to
upload the message. If the Atom cannot connect for some reason, Events, along with
other messages will be queued and uploaded once the connection is re-established.
• Process Events are generated and uploaded to the platform asynchronously during and
after the execution completes. This can mean Events may not be available until
sometime after the execution finishes. You should not rely on these Events if you need
to trigger immediate/real-time workflows.
• Events reside in the AtomSphere platform, not the local Atom runtimes. You cannot
query/obtain Events from the local Atom.
• Event records are purged after 7 days.
• Events are NOT generated for Test Mode executions.
• For Low Latency executions, process.execution Events are NOT generated, even for
process errors. user.notification Events ARE created however be cautious when using
them in high volume scenarios. See Functional Monitoring of Integration Processes Best
Practices for considerations.
• Events ARE generated for executions and Atoms in environments classified as "Test"
however email alerts are not sent by the platform.

Types of Events

There are four types of Events.

Event Type Description

System-generated message created based upon the Atom's status,
atom.status including when the Atom comes online or goes offline.

467

Internal Use - Confidential

 Event Type Description

Keep in mind this status is from the AtomSphere platform's

perspective. For example, if the Atom cannot connect to the
platform for some reason, the platform may determine it is offline
because it has not received a ping from the Atom however the Atom
may actually still be running and executing processes as normal.
User-generated message created when a process contains a Notify
shape with the "Enable Events" option enabled. When enabled, the
user.notification
Notify shape message creates an Event record in addition to writing
to the process log as normal.
System-generated message created when a process execution begins
and ends. The completion event can be the result of a successful or
process.execution
errored execution, including one that ended with the use of an
Exception shape configured within the process flow.
System-generated message created when the Atom is so busy it is
unable to initiate a scheduled process execution. This event type is
rare and is usually indicative of an Atom reaching its processing
capabilities.

Schedules are "missed" if the scheduler service does not run once
each minute. There are generally two scenarios which can cause this
to happen:

1. There are too many schedules or the container is generally so

process.missedSchedule
overloaded that a given run of the scheduler takes longer than
a minute
2. The system clock is having issues and the scheduled tasks does
not run once per minute as intended

Note: Missed scheduled events are not related to the "Maximum

Queued [Forked] Executions per Node" or "Timeout for Queued
[Forked] Executions per Node" Atom properties. They also do not
represent scheduled processes that did not execute while an Atom
was stopped.
Table 43: Four Types of Events

468

Internal Use - Confidential

 Key Event Attributes

For the full list of attributes see Event object. In addition to eventType described above,
some of the notable attributes include:

• eventLevel - The log level (e.g. INFO, WARNING, ERROR).

• status - The user-specified message or Event status (e.g. COMPLETE, ERROR).
• processName - Name of process for applicable Events.
• environment - Name of the environment.
• classification - Classification of the environment (e.g. PROD, TEST).

Events are distinct from process execution results that are displayed in the Process Reporting
console.

Creating User Notification Events

Most types of Events are created automatically by the Atom and/or platform, however you
can create your own user notification events by using a Notify shape with a process. Notify
shapes can be used "inline" because they do not modify document data.

You can also create Events with your own error message by using an Exception shape
(assuming it causes the process execution to fail). However it manifests as a
process.execution Event type.

Notify shapes always write to the process log but if you want it to additionally create an
Event, simply check the option Enable Events:

469

Internal Use - Confidential

 Figure 245: Enable Events in Notify Shape

The Notify shape's Title, Message Level, and Message will be available in the Event record.

See Notify shape for more information and considerations on creating Events.

6.4.3 Consuming Events

Once Events are uploaded and captured in the platform database, they can be consumed via
several channels. See Functional Monitoring of Integration Processes Best Practices for
considerations of using each.

470

Internal Use - Confidential

 Channel Description
AtomSphere sends email notifications to users based on their Email Alert
Email subscription preferences. See Email alert management for configuration
information.
Events are always published via RSS. There are two feeds, "Monitor" (all Events)
RSS and "Alerts Only" (only Warning or higher Events), available at the overall
AtomSphere account level as well as the individual Atom runtime.
Events can be queried directly using the AtomSphere API Event object. This
provides the most flexibility for filtering and routing based upon whatever
AtomSphere
criteria is required. By creating an integration process within AtomSphere (or
API
using an external client), you can integrate Events into an incident reporting
system or monitoring tool within your organization.
Event types process.execution and user.notification are also captured in the
Process Log execution's process log. The process log can be viewed through the Process
Reporting console or the Download process log API.
Table 44: Event Channels

6.5 High-volume Troubleshooting

6.5.1 OutOfMemoryException errors

Why am I getting an OutOfMemoryException error while attempting to process data in an
Atom? The reasons include, but are not limited to:

• You are attempting to process a single large data file without batching the data.
• You are executing many processes simultaneously.
• You are executing many large data sets simultaneously.
• There is a memory inefficiency in the Atom.
• Your computer is out of physical or virtual memory that it can allocate to the Atom.
These troubleshooting questions are listed in order (first being most common) for
OutOfMemoryException errors.

6.5.2 The Atom's memory efficiency

Does the Atom’s memory require special configuration to process millions of records?

The Atom employs various techniques to limit memory usage. The primary technique is
buffering data to disk. When you read in a very large file, only a small portion of it ends up in
memory. As data travels from shape to shape, a set of file pointers is being passed. At some

471

Internal Use - Confidential

 point during processing, the data is read into memory, so it can be transformed, manipulated,
or inspected. When this occurs, the size of the data that is read in affects whether you get an
OutOfMemoryException error.

A base level of memory usage is required to track each document through a process. A
process that reads in 100,000 documents uses memory to store metadata for each document
in memory. There is a memory pattern difference between the Start shape reading in many
documents and any other shape handling many documents. While the Start shape consumes
memory for each document, other shapes that produce many documents do not use any
additional memory.

Be aware that the Start shape consumes memory, especially when you build large master
processes that initiate many subprocess calls that run asynchronously. Each subprocess
produces many documents in the Start shape and they all consume memory until each
subprocess is complete.

To avoid heavy memory usage when processing large amounts of data, spread the load of the
Atom over time or configure the process to handle large amounts of data.

• For technology connectors, run large data sets at different times to avoid
simultaneously processing large numbers of documents.
• To process large CSV files, use the Data Process shape with Split Documents to batch
the data into smaller chunks. However, be aware when using the Split by Profile that
it will attempt to read all of the data so that it can sort and organize each flat file
line.
• Use the Flow Control shape to execute each document one at a time after you split.
This spreads the memory usage over time instead of processing all documents at one
time per shape.
• When reading from the database, use the Grouping options to batch the data into
smaller record sets by count or by a matching link element from the Database Read
profile. For example, use "OrderNum" to group inbound records that have the same
order number.
• When writing to the database, use the Commit options to group database commits by a
specified count or multi-table data set.

472

Internal Use - Confidential

 6.5.3 Repeated exception errors
What if I get exception errors after I employ all of these techniques?

By default, the Atom attempts to use only 512 MB of RAM when it processes a low- to
medium-level data load; it can safely process thousands to tens of thousands of records at
one time. Data sets of hundreds of thousands or even millions of records require increased
resources to the Atom. The key question is how much RAM should you allocate to the
Atom? Boomi recommends not limiting the Atom by minimally following this guideline:
Table 45: Records and Usage

Records RAM usage

100,000 1 GB
200,000 to 500,000 2 GB
500,000 and above 4 GB or more
These are simultaneous records being read from an application. An Atom can easily process a
one-million record CSV file by splitting and mapping with only 512 MB of RAM. This is because
inside the Atom the single file will be only one "record" which is split by the process.

6.5.4 Usage of all available memory on the machine

Why can't the Atom use all available memory on the machine?

Memory usage is based on the architecture of the Java virtual machine (JVM). Java handles its
own memory and sets up its own memory space, called heap space, from the available
operating system memory. Java uses heap space only to perform its work. If the heap space is
full and objects that have been created by the Atom cannot be removed from the heap space
safely (called Garbage Collection), then OutOfMemoryException errors occur.

There is a benefit to this approach. A regular Windows/Linux program can consume all
available memory in a machine even extending into Windows virtual memory or Linux swap
space. Virtual memory or swap space is on-disk memory space that is very slow and is used
primarily to save old states of an application. When an application uses this memory space, it
can slow down the entire machine. Moreover, when an application uses all available physical
and virtual memory, then the machine is unable to use that memory for regular operation and
eventually crashes.

473

Internal Use - Confidential

 By putting all Java memory into the heap space, Java protects the operating system and the
other applications running on the machine. Only the Java application itself encounters the
out-of-memory problem so that users can attempt to fix the problem. If the machine crashes,
it is much harder to diagnose and fix the issue.

One peculiarity about this type of memory management is that after machine RAM has been
allocated to Java heap space, Java is reluctant to return that RAM to the operating system.
This is because, for performance reasons, Java attempts to maintain as much RAM as you will
allow it. As a result, reallocating memory can be a time-consuming operation. Even if the
Atom is not doing anything, it still may be using a large amount of memory because Java has
not freed the memory for the operating system. This does not mean that there is a memory
leak—it just means that there is a set of free memory inside the JVM that the operating
system cannot see.

Although you can change this behavior, Boomi recommends that you do not do that. If you
decide to make this change, you can set the following options on the Custom tab of the
Properties panel or add them to the <installation_directory>/bin/atom.vmoptions file.
-XX:MinHeapFreeRatio
-XX:MaxHeapFreeRatio
A good article that describes these options is located on Oracle’s website.
The options above, when set on the Properties panel or in the atom.vmoptions file, apply only
to the primary process running on an Atom, Molecule, or private Atom Cloud. They do not
affect forked executions. To set these options for forked executions, you must set them on
the Custom tab of the Forked Execution Properties panel, or in your
customized procrunner.*, procworker.*, and/or procbrowser.* script files.
Note: The scripting engine that is used depends on whether you use Windows or Linux. If you
are running:
• Windows and an Atom, or if you are running a Molecule that does not use UNC paths or
forked execution, then batch scripts (.bat) are used.
• Windows and a Molecule that uses UNC paths or forked execution, or if you are running
an Atom Cloud, then Powershell scripts (.ps1) are used.
• Linux, then shell scripts (.sh) are used in all situations.

474

Internal Use - Confidential

 6.5.5 If you still have problems with your Atom
None of that works. Now what?

We may need to investigate the Atom's internals to determine if there is some efficiency we
can gain from the code. There are always possibilities of memory inefficiencies in the Atom.
Any memory analysis that we do starts with setting the Heap Dump on Out of Memory Error
property (-XX:+HeapDumpOnOutOfMemoryError) on the Advanced tab of the Properties
panel.
When the OutOfMemoryException error occurs, a very large file that is the entire Java
memory space is generated in the root directory of the Atom. You can zip up this file and
send it to Boomi Support for further analysis.
In some cases, we may find a memory inefficiency. In other cases, we may not, and the
solution may be to spread out the load or to redesign the process.

6.6 Maintainability and Maintenance

6.6.1 Process Design & Build Documentation

• “Add Notes” Feature

o Use of the notes feature encouraged to explain processing and intent
directly within the process for future use and maintainability. This
feature available within process components
• “Add Description” Feature
o Use of the “Add Description” encouraged to explain process purpose for
maintainability. This feature is available within process components.
• Offline Process Documentation
o Creating thorough offline process documentation encouraged so both
business and development staff may understand processing. This
documentation usually includes processing descriptions, endpoint
configuration information, logic and mapping data, error handling
approach, and performance information

475

Internal Use - Confidential

 6.6.2 Process Execution Documentation

• Document execution instructions including schedules and rationale

• Document steps for process failures for each process. Example: Upon
failure…Retry docs? Re-run process? Reset flags or dates? Etc.

6.6.3 Process Notifications

The platform is a low code platform, but it can still use any programming concepts you might
have heard of from before. In order to know if your process works or not, you need to know
what values are, inputs and outputs and anything else to inform you when something goes
wrong during testing or when it is in production.

The main way to view a value while it is in the data flow is through the notify shape which
will show the value in the process logs or when you click on the notify shape in test mode.
The notify shape uses the same exact variable system to construct the body. If you add a
variable of a dynamic process property called "currentDay", it will be assigned to {1},
wherever you use {1} the value of DPP currentDay will show up. View 2.5 for more
information on the variable system.

During the process

Notifications during the process will show up in test mode if you click on the notify shape.
Notifications during the process will show up in process logs if you have the process deployed
or in test mode. You can use a notify shape directly anywhere to see what a value will be in
some location (before an error, after, etc). The notify shape shows up in the process logs, but
can only show 10,000 characters of what's in the notify shape.

Otherwise, you can show notifications in the process log during scripting also.

476

Internal Use - Confidential

 Figure 246: Script to Show Notifications

After the process

After the process, you can also have notifications sent to you. The first kind of notification is
email alerts that you can sign up for. Email alerts let you choose a log level and event type.

Log Level - event severity levels applicable to the subscription:

• INFO - the email alerts include all execution information, warnings, and errors for
active processes and all atom status information
• WARNING - the email alerts include all warning and error information for active
processes and disabled atoms
• ERROR - the email alerts include all error information for active processes and disabled
atoms
Event Type - category of information included in the subscription

• ALL - provides information for all three event types below

• ATOM.STATUS - when an atom in the account is online or offline
• PROCESS.EXECUTION - provides information about a process execution
• USER.NOTIFICATION - provides information from a notify shape having the Enable Events
option that was included in a process execution
USER.NOTIFICATION works only with the notify shape on the conditions:

• Enable events has to be checked

• Only production environments will send out notifications
• Only deployed processes will cause a notification to be sent out, test mode does not
trigger a notification from email alerts

477

Internal Use - Confidential

 • Notification is sent out based on message level in notify and log level selected in the
email alert

Email alerts
Some things to note about email alerts:

• Alerts are grouped in 15 minute intervals, if multiple things happen that causes alerts
happen within the 15 minute interval, all of them will be sent together by event type
• Low latency does not generate email alerts
• Test environments (or atoms attached or processes deployed on it) will not generate
email alerts for executions or atom statuses
• Only the user can see their email alerts configuration
• We have a limited report to show some information about a user's subscribed email
alerts in a Boomi Admin report (User's Email Alerts, if you provide an email address)

Process Logs
Process logs show what happened during the process. It will tell you how long a shape took,
where documents went to and what happened in the process. Test mode and scheduled
executions will have process logs. Only test mode will allow you to inspect each shape.

6.6.4 API Procedures

Keep Processes Simple

Our first recommendation is to reduce the risk of problems down the road. Keep your linked
implementation processes simple. Using smaller pieces with one operation in your process is
easier to maintain than working with more complex pieces.

Conflicting Profiles
You should not deploy API components that specify objects for which a profile type conflict
exists. A conflict occurs when multiple referenced XML profiles include the same type and
namespace or when multiple referenced JSON profiles have the same root name. You can
resolve a conflict by standardizing on one profile.

478

Internal Use - Confidential

 To resolve this issue, go to the Profiles tab in the API component, select the profile you want
to standardize on. Then, endpoint definitions are automatically updated to reference that
profile, thereby resolving the conflict.

Adding Authentication
Authentication is a complicated process that involves many steps. Publish your API first, to
make sure the service is publishable. Deploy your API to a local atom with the Authentication
Type set to None. Then work with the more advanced publication setup of adding
authentication.

HTTP Status Dashboard

To see HTTP status codes returned over a period of time, visit the HTTP Status Dashboard
where you can view codes by summary and codes by time. 200-level codes do not appear
here, since those are success codes. The HTTP Status Dashboard page is divided into two
sections, called gadgets, which display a Status Code Summary and Status Codes by Time.
These logs cannot be viewed in other displays, such as Process Reporting, or the API
Management Dashboard, but are available in the shared HTTP server logs.

API Management
The main purpose of the API Management feature is to enable a web service publisher to
expose versioned APIs for logical groups of web services. This web service API consists of a set
of REST, SOAP, and/or OData endpoints. This is the foundation for our API Management
offering and serves as an extraction layer over web service enabled processes.

This allows for web service components to be consolidated into a single, explicit location,
making API design easier and more efficient.

Rather than having multiple listeners at different addresses, API Management can organize
each operation into one WSDL that can be easily consumed by customers and/or partners.

6.6.5 Identify Who Will Monitor Executions and How

• Identify resources to monitor processes

• Monitoring resources should be client resources unless specifically coordinated
otherwise
• Example monitoring methodologies: Email alerts; Error Emails (email
connector); identifying error records in an endpoint; etc.

479

Internal Use - Confidential

 6.6.6 New Release Best Practices
To minimize the impact of new releases on your business, we highly recommend all
AtomSphere accounts to follow the below release best practices.

Apply Release Control

Release Control occurs two weeks before the full release. You can apply pending Atom and
connector updates at any time during those two weeks. The Release Control period allows you
to select a time within that window that is convenient for you to apply and test the updates.
The pending updates are applied per Atom (or clustered Molecule or Atom Cloud). You can
update your "Test" Atoms and test the changes against your processes before the full release.
If you are happy with the test results, you can apply the changes to your "Production" Atoms
and move the updates into your production environment before the full release. If you apply
the pending updates during the Release Control period and then change your mind, you can
remove the updates and roll back to the currently released version.

NOTE: Pending Atom and connector updates are also available on the Boomi Test Atom Clouds
during the Release Control period.

For more information regarding Release Control, please refer to this user guide.

To schedule Release Control, please follow the step by step instructions here.

Set Property 'Force Restart After X Minutes' to a value that is greater

than 0

Force Restart After X Minutes - The number of minutes that the Boomi AtomSphere platform
will wait for processes to finish before it forces the Atom, Molecule, or Atom Cloud to restart.

If the property value is set to 0 or a negative value, atom/molecule/cloud will not restart
until all the running processes finish. In the scenario where there is long running process,
atom/molecule/cloud will not restart after the release/RC takes place. We recommend

480

Internal Use - Confidential

 understanding the average response time of your executions and set this property 'Force
Restart After X Minutes' to a value that is greater than 0.

To set the property, please follow the instructions here.

481

Internal Use - Confidential

 7 Boomi Help and Support Options

7.1 Introduction
This guide provides an overview of the various help and support options available to you as a
Boomi customer or partner. As a new or existing user, you have access to a vast list of
resources including the Boomi Community, Support Center, User Guide, Status site, Services
resources, and Customer Success to help you maximize your success with the tool.

We are here to help you succeed. Please refer to this guide often to help you ramp-up
quickly.

7.2 Quick Links

Resource Link
AtomSphere Platform Login http://platform.boomi.com/
Within the platform, Help menu > User Guide

or

User Guide
Integration (https://help.boomi.com/category/integration)

Official product documentation

Master Data Hub (https://help.boomi.com/category/hub)

API Management (https://help.boomi.com/category/api-

management)
Boomi Community Within the platform, Help menu > Community

482

Internal Use - Confidential

 Forums, knowledge base, or
example processes, ideas, user
groups https://community.boomi.com/

Support Center

Within the platform, Help menu > Support

Contact technical support, case
management
Status Within the platform, Platform Status &
Announcements (footer)
Current system status and
announcements for platform or
and hosted Atom Cloud
environments https://status.boomi.com

Training and Certification https://train.boomi.com

Professional Services
https://boomi.com/services/
Project Consulting
Boomi Website https://www.boomi.com/
Table 46: Resources and Links

7.3 Community
The Boomi Community is your one-stop-shop to find answers, share best practices and discuss
integration topics with fellow users, and stay up to date with Boomi products. Whether you're
a customer, partner, or developer, a brand new user or experienced architect, the
community is your single destination for all the resources you need to be successful.

7.3.1 Community Features

• Getting Started - Learn how to get started and best leverage the community. Also get
community site updates, help, and feedback.
• Community Forums - Ask questions and discuss integration topics with your fellow
users.

483

Internal Use - Confidential

 • Knowledge Base - Browse Boomi-authored content including how-to's, common errors,
implementation guides, and more.
• Support Center - Contact Support, manage your cases, read technical bulletins, and
browse known issues.
• Community Share - Example processes, templates, and patterns to import into your
account.
• Boomi Buzz - Read blogs about integration musings, product tips, and Boomi news.
• Ideas for Boomi - Share your ideas for innovating and enhancing our products. Ideas
for Boomi is restricted to registered customer users only.

7.3.2 Accessing the Community

The community is publicly accessible and anyone including guests can view the community.
However, you will need to log in with your AtomSphere credentials to post questions and
comments.

To access the community from within AtomSphere, go to Help & Feedback menu (question
mark icon) > Community.

Alternatively you can access the community directly at https://community.boomi.com.

7.3.3 Community Forums

The forums are where community members come together to ask and answer questions about
integration development and design, error messages, and product features. You can easily
search existing discussions to find correct and helpful answers or post a new question. If you
find a question or topic that you about, why not respond and share your expertise?

7.3.4 Knowledge Center

Browse Boomi authored content including knowledge articles, implementation guides, and
more. Here you can learn more about our products and get help resolving technical issues.

484

Internal Use - Confidential

 7.4 User Guide
The User Guide is the official product documentation. It contains the core terms and concepts
you need to understand when building integration processes. To access the User Guide from
within your AtomSphere account, go to Help & Feedback menu (question mark icon) > User
Guide.

Alternatively go directly to:

• Integration (https://help.boomi.com/category/integration)
• Master Data Hub (https://help.boomi.com/category/hub)
• API Management (https://help.boomi.com/category/api-management)

7.5 Contacting Technical Support

Support Services: Boomi’s goal is to provide support according to the tables below,
depending on the level of support you have purchased. Standard Business Hours are defined
by region:

Asia Pacific (APAC): 8am — 8pm GMT+11, Monday — Friday

Americas: 8am — 8pm ET, Monday — Friday
Europe, Middle East, Africa (EMEA): 8am — 8pm GMT, Monday — Friday

Extended Business Hours are from Sunday 5pm – Friday 8pm.

Standard Premier Premier Plus

24x5 for all Severity
Hours of Standard Business Hours 24x7 for all Severity
Levels
Coverage 24x7 for Severity 1 Levels
24x7 for Severity 1

485

Internal Use - Confidential

 Standard Premier Premier Plus
User Forums
24x7 Support Center
User Forums
Support 24x5 Support Center Access
Support Center Access During
Channels Access Live Chat
Business Hours
Live Chat Phone
Phone
Table 47: Coverage and Support

7.5.1 Support Options

• Support Center (Recommended) – Through your AtomSphere Platform account, a

client may access the support center to log a case with the Boomi Technical Support
team.
• Email – The client may submit an email problem request to support@boomi.com which
will log a support case. Response times are not guaranteed with this method.
• Phone – Available 24x5 starting Sunday 8 p.m. through Friday, 8 p.m.. The client may
call the technical support number located within the Support Center to submit the
details of their issue to create a support case or to report a Sev1 issue (24x7 is
available with Premier Plus Support, 24x5 is available with Premier Support).
• Live Chat – Available 24x5 starting Sunday 8 p.m. through Friday, 8 p.m.. Live Chats
are available for simple problems that can be resolved quickly (24x7 is available with
Premier Plus Support, 24x5 is available with Premier Support).

7.5.2 Severity Definitions and Response Times

Support Response Time Standard Premier Premier Plus

486

Internal Use - Confidential

 Severity 1 (Urgent): Security breach, 1 hour 1 hour 1 hour
production down, or complete system
failure. Significant parts of the system are
not secure or are inaccessible or
inoperable. There is no viable workaround.
Severity 2 (High): Primary business 2 business days 8 business 4 hours
requirements could not be met. There are hours
no easily apparent viable workarounds.
Performance, functionality, or usability is
seriously degraded.
Severity 3 (Medium): Business requirements 2 business days 8 business 4 hours
can be met with the system. Workaround is hours
apparent. Performance, functionality, or
usability is not seriously degraded.
Severity 4 (Low): May be addressed in a 2 business days 8 business 4 hours
future release at Boomi’s discretion. Minor hours
typos, wish list suggestions, but not a
required change. Would not affect release
accuracy or usability in any significant way.
Table 48: Severity and Response

7.5.3 Accessing the Support Center and Creating Cases

See Boomi Support Center Guide

7.5.4 Case Submission and Follow-Up

After submitting the case, you will receive a response according to your Boomi Support Policy.
As we troubleshoot the issue, we will communicate with you by posting comments to the
case. You will receive email notifications alerting you an update to your case is available and
you can view the case updates via the Support Center> Cases tab. Please do not reply to the
email notification. To respond, simply add a comment to the case through the Support

487

Internal Use - Confidential

 Center. Boomi Support can also add additional team members to your case so that others on
your team can receive case comment notification emails as well. If you wish to add additional
team members to your case, please add a comment in the case with the email addresses of
your team members and request that they also be added to the case.

Severity 1

If the platform user interface is inaccessible or your process running on a hosted Atom Cloud
environment are failing, please check Status (https://status.boomi.com) for the latest
platform status before contacting support.

Again a Severity 1 issue means Production use of Boomi products on a primary business
service, major application, or mission critical system is stopped or so severely impacted that
the client cannot reasonably continue work.

Severity Level 1 problems could have the following characteristics:

• Platform down
• Atom Cloud down
• MDM Cloud down
• "Production" processes hang or crash situations
• "Production" data loss or data corruption
• Critical functionality not working after a Boomi release
• Rollback functionality not working after an atom or connector update was applied
during release control

For Severity Level 1 problems, we will begin work on the problem within the guidelines of the
Boomi Service Description. For issues that may involve a Boomi OEM or Reseller Partner, the
expectation is that partner resources will be available in Severity Level 1 situations, and the
partner will reasonably cooperate with Boomi to resolve the issue.

Severity 1 Call Procedure

Call the Premier Support Phone Number for your region (available in support center):

488

Internal Use - Confidential

 • Please have your PIN # available.
• The support team will respond to the call by creating a new case (if not already
created) and starting to investigate the issue.
• Weekend calls will be retrieved by Boomi’s call service and immediately relayed to the
on-call support team member via phone and email.
• Support team member begins the investigation of the issue and updates the case
comments within the guidelines of the Boomi SLA.
• Support team will engage the backline operations and engineering teams as necessary.
• case comments will be updated on regular intervals to keep the client informed of any
updates with the case. Comments can be viewed through the support center.
• Support team will communicate the resolution to the client through email and case
comments.
• Support team reviews any client comments and follows up to verify a complete
resolution.

Escalating a Case (Non Severity 1)

• Contact Boomi Support through the case or the Premier Support number with
additional information to pass along to the Boomi Support Team regarding escalating
the case.
• Based on current case volume, the escalated case will be prioritized for additional
focus and the Support Manager will instruct the support team to provide attention to
the escalated case.
• If the escalated case requires backline engineering attention, you will be notified and
the Support Manager will act as a point to the Engineering team.
• The Support Team will provide updates regarding the case through the support center,
email and phone.

7.5.5 Best Practices for Contacting Support

1. For faster resolution, enter a case through the support center and provide as much
diagnostic information as possible. This reduces the number of interactions to resolve
a case.

489

Internal Use - Confidential

 2. For Severity 1 cases, enter the case through the Support Center with diagnostic
information, include adding ‘Severity 1 / Production Down’ in the case description and
then call the Premier Support phone number (in support center) with your PIN # and
referencing the case number. If your support plan is not Premier or Premier Plus, call
the number referenced on the status.boomi.com website, 1-866-407-6599 (Toll free in
US), 1-503-470-5056 (International) and reference the case number.
3. Leverage the Boomi Community resources for knowledge base, community forums, and
training materials.
4. Use Live Chat for simple questions and the support center for complex questions.
5. Check status.boomi.com for the current platform status.
6. Be prepared to share your business impact and intended process flow.

7.5.6 Status Site

The Status site (https://status.boomi.com) displays the current status of the AtomSphere
platform and various Atom Cloud hosted environments.

In addition, you can find:

• Current status and updates to system-wide issues

• Live and historical data on system performance
• Notifications on scheduled maintenance windows and release schedule
• Information on how we secure your data

7.6 Professional Services

With our training, support, and consulting resources, you can accelerate implementation,
reduce time-to-value with the Boomi platform and ensure the success of your business-to-
business (B2B) application and data integration initiatives.

490

Internal Use - Confidential

 7.6.1 Training and Certification

Our training curriculum includes instructor lead on-site instruction at a Boomi training facility
or your location. We also offer a remote training option and other self-service options.

• Classroom training at Boomi: Our live, interactive classes focus on real business
scenarios and Boomi best practices. Classroom training is relevant for novice users and
advanced developers and is the fastest, most thorough way to build your integration
skill set.
• Remote training: We provide remote training options offered in a public class setting.
• Custom training: A Boomi Certified Instructor will work with your staff to develop
custom courses and deliver them remotely or in-person at your facility.
• Self-service training: Boomi customers have on-demand access to a growing library of
recorded training sessions, tutorials, and fully configured integrations.

For more information please see Training and Certification.

7.6.2 Consulting Services

The Boomi professional services team offers expertise in integration development and best
practices. We can advise your organization at critical implementation checkpoints, deliver a
complete, production-ready integration, or provide custom training for your IT staff. Working
with our consultants, you’ll gain confidence, knowing that your implementation will succeed.

With Boomi consulting, you can:

• Reduce time-to-value through faster implementation

• Improve integration design
• Avoid costly rework
• Identify additional opportunities to increase ROI

Our consultants can help your organization with:

• Architectural design and validation

• Process development
• On-site or remote JumpStarts

491

Internal Use - Confidential

 • Team augmentation
• Best practices review

Our engagement models are flexible so please click here to learn more.

7.7 Customer Success

The Customer Success team provides assistance to help you get the most out of your Boomi
investment. We want you to be a completely satisfied "Customer for Life". We offer a variety
of programs to improve your overall experience including quarterly check-ins, proactive
monitoring, as well as serving as an escalation point for account and technical issues.

For more information please click here.

7.8 Summary
Boomi AtomSphere integration Platform as a Service enables you to integrate any combination
of cloud and on-premise applications without software, appliances, or coding. The resources
in this guide are available to help you be successful from day one.

Thank you for choosing Boomi!

8 Appendix - Atomsphere Best Practice Checklist

8.1 Accountability

492

Internal Use - Confidential

 Document Tracking feature should be used. □

Implement appropriate error handling, notification, and logging. □

Use appropriate file names (when using Disk or Ftp connector) □

When possible, pass internal keys from the source system to a field in the destination □
system. Optionally you may also want to write keys from the destination system back to a
field in the source system.

In most cases, going directly to a Stop shape from a Route, Decision, Try/Catch, or Business □
Rules is discouraged. If any documents go down this branch, it will be difficult to trace
after the execution is finished.

Terminate your branches with Stop shapes where possible, even if it will not change the □
results of the process. This will communicate your intentions as the developer to
whomever may need to maintain your process in the future.

8.2 Maintainability

Naming conventions □

o Shape labels
 use labels for set properties, data process, decisions
 optional for branch and route
 generally discouraged from labeling start shape and maps
(unnecessary with good component naming), and stop shape
o Component names
 All components should have names (not “New Map” or “New
Profile”)
 Rename any components that have had a numeric suffix added by
Integration to enforce uniqueness. For example “SF account query

493

Internal Use - Confidential

 1” and “SF Account query 2” could become “SF account query by
id” and “SF Account query by name”.
 Avoid putting indicators such as “production” or “test” in your
component name if you are using environments/extensions
o Folder names

Proper folder structure is used. □

Sensible process layout □

Use of the notes feature encouraged □

Use of sub-processes where appropriate □

Should not filter on minus "n" days in connector call. Potential to miss records if and when □
a process fails over a period of time greater than “n”.

Eliminate redundant copies of components, especially connections □

Ideally single connection component per endpoint for deployment (and possibly 1 more for □
browsing). For example, a single component for Salesforce, use environments/extensions
to point it to Sandbox/QA/Production instances

Ideally a single copy of your processes. □

o "Test copies" of your processes are discouraged

o Changes are normally done in your single process, unit tested in test mode,
deployed to a TEST or QA environment for further testing, then promoted
to Production
o If you're planning extensive changes to a process, a v2 of your process is
acceptable

494

Internal Use - Confidential

 o A backup copy of a folder is acceptable only if done infrequently; consider
copying to another Boomi account if one is available.

Keep components out of the root folder. For small accounts, putting shared connections in □
the root folder is acceptable.

Avoid using personal logins for endpoints. □

Use shared components where appropriate. Keep shared components together in a shared □
folder.

8.3 General

Appropriate use of Options in the build tab. For example, do you want to allow □
simultaneous executions? Is your listener process set to Low Latency?

Process should be designed to be re-runnable (for example, should handle upserts rather □
than just creates). This helps when re-running a process after a partially failed execution.

Date handling in maps. where possible, define as true date in both profiles, and avoid the □
"date convert" function

Decoupling process flow when appropriate (e.g. JMS or atom queues) □

495

Internal Use - Confidential

 The Business Rules shape is preferable to chaining together decision shapes □

Multiple Disk Connection components should be avoided. Directories can be specified in a □

set properties shape or connector parameters tab

If possible use sync flags to select source data instead of using LastSuccessfulRunDate □

If you need to use a date for source data selection, store and use the highest date from the □
source system instead of using LastSuccessfulRunDate

Split the document if necessary before document cache step □

8.4 Performance

Minimize connector calls □

Pull data all together when possible (e.g. use joins on DB selects or SAAS queries when □
available)

Caching documents where appropriate □

496

Internal Use - Confidential

 Function caching where appropriate □

Avoid making repeated connector or DB calls on a per-document basis where possible. □

consider caching or other techniques

Minimize or eliminate use of "run each document individually" feature for high-volume □
processes

Batching where possible □

Parallel processing if needed and appropriate □

Eliminate use of “map function ordering” if it is not needed □

How many records per unit of time is this process expected to handle? Has it been tested □
at this volume? If high volume, is this process set up appropriately for high volume
(caching, parallelism, etc.)?

8.5 Final Checks

Verify that the process is tested, signed off (by the appropriate approvers) and □
documented.

497

Internal Use - Confidential

 High volume processes should be performance tested. Results record as a benchmark. □

Be sure execution instructions are documented. □

What should a user do if this process fails?

• Retry docs?
• Re-run process?
• Any resetting of flags or dates? etc.

Identify who will monitor executions, and how □

Verify that proper user management is in place □

On bi-directional synchs, be sure “ping-ponging” has been avoided. For example, if □

accounts are synched from system A to system B in one process, and then system B back to
system A in another process, and if both processes are run using “last mod date” logic, a
record could bounce indefinitely between one system and the other.

8.6 Process Development Checklist

Upon completion of building and testing your process within Integration, the following
checklist should be used to determine if a deployment ticket should be submitted.

1. Are all shapes/steps connected?

498

Internal Use - Confidential

 • There should be no red connector endpoints.

2. Are the appropriate connector(s) assigned?

• Make sure there are no BROWSING ONLY connectors assigned.

3. Have the extensions been enabled in the process?

• Reference section 5.3
• Verify that the correct connector fields have been extended.
• Verify the appropriate process properties/dynamic process properties have
been extended.
4. Have the shapes/steps been properly labeled?

5. Have the correct Process Options been set?

• Purge data immediately should be set for any processes that contain
sensitive data.
6. Is the error handling sub process assigned?

7. Is the correct map and values for error handling canonical format assigned?

8. Has the process been successfully unit tested?

9. Were the development standards (naming convention, folder layout, design patterns,
etc.) followed?

499

Internal Use - Confidential