Professional Documents
Culture Documents
Introduction To WTX
Introduction To WTX
Kimberly Drouot
WebSphere Transformation Extender Technical Lead SW IOT
kimberly.drouot@fr.ibm.com
ANY Data
Many-to-many Integration
ANY Data
Ex. - Hierarchical Data , Binary Data, Packed Data, Tabular Data, Relational Data, Nested Structures, Mixed-Type Data, and on and on
Transform Anything
Transform Anywhere
Everyone Transforms
Two of the biggest and most significant challenges of dealing with data are
transformation and semantics.
Zapthink
There are many integration styles and challenges, each with their own unique
demands; what is consistent across every one of them is the need for data
transformation and mediation.
IBM
Resources
and IT Assets
(insurance company)
Legacy
Claims
Database
Auto
Claims
System
Home
Claims
System
Life
Insurance
Claims
Business
Partner
Claims
System
1.
Each system has its own data definitions, business rules and interfaces
2.
3.
Interactions between applications expose data and business rule anomalies which
result in high maintenance costs and inflexibility to change
4.
The Validation and Use Rules must align with the Transformation needs and execute in
the same unit of work with Transformation
5.
Ever increasing integration logic points with duplicate or slightly modified templates
rob from anticipated benefits and efficiencies
Hard/Complex
Processing
Interdependent
Data
Many
to
Many
Many
to
Any
Any to Any
Any
To
Many
Easy/Simple
TABLE
COPYBOOK
Make
Model
PKG
Extended_Feature
s
Ford
Prefect
34890
2984782q,
93847920,
3438084
01
03
05
TP-API-CB.
TP-AGAPI-CB.
TP-AGAPI-REQUEST
PIC X(40).
TP-AGAPI-INITIALIZE-REQUEST
VALUE 'INITIALIZE-MAPPING'.
TP-AGAPI-PERFORM-MAPPING
VALUE 'PERFORM-MAPPING'.
88
TP-AGAPI-FINISH-MAPPING
VALUE 'FINISH-MAPPING'.
05
TP-AGAPI-VERSION
PIC X(04).
88
TP-AGAPI-VERSION-VALID
VALUES ARE '0100' '0200'.
88
TP-AGAPI-VERSION-0100
VALUE
'0100'.
88
TP-AGAPI-VERSION-0200
VALUE
'0200'.
05
TP-AGAPI-RESPONSE.
10
TP-AGAPI-RESPONSE-CODE
PIC 9(04) COMP.
88 TP-AGAPI-ALL-OKAY
VALUE 0.
88 TP-AGAPI-REQUEST-ERROR
VALUE 1.
88 TP-AGAPI-INITIALIZE-ERROR
VALUE 2.
88 TP-AGAPI-MAP-ERROR
VALUE 3.
88 TP-AGAPI-FINISH-ERROR
VALUE 4.
88 TP-AGAPI-UNKNOWN-LOOP-ID
VALUE 5.
88 TP-AGAPI-NO-ALGORITHM
VALUE 6.
88 TP-AGAPI-NO-PARTNER
VALUES 7 15.
88 TP-AGAPI-NO-APPLICATION
VALUE 8.
88 TP-AGAPI-ALGORITHM-IO-ERROR VALUE 9.
88 TP-AGAPI-FATAL-GATEWAY-ERROR VALUE 10.
88 TP-AGAPI-GATEWAY-WRITE-ERROR VALUE 11.
88 TP-AGAPI-PARTNER-IO-ERROR
VALUE 12.
88 TP-AGAPI-BAD-VERSION
VALUE 13.
88 TP-AGAPI-BAD-NUMERIC-TYPE
VALUE 14.
88 TP-AGAPI-NO-ALIAS
VALUE 15.
88 TP-AGAPI-ACCESS-ERROR
VALUE 16.
10
TP-AGAPI-RESPONSE-MESSAGE
PIC X(80).
05
TP-AGAPI-APPLICATION-ID
PIC X(10).
88
Ford
Prefect
34890
2984782q,
93847920,
3438084
Ford
Prefect
34890
2984782q,
93847920,
3438084
Ford
Prefect
34890
2984782q,
93847920,
3438084
Ford
Prefect
34890
2984782q,
93847920,
3438084
Ford
Prefect
34890
2984782q,
93847920,
3438084
CASH RECONCILATION
27 files
88
10029847 - $100,000,000.00
13948589 - $679,495,094.98
13950967 - $588,345,058.00
13950968 - $000,000,000.00
14001321 - $098,957,038.12
AAA: HT4459
AAA: B33566
AAA: C4058G
AAB: 948409
AAB: 874931
03
01
PROPRIETARY
OH,257*IN,142*MI,1
54*WI,80*MT,5*ID,8
*WY,3*CO,21*NM,8
*AZ,15*UT,13*NV,4*
MN,48*MO,67*ND,9
*SD,9*KS,27
DUNS 0123
F046000INV
for 2
months 120799
12 718-339-1700I43989D-2
120799999-b
003000010000
DUNS 4445
P55590
120799499QR
000004004000
EBCDIC
<MSG 19934749>
<ACCT BAL RPT>
@
z@
@@
@
<ACCTNUM><14001321>
<ENDBALANCE>< $098,957,038.12>
<DEBIT><103048382$394,394.00>
<DEBIT><103048383$001,293.65>
<CREDIT><987463921$928,943.67>
</ACCTNUM>
</MSG>
413 msgs
Unix
30+ Adapters
SOA
Linux
ETL
WebSphere ESB
SOAP
Enterprise
Applications
Application
Servers
WebSphere DataStage
z/OS
EJB
Batch
HTTP
CICS
Event driven
IMS
Batch
USS
Java
C/C++
COBOL
Health Care
SWIFTNet
SWIFTNet Funds
SEPA*
Many Services Based Solutions
FIX
FedWire, CHIPs
NACHA
BAI, BAI2
AL3
ACORD
Etc.
HIPAA
NCPDP
HL7
Many Services Based Solutions
HealthCare Hub
Payment Processing
Clearing House Processing
CPG, Manufacturing
Embedded Software
ERP/CRM Integration
EDI (ANSI X12)
TRADACOMS
ODETTE
EDIFACT
Many Services Based Solutions
WebSphere TX
A Long And Storied Road
V8.1
V8.0
V7.5
Proven
V6.7
Q3 2006
V6.5
July 2005
V6.0
Reliable
June 2004
V5.0
Jan 2003
V2.1
Jan 2002
V1.4
Sept 2001
V1.3
V1.2
Innovative
June 2000
V1.1
Dec 1999
V1.0
Aug 1997
June 1996
Nov 1995
June 1995
Jan 1994
Mature
MANUFACTURING,
RETAIL & DISTRIBUTION
HEALTHCARE /
INSURANCE
J2EE
Standalone
System z (native)
Windows, UNIX, Linux
Embedded
Standalone
Event driven
App. Server
Process Server
Multi-In
Advanced ESB
ESB
Multi-Out
Complements
existing portfolio of
products
Delivers new sales
opportunities
For batch
environments
WebSphere
WebSphere
TX
TX
MAP
TRANSFORM
Unifying
Unifying
Enterprise
Enterprise Data
Data
Transformation
Transformation
For WebSphere
Message Broker
For WebSphere
Process Server
For WebSphere
ESB
For WebSphere
Partner Gateway
For 3rd Party
Integration Products
(as an embeddable engine)
Source
Type Tree
Rules
Mapping
Type Tree
Adapter
Destination
In other words
An adapter is a technical connector which can be
plugged into a source or destination - without parsing!
A Type Tree is a graphical representation of meta-data
Rules define how a Source structure is transformed to
a Destination structure
Together these components form a WebSphere TX Map
Type Tree
Adapter
Type Tree
Adapter
Type Tree
Sources
Type Tree
Adapter
Type Tree
Adapter
Rules
Mapping
Destinations
DATABASE UPDATE
TRANSFORMED MESSAGE
ON OUTPUT
CROSS-REFERENCE
TABLE
Orchestrating Transformations
10
Type Designer
Database Interface
Designer
Resources
Communication
Adapters
Map Designer
Message Launcher
Broker
Batch / JCL
IMS
CICS
11
Type Designer
Database Interface
Designer
Resources
Communication
Adapters
Map Designer
Message Launcher
Broker
Batch / JCL
IMS
CICS
12
Map Designer
Create maps to:
Validate data from data sources
Identify data targets
Specify data transformation logic (rules)
Identifying Data
Sources and Targets
Identify
Identify type
type tree
tree and
and type
type
tree
tree components
components that
that define
define
data
data target
target
Identify
Identify type
type of
of data
data and
and
location
location of
of data
data
13
WebSphere TX Adapters
Communication
CICS *
COM
CORBA
E-MAIL *
FTP/FTPS *
HTTP/HTTPS
JAVA Class
JCA Gateway
SOAP
SOCKET
VAN
Messaging
BEA MessageQ & Tuxedo
Candle Roma
IBM WebSphere MQ *
(MQSeries)
JMS
Microsoft MSMQ
Oracle AQ
TIBCO Rendezvous
Applications
Siebel
- BIM, EIM, OI COM
SAP R/3 & BW
- ALE, BDC, DMI, DXOB
- BW, EDI, BAPI
PeopleSoft
- Message Agent API,
- Open Query, EDI
QualityStage
WebSphere
TX
Design Studio
Utility
Archive (ZIP/TAR)
Base 64
Batch
File *
GZIP/ZLIB
JNDI
LDAP
MIME
Quoted-Printable
Shell Script
Sink *
SNMP
XML *
Database
DB2 *
Informix
ODBC *
OLE-DB
Oracle *
Microsoft SQL Server
Sybase
HP NonStop SQL/MX &
SQL/MP
Security
S/MIME
OpenPGP
SSL Module
* Adapters supported on z/OS, varies by execution method
EDIFACT
COBOL Copybooks
SWIFT
SEPA*
ACORD
HIPAA
HL7
14
15
Pre-defined
Mapping
Functions
No Procedural Coding!!
Easily represent complex business rules
16
Rejecting Errors
17
Serveur WTX
(moteur)
Z/OS
API
Batch / JCL
Event-Driven / ESB
WebSphere ESB
Deploy
XML Schema
000520****************************************************************
000530 01
TP-API-CB.
000540 03
TP-AGAPI-CB.
000550
000560 05
TP-AGAPI-REQUEST
PIC X(40).
000570
88 TP-AGAPI-INITIALIZE-REQUEST
000580
VALUE 'INITIALIZE-MAPPING'.
000590
88 TP-AGAPI-PERFORM-MAPPING
000600
VALUE 'PERFORM-MAPPING'.
000610
88 TP-AGAPI-FINISH-MAPPING
000620
VALUE 'FINISH-MAPPING'.
000640
000650
000660
000670
000680
05
88
88
88
Importer
Map
Importer
TP-AGAPI-VERSION
PIC X(04).
TP-AGAPI-VERSION-VALID
VALUES ARE '0100'
'0200'.
TP-AGAPI-VERSION-0100
VALUE '0100'.
TP-AGAPI-VERSION-0200
VALUE '0200'.
COBOL
Copybook
Design Studio
Windows
18
Map
Integration Flow
Designer
Designer Importaters
WebSphere TX Server
Server A
19
WTX for z
Transformation and execution on z/OS
Product Offering
Execution Method
Batch / JCL
CICS
IMS TM
IMS/DC
Event-driven,
multi-threaded,
multi-process transformation
engine
WTX Parser
WTX Map Node
SMP/E installation
Supports the following sources/targets natively
Files (QSAM, VSAM)
DB2
FTP
MQ
20
Z/OS
key-sequenced
entry-sequenced
d
CICS tee n
n tio
a
r c
ua a
G ans
Tr
CICS
Appli
WTX
CICS
WTX Map
DB2
MQ
Transient data queues
Temporary storage queues
Storage buffers (COMMAREA)
21
Z/OS
IMS
WTX
22
XSLT
transformation
Compute
ESQL
Compute
Java
transformation
transformation
Mapping
transformation
WebSphere
TX
Universal
Transformation
Industry
Packs
(SWIFT, EDI,
HIPAA)
optional
23
Plug-in node
Transforms anything to anything
Icon installs into Message Flow palette
Easy configuration
24
Grazie
Italian
Thai
Traditional Chinese
Obrigado
Russian
Merci
Brazilian Portuguese
French
Gracias
Arabic
Spanish
Danke
German
Simplified Chinese
Japanese
25