Professional Documents
Culture Documents
com
Introduction to
IBM Integration Bus (IIB)
V9.0.0.1
Agenda
What is IIB (Message Broker)?
Programming Concepts
Message Flows
Nodes
Message Model
Product Overview
Components
Architecture
IIB on z/OS
Whats new in Message Broker V8
Whats new in IBM Integration Bus V9
Edge
Product Roadmap
IBM Integration Bus V9 Q2 2013
Policy-based Workload Management and Flow Management
Web-based Visualization and Performance Analysis
MQ and Database Service Discovery
Business Rules and BPM Integration
.NET Input, Dynamics, MSMQ
V9 FP1 Q4 2013
Message Broker
V8 FP2 Q1 2013
Message Broker
V8 FP1 Q2 2012
Message Broker
V8 Q4 2011
Application B
Application A
CICS
IBM Integration
Bus
Adapter
Adapter
Transport
Transport
The Broker (or hub or gateway or ) is central and the intermediary between the applications
It typically has responsibility for routing and transforming data between the applications
A Broker must be polyvalent with support for multiple data formats and protocols, extremely reliable and scalable
IIB z/OS
ESB
IIB z/OS
IIB z/OS
Application Connectivity
Protocols
e.g. MQ, TCP/IP, HTTP, File system, FTP, SMTP etc.
Message Formats
e.g. Binary (C/COBOL), XML, Industry (SWIFT, EDI, HL7), User-defined
Mediation Patterns
e.g. Route, Transform, Enrich, Filter, Monitor, Distribute, Decompose,
Correlate, Fire and Forget, Request/Reply, Publish/Subscribe, Aggregation,
Fan-in, Complex Event Processing
2011 IBM Corporation
MQ
Mr. Smith,
Graphics Card, 32,
100, 11/07/2009
WS
MQ
MQ
WS
10
Message Flows
Output target
(Failure)
Input source
Output target
Reusable
Scalable
Transactional
Transform
Output target
11
12
Nodes
The building blocks of message
flows
13
14
15
Node Terminology
node
error
terminal
output
connectors
input
connector
input
message
tree
16
input
terminal
Action
output
terminals
output
message
trees
IIB z/OS
17
Navigator
pane
Editor
pane
Outline
pane
2013 IBM Corporation
Properties
IIB z/OS
18
IIB z/OS
19
Patterns
20
Pattern example
1.Select a pattern
5. Name artifacts
it! Working flow generated!
2. Validate Thats
it
4. Input Q info
3. Fill in basic info
IIB z/OS
21
Message Modeling
<Person age=32 height=172>
<name>
<first>Fred</first>
<last>Smith</last>
</name>
</Person>
struct {
int height;
int age;
char firstname[24];
char lastname[24];
} Person;
172
32
Fred
TYPE
Person
age
name
height
integer
integer
Smith
first
last
string
string
22
Properties
MQMD
OutputRoot
Body
Properties
CustomerOrderMessage
Item
Age
First
Examples:
Item
Name
Other
Body
SupplySystemMessage
Person
Address
MQMD
Height
Supplier
DespatchTo
ID
Address
Last
InputRoot.Body.CustomerOrderMessage.Person.Age
23
Parsers
Model
Input Message Bit-stream
F r e d
S m i t h , G r a p h i c s
C a r d
Parser converts
logical structure
to bit-stream
Parser converts
bit-stream to
logical structure
< o r d e r > < n a m e > M r . S m i t h < / n
Model
24
COBOL
WSDL
Copybook
Pre-built
XML
Schema
DTD
Enterprise
Information
System
SOAP, MIME,
CSV, IDOC,
SWIFT,
EDIFACT, X12,
FIX, HL7,
etc
(SAP, Siebel,
PeopleSoft)
Define
your own
using the
Eclipse-based
Tooling
WebSphere
Transformation
Extender
File Import
Message Set
Type tree
Parsers
IBM Integration Bus
25
Message Transformation
Describe powerful
transformations quickly
Convert XML to
anything
Run a WebSphere
Transformation
Extender map
26
27
Development
When drag n drop development with the standard nodes isnt sufficient, programming
can be done typically with eSQL (procedural language based upon the SQL99
standard), Java, PHP, .Net, XSLT or additional add-on extensions such as WTX
Custom nodes can also be developed either in Java or C++
Toolkit supports tracing and interactive debugging
Team development and administration is supported by scripting and/or standard market
plugin extensions to the IIB Toolkit, eg. CVS, ClearCase, PVCS, TeamCode, etc.
IIB z/OS
28
eSQL
DataInsert
Compute
Data types
INTEGER
FLOAT
DECIMAL
STRING
DATETIME
BOOLEAN
REFERENCE
NULL
...
Operators
- + * /
||
AND OR NOT
= <> > >= < <=
IN BETWEEN
LIKE
IS EXISTS
...
Statements
Basic
DECLARE
SET
IF ENDIF
WHILE
Tree
MOVE
CREATE
DETACH
ATTACH
Database
INSERT
DELETE
UPDATE
PASSTHRU
EVAL
Node
PROPAGATE
RETURN
THROW
...
Functions
String
LENGTH
TRIM LTRIM RTRIM
OVERLAY
POSITION
SUBSTRING
UCASE LCASE
Numeric
ABS
BITAND NOT (X)OR
MOD ROUND
SQRT
TRUNCATE
EXTRACT
Datetime
EXTRACT
CURRENTDATE
CURRENTTIME
Field
BITSTREAM
CARDINALITY
FIELDTYPE
SAMEFIELD
Complex
CAST
SELECT
...
IIB z/OS
29
IIB z/OS
30
IIB z/OS
31
IBM
Integration
Bus
Explorer
IBM Integration Bus Toolkit
Development and Test Environment
Integration
Server
32
Broker
Exe grp
Exe grp
Exe grp
33
IIB Administration
IIB z/OS
34
9 IBM Integration Explorer is the visual (GUI) interface for casual administration
Eclipse- based plugin for WMQ Explorer
Available on Linux and Windows
IIB z/OS
35
IIB z/OS
36
IIB z/OS
37
IIB z/OS
38
Possible outputs
MQ(XML) by Pubish/Subscribe
UserTrace
BIP2380/2381/2382/2383 Messages
z/OS SMF
Type 117
subtype 1: Flow and threads
subtype 2: Nodes, optional terminals
Pub/Sub
Writer
ENF37
11
12
1
2
10
D
A
T
A
publish
subscribers
subscribe
UserTrace
Writer
Broker
Broker Domain
archive
snapshot
SMF
Writer
SMF
SYS
PLEX
IIB z/OS
39
<Nodes Number="3">
<NodeStatistics Label="FAILQueue" Type="MQOutput" TotalElapsedTime="0"
MaximumElapsedTime="0" NumberOfInputTerminals="1" NumberOfOutputTerminals="2">
<TerminalStatistics Label="failure" Type="Output" CountOfInvocations="0" />
<TerminalStatistics Label="in" Type="Input" CountOfInvocations="0" />
<TerminalStatistics Label="out" Type="Output" CountOfInvocations="0" />
</NodeStatistics>...
</Nodes>
</WMQIStatisticsAccounting>
2013 IBM Corporation
IIB z/OS
40
IIB z/OS
41
SubType 1
SubType 2
SubType 2
Message flow
Message flow
Message flow
Threads?
Nodes
Nodes
Terminals==0
Terminals>0
Thread Data
Node data
Node data
Thread Data
Node data
Terminal data
Node data
Terminal data
...
...
Terminal data
...
Flows/Threads
Including
Nodes only
Including
Nodes+Terminals
IIB z/OS
42
IIB Architecture
IIB z/OS
43
Runtime
Platforms
Message Flows
Message Dictionaries
Management Messages
Integration
Admin Appl
Monitoring Profiles
Toolkit
XML Schemas
Policy Sets
External Repository
Client
Client
Appl
Appl
Appl
Integration
Explorer
Broker(s)
Message Flow
HTTP Listener
Controller
Messages
Filter
Input
Node Group(s)Node
Execution
Msg
Flow
Filter Node
SOAP
Listener
Warehousing Node
Source
Administrative
Neon
Agent
Client
Client
Appl
Appl
Appl
Rules
Output
Runtime
Node
Security Mgr
Messages
Target
yIBM AIX
ySun Solaris SPARC
ySun Solaris x86-64
yHP-UX
yIBM z/OS
yWindows Server 2012
yWindows 2008
yWindows 7 & 8
yLinux (Intel) (Suse &
RHEL)
yLinux on POWER
yLinux on System z
yUbuntu
RDBMS
Input
Node
Filter
Node
Filter Node
Warehousing Node
Compute
Node
Output
Node
IIB z/OS
44
IIB on z/OS
IIB z/OS
45
Broker on z/OS
/s nnBRK
Infrastructure
main
Infrastructure
main
Infrastructure
main
USS
bipservice
DataFlow
Engine
DataFlow
Engine
wmqi
command
IMS
OMVS
CICS
Region
DB2
RRS
bipbroker
Threads
Threads
biphttp
listener
z/OS
Broker
Controller
Message Flow Engine(s)
Administrative
Agent
Input
Node
Filter
Node
Filter Node
Warehousing Node
LE
process
Neon
Rules
Output
Node
IIB z/OS
46
IIB control
IIB ExGroup
IIB ExGroup
IIB control
IIB ExGroup
IIB z/OS
47
Customers choose to run the Broker on z/OS typically when interfacing with
host data and/or when they require the best QOS only found on z/OS
2013 IBM Corporation
IIB z/OS
48
DB2
QMGR
QMGR
Broker
Broker
Execution
Group
Message
Flows
Coupling
facility
Execution
Group
Message
Flows
DB2
2013 IBM Corporation
IIB z/OS
Prepare component
1. Install Path
Tape
<hlq>.QMZABRK
Submit jobs
7. Create component
BIPCRBK
//DD
BIPCRBK, (QMZABRK)
/usr/lpp/mqsi/V7R0M0
...
bin classes lib
SBIPPROC
SBIPSAMP
8. Start component
/S QMZABRK
6. Tailor BIPEDIT
2. Authorization
extattr +a bipimain
extattr +l *.lib *.lil
per install
z/OS
per broker
IIB z/OS
50
ControlService
command
response
Console
Listener
Console
Parser
Command
Library
z/OS console
start/
stop
Admin
Agent
/F
/F
/F
/F
/F
start/stop
MQRBBRK,PC
MQRBBRK,cb l=/usr/lpp/wmqi/lil:/u/carlf/lil
MQRBBRK,SC
MQRBBRK,list e=default
MQRBBRK,changeflowstats a e default o smf
Short
Long
SC
Start component
PC
Stop component
CT
Change trace
RT
Report trace
List
RE
Reload
CB
Change broker
CS
RS
DP
Deploy
CX
RX
RC
Reload security
CM
RM
CR
RR
Start a broker
/S <Broker>
Stop a broker
/P <Broker>
Modify a broker
/F <Broker>,cmd
IIB z/OS
51
Business Monitoring
IIB
IIB
IIB
Making an Application
Inventory and Governing
Processing
IIB
IIB
IIB
TFIM
LDAP
WSRR
52
Participating in a Secure
Infrastructure
V8.0.0.1
record edit
replay
CMP
MQ
REST
HTTP
Broker
GET /admin/eg/MYEGNAME HTTP/1.1
From: machine@ibm.com
User-Agent: MyApp/1.0
9 DFDL:
58
Version 1.0
Proposed Recommendation
status
61
Graphical Transformations
IBM Graphical Data Mapper (GDM)
Visually map and transform source to target data
Code-free, high performing & scalable
GDM designed for whole IBM product set
e.g. Full map exchange with MDM Server V9
Mapping Script Language (MSL) format
Simple & powerful graphical mapping experience
Apply transformations to single and multiple elements
Conditionals (if then else), Loops (for each), Functions (target = f(source))& more!
Database mapping sources and targets for routing and enrichment scenarios
Broad database support (Oracle, DB2, SQL Server, Sybase, Informix)
Complements and supports existing transformation languages
Call user defined transformation in Java, SQL, XPath, .NET
Simple deployment, high performing & scalable
Maps can be deployed with solution BAR file or stand-alone
JIT compile means technology advances improves existing, deployed solutions
Source deploy + runtime compilation = enhanced performance
Migration from pre-version 8 maps
Existing maps developed before version 8 continue to work as-is
Existing maps opened in read-only mode for visualization & comprehension, cannot be modified
Automatic conversion of .mfmap format to MSL not currently built-in
62
63
65
Install IB V9
Stop broker
Start broker
Install IB V9
or
6
8
6
9
Migration from v8
70
DFDL Adoption
IBM DFDL reusable component ships with:
72
7
3
7
4
Graphical Transformations
request
Global
cache
response
REQUESTOR SYSTEM
77
PROVIDER SYSTEM
78
Virtualizes free memory within a grid of JVMs into a single logical space
Accessible as partitioned, key addressable map by applications and subsystems
Easy to Scale
Add more JVMs dynamically while its running without restart
MB developer has simple artefacts for working with the global cache
Unaware of the underlying technology (WXS) or topology
MyVar = Cache.Value;
Broker1
Broker2
Cache.Value = 42;
External Cache
Specify a value in seconds. The default value is 0, which means data never gets automatically removed.
Programming and operational enhancements
Insert and lookup map data using a wider range of Java object types for simplified programming logic
Support for highly available multi-instance configurations
8
1
8
2
8
3
8
4
Demo
85