Technical Track Session 

Data Transport Standard
Nathan Chitty ­ PESC Gary Sandler ­ PESC

Data Transport Standard ­ DTS
• DTS uses Internet technologies to  facilitate real time data exchange and  transaction processing • DTS builds on stable technologies, not  specific products • DTS, once implemented, reduces  programming and per­transaction costs  through standardization 

2

DTS Defined
• Data Transport Standard is a  specification not a product • Established by Postsecondary  Education Standards Council (PESC)  for exchanging data for:
– Inquiries – Reports – Transactions

3

DTS Defined
• A specification for an adjunct to or  a replacement for existing data  transport mechanisms
– PGP / GnuPG encryption – SecretAgent w/ Email – FTP and SecureFTP
4

DTS Benefits
• A Web Services implementation
–  Delivery confirmation included – no  guessing – All requests get a response – All submissions get an answer of  some kind
5

• Facilitates real time data exchange

DTS Benefits
• • • • Includes automatic data encryption Uses digital signature standards Platform independent Strong authentication with non­ repudiation
6

Benefits To “System Providers”
• Add value to schools’ systems
– Schools want transport added to systems  and are generally not concerned with the  technologies

• Easier to build one transport protocol  for all recipients 

– Just as CommonRecord created the drive  to build one XML format

7

Benefits to “Service Providers”
• As everyone implements DTS, the need  to support other transports will drop • If any school implements DTS, service  providers will have to support it • Also provides a single communication  infrastructure option for internal systems
8

DTS Specification
• Specification covers
– Technical interchange rules and  processes – Recommended best practices

• Technical Specification is the pure  Simple Object Access Protocol  (SOAP) interface • Implementation Guide is for both .Net  and Java reference implementations

9

DTS Specification
• Reference implementation  examples are available • Specification does not cover

– Business rules for transaction  processing – Operational oversight, monitoring or  escalation

10

Data Transport Issues in Higher  Ed
• E­mail is not reliable or flexible  enough
– No guarantee of delivery – No guarantee of order of delivery for  sequence dependent data – No automatic confirmation of receipt  or facility for retransmit
11

Data Transport Issues in Higher  Ed
• E­mail is not reliable or flexible  enough (continued)
– No synchronous response available  – Email size limitations

12

Data Transport Issues in Higher  Ed
• FTP data exchange has own  challenges
– Possible to overwrite earlier files – No confirmation of receipt – No synchronous response
13

Data Transport Issues in Higher  Ed
• Encryption today is always  separate and subject to its own 
– Issues – Maintenance – Failures
14

DTS Addresses These Transport  Issues
• DTS addresses
– The confirmation issue with a send­ receive protocol – confirmation is  built in – The order of delivery problem by  actively delivering and receiving the  data – no unconfirmed hand­offs
15

DTS Addresses These Transport  Issues
• DTS addresses
– The size problem through data  compression – The FTP overwrite problem by not  using filenames
16

DTS Addresses These Transport  Issues
• DTS addresses
– The lack of a synchronous response by  building in a required synchronous  response, even if only for handling status –  The encryption issue by using standard  HTTPS for encryption – the same  technology as for online banking 
17

DTS Technical Workgroup
• Task: Create a written specification for  real­time exchange of data between  organizations
– Meets business requirements – Standards based – Standard technologies (Java, .Net) – Payload Insensitive – Secure and reliable

18

DTS Technologies
• Global XML Web Service Architecture  (GXA), generally accepted as the  foundation for building Web Services
– WSDL (Web Service Definition  Language) – SOAP (Simple Object Access Protocol) – WS­I (Web Service Interoperability) – WS­S (Web Service Security)

19

DTS Technologies
• WS­Security (Digital Signatures)
– Strong authentication with non­ repudiation – X.509 encryption keys and  certificate authorities

• SSL encryption of HTTP streams
20

Anticipated Architectures
• Immediate processing

• “Push/Push” deferred processing
• “Push/Pull” deferred processing

– Request and processed Result Response – Request and Acknowledge Response – Request with Result and Acknowledge Response – Request and Acknowledge Response (just send) – Request for Result and Result Response

21

Immediate
Data Transport Standard: Immediate Processing DTS implementation

This configuration could have a server acting as a DTS Client that received data for transport from the back end.
1.

ie Cl

n t ai e ob ic to serv CR per c ts ient nta p co reci nt

dp En

t oin

r fo

Central Repository (CR) Key/URL/Services

3.

Se r se vice nd c er o n t o t ac a u ts th CR en t tic o o at b e a ta i nd n p au ub th lic or ke ize y o

f

Data passed to back end for processing.

2. submitDTS invoked against enpoint

Data passed to back-end for processing Processed result returned to be sent as response

`

4. Response: Actual processed response to payload <DTSAcknowledge>Immediate</DTSAcknowledge>

SBS/FAMS DTS Client Enabled

DTS Service submitDTS()

Back-end Processing

22

“Push/Push”
Data Transport Standard: Deferred Processing DTS implementation where originator is running a DTS Service to accept the processed result.
Central Repository (CR) Key/URL/Services
3. Se of rvi ce se nde co n r to tact au s CR the nt ic to o at e bta an in pu d a bli uth c k oriz ey e

This configuration could have a server acting as a DTS Client that received data for transport from the back end.

t po in End btain to o rvice s CR r se ntact nt pe nt co r ecipie Clie 1.

for

Validation could be performed by the back-end system in a synchronous manner allowing errors to be reported back immediately

2. submitDTS invoked against enpoint

`

4. Acknowledgement- <DTSAcknowledge>Received</DTSAcknowledge> Payload of acknowledgement may or may not contain a “token” as the payload. In this scenario it doesn’t make much sense (see deferred implementation without DTSService at client site) for one to be returned

Data passed to back end for processing.

SBS/FAMS DTS Client Enabled

DTS Service submitDTS()

Processing performed in the middle of accepting request and delivering processed result- Time frame of this step is immaterial

DTS Service submitDTS()
Data passed to back end for processing. 2. submitDTS invoked against enpoint 4. Acknowledgement- <DTSAcknowledge>Received</DTSAcknowledge>

Server as DTS Client
Back-end Processing Response Data passed to transport for delivery

3. Se rv sen ice c de ont r to ac au ts C the R t ntic o o a te b ta an in p d a ub uth lic k oriz ey of e

Central Repository (CR) Key/URL/Services

r t fo oin to dp g en ss in t ain oc e ob pr to of C R es ult t ts R en t ac t he cipi on re t c ack ien g b Cl in 1. end s

Exact same repository as the CR at top of page

23

Push/Pull”
Data Transport Standard: Deferred Processing DTS implementation where originator is ONLY running a DTS Client (a web service is not implemented for receiving the result )
Central Repository (CR) Key/URL/Services
3. Se of rvi ce se nde co n r to tact au s CR the nt ic to o at e bta an in pu d a bli uth c k oriz ey e

This configuration could have a server acting as a DTS Client that received data for transport from the back end.

t po in End btain to o rvice s CR r se ntact nt pe nt co r ecipie Clie 1.

for

Validation could be performed by the back-end system in a synchronous manner allowing errors to be reported back immediately

2. submitDTS invoked against enpoint

`

4. Acknowledgement- <DTSAcknowledge>Received</DTSAcknowledge> Payload of acknowledgement may or may not contain a “token” in the payload. If the token is present it is expected back as-is. The client is not expected to process the “token” in any manner thus allowing the “token” to be whatever the provider deems necessary.

Data passed to back end for processing.

SBS/FAMS DTS Requestor

DTS Responder submitDTS()

Processing performed in the middle of accepting request and delivering processed result- Time frame of this step is immaterial

DTS Requestor
2. submitDTS invoked against enpoint; <DTSPayloadType>DTSRetrieve</DTSPayloadType>

DTS Responder submitDTS()
Back-end Processing Response Data passed to transport for delivery

Data passed to back end for processing.

`

4. Response- processed result returned as payload

f 1. yo Cli ke en lic t co ub riz e p o n ta cts tain uth ob nd a C to p ro R to R ate a ce C sse obta ts tic d r in e tac en esu n dp on uth lts oin e c to a . t to rvic er If token was received in response to r et Se end rie s 3. original request and client wants that ve Central Repository specific response , the token must be (CR) passed as payload. Otherwise, first Key/URL/Services response available for that client would be returned. Subsequent “DTSRetireve” would get next Exact same repository as the available response . CR at top of page

24

DTS Analogy
• DTS is the definition of the “Pipe” and the  structure of its contents
– The “Pipe” is the internet – The content is SOAP – The end points/junctions are Web Services – The sources are Web Service enabled clients
25

DTS Analogy
• DTS defines how others can connect to  the “Pipe” already installed
– Any connections must have certain  “threads” – Any connections must handle two way  traffic independent of how the traffic will  be used
26

DTS Analogy
• By knowing about the  pipe and the type of  connections, any  “plumber” can use  his/her own tools to  make connections;  just so long as the  threads match
27

Extending the Analogy
• We all have plumbing and fixtures • Very unlikely we all have the same  type of fixtures • Yet our water companies still deliver to  us all • All our fixtures use (“process”) it • All our drains return it

28

How Did We Do It?
• Created basic HelloWorld service and  client • Added simple Headers to HelloWorld • Added complex Header to HelloWorld
– Was not interoperable
29

– Worked interoperable

– Was not interoperable

Why SOAP Headers
• To answer routing and processing  expectations without opening the  payload • Remain payload insensitive • Allow extensibility for new  processes

30

• • • • • • • •

DTSRequestRouting DTSRequestServiceExpectation  DTSRequestPayloadType DTSRequestSignature DTSResponseRouting DTSResponseAcknowledge DTSResponsePayloadType DTSResponseSignature

DTS SOAP Headers

31

Convoluted Filename vs Header  Elements
• A [B] <X.Y.Z:M>    A = File Type, B = Encrytption, X.Y.Z = key  • Encryption unnecessary because using HTTPS • DTSRequestPayloadType = A • DTSRequestRouting
– SourceIDSubCode = X, SourceID =Y(.Z) – UUID = M
32

identifier, M = Unique message ID

Interop Problem with SOAP  Headers
• xsi:type attribute in Header  elements
– Java includes and requires this  attribute – .Net does not
33

All about SOAP
<soap:Header> <DTSRequestPayloadType  xsi:type="DTSRequestPayloadType"  xmlns="http://www.datatransportstandard.com"> <value>CRC01Request</value> </DTSRequestPayloadCode>
34

SOAP is the Key
• The SOAP transmitted across the wire  is of primary importance
– Element names – Type attribute –  Not Namespace moniker (Java uses one  by default, .Net does not)

• How you get the correct SOAP is not  important

35

Java: <ns1:DTSRequestSignature  soapenv:mustUnderstand="0"  xsi:type="ns1:DTSRequestSignature"  xmlns:ns1="http://www.datatransportstandard.com"> <ns1:value>SignatureValue</ns1:value> </ns1:DTSRequestSignature> .Net: <DTSRequestSignature  xsi:type="DTSRequestSignature"  xmlns="http://www.datatransportstandard.com"> <value>SignatureValue</value> </DTSRequestSignature>

SOAP Differences That Do Not Matter

36

Reference Implementation  Architecture • • • • Client Application Client Core Service Core Service Application
37

Client Application
• Knows nothing of SOAP or Web  Services • Implements Client Core Interface
– “Setters” and “Getters” of DTS  specific elements

• Houses specific business logic

38

Client Core
• Knows nothing of business logic • Uses properties set to construct the  SOAP • Interface for “setting send” and  “getting returned” elements • Handles the communication to Service  Core­ DTS Specification
39

Service Core
• Accepts transmissions from Client Core • Implements Service Application Interface • Creates return SOAP 
– “Setters” and “Getters” of DTS specific  elements

– Format return acknowledgement or data from  Service Application – Construct SOAP faults
40

Service Core (continued)
• Isolated business logic
– Examples
• Invoke Service Application based on  payload • Place payload in “queue”
41

Service Application
• Interface for “setting sent” and  “getting to be returned” elements • Houses specific business logic • Knows nothing of SOAP or Web  Services
42

Connecting the layers
Entity A
Client Application Service Application

Platform specific communication

Entity B

Client Core

Service Core

Platform specific communication
DTS Spec

Internet

DTS Spec

43

Connecting the layers
Entity A
Client Application Service Application Client Application

Platform specific communication

Entity B

Client Core

Service Core

Platform specific communication
DTS Spec

Internet

DTS Spec

44

Connecting the layers
Entity A
Client Application Service Application Client Application

Platform specific communication

Entity B

Client Application

Client Core

Service Core

Platform specific communication
DTS Spec

Internet

DTS Spec

45

Connecting the layers
Entity A
Client Application

Platform specific communication

Entity B
Service Application

Service Application

Client Application

Client Application

Client Core

Service Core

Platform specific communication
DTS Spec

Internet

DTS Spec

46

Connecting the layers
Platform, but DTS interface

ESB

Platform, but DTS interface

Point A
Client Application Service Application

Point B

Service Application

Client Application

Client Application

Client Core

Service Core

DTS Spec

Internet

DTS Spec

47

Additional DTS Information
• Visit PESC at www.pesc.org  • Materials available include
– Executive summaries – Specifications – Reference (proof of concept)  implementations

48

Adding SOAP Headers
• Change WSDL – Still has  problems • Create Container Classes • Container Classes require  serialization/de­serialization  directives

49

Adding SOAP Headers
• Augment Service Code • Augment Client Code

50

Java Creating the Container Classes for the Service (IN)
package com.datatransportstandard.www.serializable; import java.io.Serializable; public class DTSRequestRouting implements Serializable { private String sourceID  = null; private String sourceIDCode = null; private String recipientID = null; private String recipientIDCode = null; private String uuid = null; private String transmissionDateTime = null; public String getSourceID() { return sourceID; public void setSourceID(String newSourceID) { sourceID = newSourceID; } }
public String getRecipientID()  { return recipientID; } public void setRecipientID(String newRecipientID) { recipientID = newRecipientID; } public String getRecipientIDCode()  { return recipientIDCode; } public void setRecipientIDCode(String newRecipientIDCode) { recipientIDCode = newRecipientIDCode; } public String getUUID() { return uuid; } public void setUUID(String newUUID) { uuid = newUUID; } public String getTransmissionDateTime() { return transmissionDateTime; } public void setTransmissionDateTime(String newTransDateTime) { transmissionDateTime = newTransDateTime; } }

public String getSourceIDCode() { return sourceIDCode; } public void setSourceIDCode(String newSourceIDCode) { sourceIDCode = newSourceIDCode; }

51

Java Creating the Container Classes for the Service  (OUT)
public String getRecipientID()  { return recipientID; } public void setRecipientID(String newRecipientID) { recipientID = newRecipientID; } public String getRecipientIDCode()  { return recipientIDCode; } public void setRecipientIDCode(String newRecipientIDCode) { recipientIDCode = newRecipientIDCode; } public String getUUID() { return uuid; } public void setUUID(String newUUID) { uuid = newUUID; } public String getTransmissionDateTime() { return transmissionDateTime; } public void setTransmissionDateTime(String newTransDateTime) { transmissionDateTime = newTransDateTime; } }

package com.datatransportstandard.www.serializable; import java.io.Serializable; public class DTSResponseRouting implements Serializable { private String sourceID  = null; private String sourceIDCode = null; private String recipientID = null; private String recipientIDCode = null; private String uuid private String transmissionDateTime = null; = null;

public String getSourceID() { return sourceID; } public void setSourceID(String newSourceID) { sourceID = newSourceID; } public String getSourceIDCode() { return sourceIDCode; } public void setSourceIDCode(String newSourceIDCode) { sourceIDCode = newSourceIDCode; }

52

Java Examples (Service) 
Modify the WSDD 

<beanMapping  languageSpecificType="java:com.datatransportstandard.www.serializable.DTSRequestRouting"  qname="ns2:DTSRequestRouting" xmlns:ns2="http://www.datatransport.com"/> <beanMapping  languageSpecificType="java:com.datatransportstandard.www.serializable.DTSResponseRouting " qname="ns3:DTSResponseRouting" xmlns:ns3="http://www.datatranportstandard.com"/>

Augment the Code (inbound)
SOAPEnvelope requestEnv = 

MessageContext ctx = MessageContext.getCurrentContext(); ctx.getRequestMessage().getSOAPEnvelope();

SOAPHeaderElement requestHeader =  requestEnv.getHeaderByName("http://www.datatransportstandard.com", “DTSRequestRouting"); DTSRequestRouting incontainer = (DTSRequestRouting)  requestHeader.getObjectValue(); System.out.println(“SourceId = “ + incontainer.getSourceId());

53

Java Examples (Service)
• Augment the Code (outbound)
DTSResponseRouting outcontainer = new DTSResponseRouting(); outcontainer.setSourceId(“TEST SOURCE ID”); outcontainer.setSourceIdType(“TEST SOURCE ID TYPE”); … SOAPHeaderElement responseHeader = new SOAPHeaderElement(“ http://www.datatransportstandard.com” ,“DTSResponseRouting”) responseHeader.setObjectValue(outcontainer); SOAPEnvelope responseEnv =  ctx.getResponseMessage().getSOAPEnvelope(); responseEnv.addHeader(responseHeader);

54

Java Examples (Client)

• Augment the Code (General)

QName qn = new  QName(“http://www.datatransportstandard.com”,“DTSRequestRouting”); call.registerTypeMapping(DTSRequestRouting.class, qn, new  BeanSerializerFactory(DTSRequestRouting.class, qn), new  BeanDeserializerFactory(DTSRequestRouting.class, qn)); QName qn1 = new  QName(“http://www.datatranpsortstandard.com”,“DTSResponseRouting”); call.registerTypeMapping(DTSResponseRouting.class, qn1, new  BeanSerializerFactory(DTSResponseRouting.class, qn1), new  BeanDeserializerFactory(DTSResponseRouting.class, qn1));
55

Java Examples (Client)

• Augment the Code (outbound)
DTSRequestRouting outcontainer = new DTSRequestRouting(); outcontainer.setSourceId(“TEST SOURCE ID”); outcontainer.setSourceIdType(“TEST SOURCE ID TYPE”); … SOAPHeaderElement requestHeader = new  SOAPHeaderElement(“http://www.datatransportstandard.com”,  “DTSRequestRouting”);      requestHeader.setObjectValue(outcontainer); call.addHeader(requestHeader);
56

Java Examples (Client)

• Augment the Code (inbound)

SOAPEnvelope responseEnv =  call.getMessageContext().getResponseMessage().getSOAPEnvelope() ; SOAPHeaderElement responseHeader =  responseEnv.getHeaderByName(“http://www.datatransportstand ard.com”,“DTSResposneHeader”); DTSResponseHeader incontainer = (DTSResponseHeader)  responseHeader.getObjectValue(); System.out.println(“Source Id=“ + intcontainer.getSourceId());

57

.Net: Creating the Container classes for Service
[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com”)]
[XmlRootAttribute(ElementName=“DTSRequestRouting”, Namespace=“http://www.datatransportstandard.com”, IsNullable=false)] [XmlInclude(typeof(DTSRequestRouting))] public class DTSRequestRouting : System.Web.Services.Protocols.SoapHeader  { public string UUID; public string transmissionDateTime; public string sourceID; public string sourceIDCode; public string recipientID; } [XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com”)] [XmlRootAttribute(ElementName=“DTSResponseRouting”, Namespace=“http://www.datatransportstandard.com”, IsNullable=false)] [XmlInclude(typeof(DTSResponseRouting))] public class DTSResponseRoutingElements : System.Web.Services.Protocols.SoapHeader  {     public string UUID; public string transmissionDateTime; public string sourceID; public string sourceIDCode; public string recipientID; } [XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com")] public class DTSResponseRouting : DTSResponseRoutingElements{};

58

.Net: Augment the Service

• Add declarations to service Class
public DTSRequestRouting public DTSResponseRoutingElements public SoapUnknownHeader[]

DTSRequestRoutingVal; DTSResponseRoutingVal;  unknownHeaders;

• Add serialization directives to  WebMethod()
[SoapHeaderAttribute("DTSRequestRoutingVal")] [SoapHeaderAttribute("DTSResponseRoutingVal",  Direction=SoapHeaderDirection.Out)] [SoapHeader("unknownHeaders")]

59

.Net: Creating the Container classes for Client
[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")] [XmlRootAttribute(ElementName="DTSRequestRouting",Namespace="http://www.datatransportstandard.com",  IsNullable=false)] [XmlInclude(typeof(DTSRequestRouting))] public class DTSRequestRoutingElements : System.Web.Services.Protocols.SoapHeader  { public string UUID; public string transmissionDateTime; public string sourceID; public string sourceIDCode; public string recipientID; } [XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")] public class DTSRequestRouting : DTSRequestRoutingElements{} [XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")] [XmlIncludeAttribute(typeof(DTSResponseRouting))] [XmlRootAttribute("DTSResponseRouting", Namespace="http://www.datatransportstandard.com", IsNullable=false)] public class DTSResponseRouting : System.Web.Services.Protocols.SoapHeader  {         public string UUID;         public string transmissionDateTime;         public string sourceID;         public string sourceIDCode;         public string recipientID;  }

60

.Net: Augment the Client • Add declarations to Client Web Reference/Proxy  Class
public DTSRequestRoutingElements  DTSRequestRoutingVal; public DTSResponseRouting DTSResponseRoutingVal;

• Add serialization directives to WebMethod()   
[SoapHeaderAttribute("DTSRequestRoutingVal")] [SoapHeaderAttribute("DTSResponseRoutingVal",  Direction=SoapHeaderDirection.Out)]

61

Contact Information
We appreciate your feedback and comments.   We can be reached at: Nathan Chitty, Nelnet, Inc. (904) 281­7235 nathan.chitty@nelnet.net Gary Sandler, ELM Resources (510) 903­7960 gsandler@elmresources.com

62