THE US NATIONAL VIRTUAL OBSERVATORY

Introduction to Web Services

Matthew J. Graham Caltech

NVOSS 2008 - Web Services

8 Sep 2008

Web service deoccultation

Invocations Strange languages Action at a distance High priesthood

NVOSS 2008 - Web Services

8 Sep 2008

Bluffers guide
Web service (think a (code) library on the web)
A piece of software available over a network with a formal description of how it is called and what it returns that a computer can understand

REST (think HTTP GET and POST) SOAP (think envelopes)

An approach to web services that uses the existing infrastructure of the web An approach to web services that uses an XML-based messaging framework An XML description of a web service (normally SOAP) and how to interact with it
NVOSS 2008 - Web Services 8 Sep 2008 3

WSDL (think a contract)

What we wont cover (look in the book)

Data formats: Security:
I want to use something other than XML
I want to protect my data/resources

Attachments State

I want to upload/download a file

I want the service to remember things

Asynchrony Messaging

Its going to take some time

I want more than just request/response
NVOSS 2008 - Web Services 8 Sep 2008 4

Web services in the VO

SkyNodes, Open SkyQuery and WESIX Footprint Services and STOMP Spectrum Services Registry Interface VOSpace CasJobs CDS VOServices (IRAF)

NVOSS 2008 - Web Services

8 Sep 2008

Numquam ponenda est pluralitas sine necessitate

WWW is the largest, most distributed and scalable application on the planet REST (Representation State Transfer) is an architectural style not an implementation:
Distinguishes resources and operations Each resource is identified by a URI The only operations are the HTTP methods Data reflecting the state of a resource is exchanged (in XML)

NVOSS 2008 - Web Services

8 Sep 2008

REST by analogy
HTTP Method PUT GET POST Think CREATE Description Create a resource with the user specified id /proc analogy Start a new process Get the status of a process Amend a process

RETRIEVE Retrieve a resource representation UPDATE Update a resource/Append to a resource/Create a resource with a server assigned id Delete a resource

DELETE

DELETE

Kill a process

NVOSS 2008 - Web Services

8 Sep 2008

By accident, not by design

Parameters in HTTP GET URLs:

HTTP GET is safe it does nothing else than retrieval HTTP GET, PUT and DELETE are idempotent the effects on the system of one or N identical requests are the same Consider HTTP GET with: Services which maintain idempotency whilst allowing parameterized URLs are accidentally RESTful 85% of web service traffic, 6x faster allegedly
NVOSS 2008 - Web Services 8 Sep 2008 8

http://processes.com/123/status http://processes.com/services?action=getStatus&pid=123

http://processes.com/services/action=cancelJob&pid=123

When to use REST

RESTful services are good for WWW-type functionality Limitations:
No formal description:
WADL not widely used yet Not an issue for pure REST (no parameters in URL) Reliable messaging Message routing Digital signatures

No messaging infrastructure: No message level security:

No resource lifecycle management No transaction support No asynchronous event notification

NVOSS 2008 - Web Services

8 Sep 2008

What is SOAP?
Simple Object/Service-Oriented Access Protocol W3C specification

http://wanderingbarque.com/nonintersecting/2006/11/15/the -s-stands-for-simple/
http://www.w3.org/TR/soap

An XML-based messaging framework for exchanging information between peers in a decentralized, distributed environment Defines the message structure but not the message content (needs other technologies) Fundamentally stateless (no memory of what has happened previously) One-way message exchange paradigm

NVOSS 2008 - Web Services

8 Sep 2008

10

Anatomy of a SOAP message

SOAP envelope
SOAP header (optional)
Routing Transactional Security Contextual User profile Message payload (actual data)

SOAP body

XML namespace and schema

NVOSS 2008 - Web Services

8 Sep 2008

11

SOAP example
Request: <soap:Envelope xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance xmlns:xsd=http://www.w3.org/2001/XMLSchema xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body> <ComovingLineOfSight xmlns="http://skyservice.pha.jhu.edu"> <z>float</z> <hubble>float</hubble> <omega>float</omega> <lambda>float</lambda> </ComovingLineOfSight> </soap:Body> </soap:Envelope> Response: <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body> <ComovingLineOfSightResponse xmlns=http://skyservice.pha.jhu.edu> <ComovingLineOfSightResult>float</ComovingLineOfSightResult> </ComovingLineOfSightResponse> </soap:Body> </soap:Envelope> NVOSS 2008 - Web Services 8 Sep 2008

12

What is WSDL?
Web Services Description Language W3C specification An XML grammar for describing the public interface of a web service in terms of:
its exposed operations the message formats protocol bindings (e.g. SOAP, HTTP) defines format of message content in the SOAP body http://www.w3.org/TR/wsdl

Most commonly used with SOAP

NVOSS 2008 - Web Services

8 Sep 2008

13

Anatomy of a WSDL
<definitions> <types> <message> <portType> <operation> <input> <output> <fault> </operation> </portType> <binding> <service> </definitions>
A WSDL document describes: a set of services which exchange messages the data in the messages is defined in a set of types each message transfer in a service defines an operation operations have bindings to specific implementations using protocols like HTTP

NVOSS 2008 - Web Services

8 Sep 2008

14

WSDL example
<definitions targetNamespace="http://skyservice.pha.jhu.edu" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:s=http://www.w3.org/2001/XMLSchema xmlns:s0=http://skyservice.pha.jhu.edu xmlns="http://schemas.xmlsoap.org/wsdl/"> <types> <s:schema elementFormDefault="qualified" targetNamespace="http://skyservice.pha.jhu.edu"> <s:element name="ComovingLineOfSight> <s:element name="z" type="s:float" /> <s:element name="hubble" type="s:float" /> <s:element name="omega" type="s:float" /> <s:element name="lambda" type="s:float" /> </s:element> <s:element name="ComovingLineOfSightResponse> <s:element name="ComovingLineOfSightResult" type="s:float" /> </s:element> </s:schema> </types> <message name="ComovingLineOfSightSoapIn"> <part name="parameters" element="s0:ComovingLineOfSight" /> </message> <message name="ComovingLineOfSightSoapOut"> <part name="parameters" element="s0:ComovingLineOfSightResponse" /> </message> <portType name="DistanceSoap"> <operation name="ComovingLineOfSight"> <documentation>Return the comoving line of sight distance...</documentation> <input message="s0:ComovingLineOfSightSoapIn" /> <output message="s0:ComovingLineOfSightSoapOut" /> </operation> </portType> <service name="Distance> <port name="DistanceSoap" binding="s0:DistanceSoap> <soap:address location="http://voservices.net/Cosmology/ws_v1_0/Distance.asmx" /> </port> </service> </definitions> NVOSS 2008 - Web Services 8 Sep 2008 15

Using SOAP and WSDL

Start with WSDL (contract-first) vs. start with code (contract-last) Frameworks provide APIs:
to handle SOAP:
(De)serialization: converting code object to/from XML representation Stub: client-side proxy to service Skeleton: server-side to handle request and response

and code bindings derived from WSDL:

Alternate approach:

SOAP (and WSDL) are just XML Use XSLT, curl, JavaScript, etc.

NVOSS 2008 - Web Services

8 Sep 2008

16

Just to scare you (WS-*)

WS-Addressing WS-AtomicTransaction WS-Attachments WS-BaseNotification WS-BPEL WS-BrokeredNotification WS-BusinessActivity WS-CAF WS-Choreography WS-CDL WS-Context WS-Coordination WS-CoordinationFramework WS-Discovery WS-DistributedManagement WS-Enumeration WS-Eventing WS-ExperienceLanguage WS-Federation WS-GAF WS-Inspection WSIL WS-I Basic Profile WS-I Basic Security Profile WS-Manageability WS-Management WS-MetadataExchange WS-Notification WS-Policy WS-PolicyAssertions WS-PolicyAttachment WS-PolicyFramework WS-Polling WS-Provisioning WS-Reliability WS-ReliableMessaging WS-RemotePortals WS-ResourceFramework WS-ResourceLifetime WS-ResourceProperties WS-Routing WS-SecureConversation WS-Security WS-SecurityPolicy WS-Semantics WS-Topic WS-Transaction WS-Transaction Management WS-Transfer WS-Trust ASAP ebXML MTOM SAML SOAP SwA UBL UDDI WSDL XACML XML Encryption XML Signature XKMS

NVOSS 2008 - Web Services

8 Sep 2008

17

Grids and clouds

Services on supercomputers:
Montage on TeraGrid NESSSI

Computing on demand:
Virtualization Providers:
Amazon EC2 Google Apps

Globus
http://www.globus.org
NVOSS 2008 - Web Services 8 Sep 2008 18

Beyond REST
Data event streams are the future of astronomy:
High volume Frequent Low latency

Hugh percentage of traffic with RESTful services is polling status SOAP is overly complicated The proposed solution: XMPP (PubSub)
Polling does not scale Not real time

NVOSS 2008 - Web Services

8 Sep 2008

19

A brief introduction to XMPP/Jabber

Extensible Messaging and Presence Protocol:
Persistent connections Stateful Designed to be an event stream protocol Natively federated and asynchronous Identity, security and presence built in

Jabber servers designed to handle at least 105 concurrent connections Sample message:
<iq type=set from=mjg@nvo.caltech.edu/events to=pubsub.nvocloud.edu id=mjg12345> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <publish node='/jobs/1234-5678-90AB-CDEF'> <job type=datadiscovery id=ivo://some.voevent/> </publish> </pubsub> </iq>

NVOSS 2008 - Web Services

8 Sep 2008

20