You are on page 1of 51

Xumii Overview

Richard Barnett Xumii Server Architect

Overview & Domain concepts API Internals

What is Xumii?
Aggregator for social network content & messaging  Optimised for mobile clients but supporting many types  Supporting subscription-based usage model  Subscription & billing handled by the customer (mobile carrier)  Supporting large numbers of ³always-on´ clients (³Social Phone´) 

What is Xumii?
Web client

WL Messenger

Native phone client

Xumii server



Other client (USSD, ...)

Customer (mobile carrier) systems (SMS, authentication, subscription, ...)


Where is Xumii Server in the ecosystem?
Operators offer value-added and branded social networking services to their subscribers

Handset Manufacturers
Create new differentiated mass market social phones driving ARPU for operators and increased subscriber demand

Xumii Server
‡ ‡ ‡ ‡

‡ ‡

Hosted White-label Aggregated Social network connectors IM service connectors Push capability

Subscriber gets a more integrated social networking experience.

Sydney:  Server core  Mobile client core Chengdu:  Server community ³adapters´  Mobile client UI  Web client  .Within Myriad...

in production with Optus & some TISA op-cos Maelstrom2 (M2) ± reimplementation of Xumii for the changed Myriad requirements. reusing M1 components where appropriate Concord ± Original name for M2.Some names Xumii ± the brand name for Myriad¶s social platform/services Maelstrom1 (M1) ± legacy implementation of the Xumii concept. started before Myriad acquisition. abandoned due to politics (it gave the impression we were rewriting M1 from scratch). still present in Java package names  .

Domain model (1/2) .

Domain model (2/2) .

Gmail . Flickr.Some domain objects (1) Community External provider of social content & messaging services In general membership is required & content sharing/messaging is restricted to other members that you have a relationship with Social network. other messaging service (eg email) eg Facebook. Windows Live Messenger. instant messaging service.

Some domain objects (2) Consumer Person using a Xumii client to view/update social network content. analytics users etc) Session Period of use of Xumii by a Consumer on a client Concurrent Sessions from multiple clients are supported . operator. send/receive messages. etc (We avoided ³user´ since there are administrator.

Some domain objects (3) Identity Consumer¶s profile on one Community Credentials (eg username/password) required to make Community api calls that authenticate as the Identity 1 per Community per Consumer CommunitySession Transient state require to make Community api calls 1 per Identity ± even when there are concurrent Sessions .

Some domain objects (4) Contact Community member with whom you¶ve established a relationship (eg friend request. adding a new friend Reaction Reaction to an Activity. uploading a photo. follow) Activity An Action in a Community which is visible to some or all of your Contacts Eg updating status. using an app. eg a Like or Comment .

Some domain objects (5) MediaItem Community-hosted media object. Video. Audio Album Collection of MediaItems . eg Photo.

Contents Overview & Domain concepts API Internals .

M2 api HTTP transport Requests. Responses & Notifications  Client sends Request to server  Server sends Response to client when it completes a Request  Server sends Notification to client when there¶s some event the client should know about (social or internal) Asynchronous & synchronous parts  .

Async api (1) Async api uses COMET: 1 url endpoint POST submits 1 or more Requests for async processing. HTTP response is empty Responses & Notifications are queued in the server GET retrieves pending Responses & Notifications. can block until there are some Always requires authentication ± used to correlate POST & GET .

Async api (2) Request properties: Object-type  1=Session. 9=Activity. may be empty Responses & Notifications are similar. 4=delete Message-id Payload .specific to object-type & operation.. 8=MediaItem. but Notifications are only C or D . 3=update. etc. Operation (CRUD)  1=create. 2=read..

Sync api Sync api uses GET or POST: Many url endpoints Request params (GET) or body (POST) encode the Request Blocks until Response is ready HTTP response encodes the Response May be unauthenticated Can still be thought of as a CRUD operation on an object .

length-encoded. ASCII except for any binary data .Encodings 2 encodings are supported: JSON ± standard. text. boolean. easy to read & edit  Primitive values (null. strings)  Arrays  Objects (collection of name-value pairs)  No support for arbitrary binary data ± use base64 RAP ± proprietary. numbers.

Authentication Create Session sync api is used to obtain a Session Token This token is  sent as an HTTP header in async requests  sent as an HTTP request parameter in sync requests where required Can be renewed when expired  .

Partition ³Category´ of Consumers ± immutable association when Consumer is created Represents a customer or subdivision of a customer:  1 Partition for each TISA op-co  1 Partition for Softbank Properties include  Subscription required flag  Methods for calling customer systems  .

Subscription Some apis require subscription. This is only checked for Consumers in a Partition which requires subscription Api calls to view available SubscriptionProducts & to purchase a Subscription are delegated to the customer¶s systems  .

"f":3.1. ³p":"<PWD>´.5" }' http://localhost:8080/api/3/url/json/consumer  .Getting started (1) Create Consumer (sync):  Request contains a set of Community credentials  If they are valid then a Consumer is created with an Identity for the given Community curl -H "Content-Type: application/json" -H 'x-msisdn: 5691234567' -d '{ "u":"<USER>´. "c":0. "t":"MidpTouch". "v":"1.

"p":"<PWD>".1. "c":0. "f":3}."t":"3-67-f4b51e3f:e4e3dc3b1ce". "t":"MidpTouch".5" }' http://localhost:8080/api/3/url/json/session {"c":0. "v":"1. creating CommunitySessions curl -H "Content-Type: application/json" -H 'x-msisdn: 5691234567' -d '{ "cr":{"u":"<USER>"."dl":3600."fs":false}  .Getting started (2) Create Session (sync):  Request contains a set of Community credentials  If they are valid for an existing Identity then a Session is created for the owning Consumer  By default all Identities for the Consumer are signed in.

"f":0. "h":"3911df0e". "a":1."d":{ "c":0."i":null. "s":"" }} . "n":"Leirbag Dairym".Getting started (3)  Long GET curl -H 'X-Xumii-Token: <TOKEN>' http://localhost:8080/api/3/json?wait=5000 {"t":301. "cu":"100002817230770".

"s":"".int/display/API/ObjectsReference Response: {³t´:´<OBJECT_TYPE>0<ACTION>´."p":[2."ul":false."u":"549683637". "t":"".myriad."d":{ "c":0."d":{ "c"". "n":"Leirbag Dairym"."s":2}}."c":0.14]}."a":1. "i":"549683637_328440327180187".. "cu":"100002817230770"."i":"http://. "k":[{"u":"http://twitter. {"t":901. "d":"twitter."pc":{"g":2}}}."t":"Post on Twitter". "tn":"Somebody". "dl":209348088. "h":"3911df0e0ae040a415433900a8f9cfca". "r":{"l":0. "tu":"549683637"."gl":{"n":"Courbevoie"}."v":6.Getting started (4) http://confluence."o":2. {"t":301."d":{"c":0. "ts"". "f":0.´}] }}]} . ³d´:{<DATA>}} Sample: {"m":[ {"t":2401. "n":"RB"..

"cu":"54968637"."s":"Enjoying Paris". "h":"e8abc11"."d":{"data":"count"."i":null. {"t":402."d":{"data":"all"."page":{"i":0."d":{"#":1}}]} ."n":"Somebody". "d":{"r":[ {"c":0."f":2.Getting started (5) Example with Read(2) Contact(4): curl -H "Content-Type: application/json" -H 'X-Xumii-Token: <TOKEN>' -d '{ "m":[ {"t":402."page":{"i":0."a":0."n":100}}}."bd":"04/17/1968"}]}}."n":100}}}]}' http://localhost:8080/api/3/json After a long GET: {"m":[ {"t":402. "i":null. {"t":402.

"t":"http://twitter."u":"549683637".com/#abc". "f":0."v":6. {"t":301."o":2. "r":{"l":0.."s":"". "ts":1325979919000."a":1.myriad. "d":"twitter. {"t":901."d":{ "c":0."c""."d":{ "c":0.´}] }}]} . "k":[{"u":"http://twitter. ³d´:{<DATA>}} Sample: {"m":[ {"t":2401.Getting started (4) http://confluence. "i":"549683637_328440327180187"."p":[2. "n":"RB"."s":2}}. "tu":"549683637"."gl":{"n":"Courbevoie"}. "n":"Leirbag Dairym". "tn":"Somebody".com".14]}. "h":"3911df0e0ae040a415433900a8f9cfca"."t":"Post on Twitter".int/display/API/ObjectsReference Response: {³t´:´<OBJECT_TYPE>0<ACTION>´. "cu":"100002817230770"."i":"http://."pc":{"g":2}}}."d":{"c":0. "dl":209348088.."ul":false.

Contents Overview & Domain concepts API Internals .

each handles a subset of the objects referred to in the api  API layer  Decodes Requests & encodes Responses/Notifications  Routes Requests to the correct Service  Manages the queue of pending Responses/Notifications  Service Bus connecting the above  Interfaces to external systems Can all fit in 1 JVM.Xumii server internals (1)  Xumii server consists of  4 Services. or be spread across many .

RAP.Xumii server internals (2) API endpoints in the API layer use codecs to convert between ³wire format´ (JSON. etc) & an internal messaging format  Codecs are pluggable. so a new API endpoint could use a completely different wire format Service Bus  Allows other components to message each other without knowledge of location  Manages the cluster of Service Instances  Manages load balancing across multiple instances of a single Service  .


User Service Responsible for Objects relating to the Consumer (not to social objects):  Consumer  Session  Subscription  Preferences Calls to customer infrastructure/services (SMS. subscription)  Customer auth services are called from API layer .

Community Service Responsible for All social objects:  Identity  CommunitySession  Contacts  Activity  MediaItem Calls to Community APIs .

Media & Admin Services Media Service is responsible for: Scaling & transcoding of community media content Caching results of the above for efficiency Admin Service is responsible for: Functionality used by non-Consumer users (community applications... schedules.) .

almost all with different apis Xumii abstracts Community-specific code into an Adapter:  1 Adapter per Community  Common interface between Xumii ³Core´ & all Adapters  .Community Adapters Xumii supports > 15 Communities.

Community Feature Matrix (1) http://localhost:8085/admin/communityfeatures/view .

Community Feature Matrix (2) Common interface to Adapters is broken up into multiple Feature interfaces A Feature interface exists for each logical unit of Community api functionality:  ACTIVITIES_QUERY for retrieving Activities  ACTIVITIES_PUBLISH for creating Activities  etc An Adapter implements only the Feature interfaces that the Community api supports  .

Community Feature Matrix can be queried to obtain the implementation for a given Community & feature combination  . and values indicate whether the Feature is implemented for the Community This is embodied in code.Community Feature Matrix (3) This produces a ³matrix´ where the axes are Communities & Features.

OAuth Xumii obtains request token from Twitter marks request token as authorised & redirects to Xumii Twitter asks Consumer to sign in Twitter validates request token & provides access token Xumii calls Twitter api to exchange authorised request token for access token Xumii uses access token to call Twitter api Twitter checks that request token is valid Twitter authorises token and responds . forms auth url for Twitter Twitter asks Consumer to grant Xumii access Xumii stores access token in Identity Consumer follows link to twitter.

Phone Community Implemented by Xumii Contacts identified by phone number Contacts are created by uploading from a mobile address book  Friend request is not sent when you do this ± contacts are ³private´ Messages can be sent (via SMS) to Contacts  .

Operator Community Implemented by Xumii Contacts are identified by MSISDN Contacts are created explicitly or by matching my Contacts in other Communities against existing Identities  If I know you on Facebook. and you have a Facebook Identity in Xumii. server makes us friends in the Operator Community Supports MediaItems for sharing Supports IM messaging  .

xml Interfaces to customer¶s authentication. subscription services can be considered to be analogous to Community apis Operator Feature Matrix is the analogous implementation.Operator Feature Matrix operator-feature-matrix. providing an implementation for a given Partition & Operator Feature combination  . SMS.

session-id. d: { c: 0...Request lifecycle (1) API layer Send message to appropriate service Comet Controller POST /api/3/json X-Xumii-Token: xyz { m: [ { t: 901. object & operation) and request object send(message) send message bytes over TCP to target Service Check subscription status ServiceLocation 200 OK . . } }]} Activity Codec Service Locator Service Caller Parse body to JSON DOM Extract/validate Token Locate Session Validate Object/Operation deserialiseRetrieveRequest(jsonNode) RetrieveActivityRequest RetrieveServiceLocationRequest(Activity) message contains metadata (async/sync flag. n: 10.

Request lifecycle (2) Community Service .

Request lifecycle (3) .API layer Add service response to the session queue .

] } }]} .Request lifecycle (4) API layer Long GET from user to retrieve response Comet Controller GET /api/3/json X-Xumii-Token: xyz ClientMessage Manager ClientMessage Source Activity Codec Extract/validate Token Locate Session getMessages(token) getMessages() Consume Messages from Session s queue List<Message> List<Message> serialiseRetrieveResponse(resp. d: { r: [ .. jsonWriter) 200 OK { m: [ { t: 902..

Database MySQL Relatively simple schema ± close to domain model Social objects are not in general stored  Things relating to sharing are an exception All messages between client & server are audit logged to a DB table  .

5.3: internal messages serialisation Jetty: http server (usually 1 by service)  .x: java framework Hibernate 3.Other infrastructures Spring 2.x: Shindig (OpenSocial server implementation used for Phone/Operator Communities) Openfire/Smack (chat for Operator Community) Protocol Buffers 2.1: java message queue JGroups 2.x: java framework (database) HornetQ 2.

myriad.References API reference: Installation: +Installation+and+Configurati .