Service Control Platform

for Network Computing

Dan Ionescu
University of Ottawa

HAVE Project 2005-2006

January 11, 2010

 NCCT- who we are
 NCCT = Network Computing and Control
Technologies
 A research laboratory of the University of
Ottawa/School of Information Technology and
Engineering
 NCCT – owner of the NCIT*net 2- a state of the
art next generation network
 NCIT*net 2- An excellent research and production
test bed network enabler of hot research topics in
Network Computing and Control

Network Computing and Control Technologies Research Laboratory 2

 NCCT Research Topics

 NCCT research is directed towards a new

vision of Distributed Computing and
Networking,
 Following research areas are considered
hot topics::
 Distributed/Network Computing
 Next Generation Network and Services
 Knowledge Management and Semantic Web

Network Computing and Control Technologies Research Laboratory 3

 NCCT- Research… in more details

 NCCT’s research team undertakes

research in:
 Distributed Computing
 Data Mining, Fusion, Monitoring and
Management : ETL, EAI, EII, in one
 Collaborative Web Desktops – Network
Computing
 Networking and Services:
 Architectures for Next Generation Networks
 Converged Services
 Next Generation Control Platforms via Web
Services
Network Computing and Control Technologies Research Laboratory 4
 NCIT*net2- The Research Infrastructure

 DWDM transport infrastructure and services

 RPR packet transport, based on SONET/SDH
 MPLS/VPLS transport and services, based on
optical GigE mesh.
 IPv4 and IPv6 connectivity services and
peering with research community.
 NextGen NCP platform to control and monitor
the network.

Network Computing and Control Technologies Research Laboratory 5

 NCIT* net

Network Computing and Control Technologies Research Laboratory 6

 Optical network topology
New network
Old network B5C1
NCIT BANK Carlton Ottawa NRC B5C2

U U B5C3
B4C4

NCIT BANK Carlton Ottawa NRC B1C1

B1C2
U U B1C3
B1C4

OC48 technology
Gig E technology
OC3 technology
Network Computing and Control Technologies Research Laboratory 7
 Logical topology
Carlton
U

NCIT Ottawa
U

NRC
OC48 technology
Gig E technology
OC3 technology
Network Computing and Control Technologies Research Laboratory 8
 NCIT* net 2- An MPLS over DWDM network

Network Computing and Control Technologies Research Laboratory 9

 NCIT* net – services – 0ver SONET

Network Computing and Control Technologies Research Laboratory 10

 NCIT* net – services – 0ver MPLS

Network Computing and Control Technologies Research Laboratory 11

 NCIT* net – connectivity to the world…

Network Computing and Control Technologies Research Laboratory 12

 Distributed/Network Computing
 A Distributed Computing Platform for
the Automation of Information
 Connectors to structure and unstructured
data
 Metadata discovery and manipulation
 Processing Data and Information (KM)
 Data and Information Monitoring
 Data Communication
 Security based on roles and politics

Network Computing and Control Technologies Research Laboratory 13

 Design Issues: A Distributed
Information Automation Platform
 Data Monitoring
 Capability of Data,Extraction, Transformation Moving,
and Loading
 Capability of Integrating Applications and Information
across the enterprise
 Provide secure access and communication at the
object level
 Capability of automatically discover software entities
 Provide a platform control/management center
 Provide an easy to use environment for modeling
data, and information flows seamlessly

Network Computing and Control Technologies Research Laboratory 14

 Architecture Design Issues
 Which middleware to use?
 Grid
 J2EE
 Jini,
 Etc.
 What is the model (the information source) to use,
and how to establish and maintain the
correspondence between the data source and its
intended subject matter:
 A database – keeping the information about the enterprise
employees, or clients, or resources, etc
 A web page/site
 A text file
 What is the semantics of data and how to
represent it in information and knowledge

Network Computing and Control Technologies Research Laboratory 15

 Network Computing

Network Computing and Control Technologies Research Laboratory 16

 An Information Automation
Platform: M3Data
 Provides information flow automation across all
departments
 Integrates & Unifies information
 Monitors & Synchronizes it
 Analyzes it and processes it
 Provides secure access to all data across the organization
 Transforms multiple stove-pipe applications into a single
application and consolidates information into a single,
unified view across the enterprise
 Transforms broken communication links between
organizational units into a single point of authentication and
interaction for company employees

Network Computing and Control Technologies Research Laboratory 17

 M3Data High Level View
 Integrates structured &
unstructured data
 Adapts to the IT
infrastructure of any
enterprise
 Flexible, scalable
architecture for easy
deployment
 Leverages existing
infrastructure assets

 Protects information flow

using complete security
suite internal or external

Network Computing and Control Technologies Research Laboratory 18

 M3Data Architecture
• J2EE & Web Service/SOA
based
– Distributed environment
– Exposed services
• Enterprise Service Bus (ESB)
• Plug-and-play architecture
• Run-time addition of
services
• Service Repository enables
clients to see exposed
services:
– Unified Service
Reporting
– Monitoring/Notification
– Federated Search
– Classification

Network Computing and Control Technologies Research Laboratory 19

 Implementing Platform’s
Versatility: DFPS
 M3Data’s Data Flow Processing Studio (DFPS): a graph
grammar based visual programming environment enables IT
specialists to build & control new, distributed information
automation solutions which involve any combination of the
functions below:
 Accessing
 Cleansing & Processing
 Distributing & Migrating
 Monitoring & Synchronizing
 Unification & Reporting
 Monitoring & Notifying
 Searching & Retrieving
 Classifying
 Logging
from a range of data sources & applications

Network Computing and Control Technologies Research Laboratory 20

 The DFPL in a Studio
TRIGGER
Head Office Data Flow Diagram Branch1
Legacy New
repository Data Converters
repository
source Data
target

Custom
Converter

Data Flow
Data Flow

Migration
operator Branch
2

Network Computing and Control Technologies Research Laboratory 21

 The DFPS

Network Computing and Control Technologies Research Laboratory 22

 M3Data Workflow Engine
 Cooperating M3Data
layers enable execution of
scalable workflow:
 Data Communication
& Transfer Layers to
collect data
 Data Processing &
Notification Layers to
process, organize &
qualify data
 Security Layer to
profile data
 Data Transfer Layer
to distribute data
 All of above under
tight control of the
workflow diagrams

Network Computing and Control Technologies Research Laboratory 23

 Applications,
Solutions
 M3Data can custom-build any
type of web service or interface
with 3rd party applications for:
Users
 Information Unification
 Dashboard, Portal
 Business Intelligence
 Reporting, OLAP, Data
Mining
 Advanced Analytics Web Services
 Search, Text Mining,
Classification
 Simply model the information
or workflow requirements in a
data flow diagram
Data Flow Diagrams
 Layers work together to
execute data flow diagram and
automate data management
tasks triggered by web service
requests

Layers
Network Computing and Control Technologies Research Laboratory 24
 Information Flow Automation:
Structure Data
 Data Migration
 M3Data can quickly and easily migrate data and
information from various legacy repositories into complex
new applications built on top of databases

 Data Monitoring & Synchronization

 M3Data synchronization of monitored data sources and
applications allows data propagation among data
repositories as well as coordination of applications for
automation of business processes

 Data Cleansing & Mitigation

 M3Data can consolidate data and information by
eliminating duplicates as well as determining “one version
of the truth” from a collection of similar data from
disparate data sources

Network Computing and Control Technologies Research Laboratory 25

 Information Flow Automation:
Structure Data (2)
 Data Unification & Reporting
 M3Data can unify and consolidate real-time and historical
data and information into a central repository for custom
reporting software and information portals

 Data Monitoring and Notification

 M3Data Information flows are triggered by monitoring data
repositories for changes and then by sending notification
traps to data management components

 Data Logging
 M3Data creates a log for every transaction that it performs
allowing enterprises to debug their data flows, detect security
violations and meet auditing requirements

Network Computing and Control Technologies Research Laboratory 26

 Automating Unstructured Data
 Advanced Analytics: enterprise search & text
mining (classify & categorize)
 Text mining of unstructured data (e.g. text documents, emails)
is one of the most difficult and complex areas related to
enterprise document management
 M3Data is capable of performing e-discovery tasks in large file
systems including exploring a wide variety of documents and
classifying/categorizing them according to training criteria

 Adaptive Web Crawling

 A general-purpose web crawler normally tries to gather as
many pages as it can from a particular set of sites
 M3Data offers an adaptive and focused crawler designed to only
gather documents on a specific topic which reduces the amount
of network traffic and downloads

Network Computing and Control Technologies Research Laboratory 27

 Automating Unstructured Data
 Email Archive Analysis
 M3Data can explore the email space of the enterprise in order to
improve the customer experience
 M3Data can also search through all CRM and ERP related emails
and documents to establish correlations among them

 Meta Federated Search

 Users can query the intra and external network for enterprise
structured and unstructured data using one single query

 Data Monitoring, Notification, and Logging is also

available for Unstructured data automation
functions as for Structured Data

Network Computing and Control Technologies Research Laboratory 28

 Enterprise Search and Retrieve
 Documents are often stored
in file systems or archived in
structures which do not
reflect their content and
require much time and effort
to find and access

 M3Data is capable of
performing enterprise search
operations in a rapid and
secure way

 M3Data indexes all data for

rapid, scalable, and efficient
find and retrieve operations

 Search queries are subject

to user credential lookups to
ensure that results are
restricted to users of the
appropriate credential level
(i.e. Role)

Network Computing and Control Technologies Research Laboratory 29

 Meta-Federated Search
 M3Data enables users to
search the enterprise in a
unified way

 M3Data indexing algorithm

allows for federated searches
which transform disjoint
structured databases into a
virtual unified database

 M3Data also indexes

unstructured data and
consolidates it in a virtual text
repository providing e-
Discovery capability

 M3Data users can then define

search criteria to query the
virtual database and repository

Network Computing and Control Technologies Research Laboratory 30

 The Control/Management Layer
 Users have the capability of interacting in real-time
with M3Data through Control Layer Interfaces:
 Control Layer Console
 Notification Console
 Search Console
 Communications Console

 Data Flow Processing Studio (DFPS) allows users to

build applications using the DFPL – M3Data iconic
programming language:
 By importing a DFPL diagram into the DFPS library, the
user can reuse the information automation feature just
imported, whenever needed

Network Computing and Control Technologies Research Laboratory 31

 Install/Deploy
M3Data /
Upgrade

Control Center Agents

Control Center
Deployment
Component
s

Logs/Audit
AAA
Roles
Security
Manage
Central Administration

Trace

Queue Managemen
Management t

Debug
Logs/Audit
Journalizer
Network Computing and Control Technologies Research Laboratory 32
 Data Processing functionality:
 Various applications and new M3Data
workflows for different verticals are
assembled, launched and supervised by
M3Data’s cooperating layer functions

Index
Search
Retrieve

Search Security
Domains Create
Monitor Deploy

Notification
Trigger
Data Flow Studio
Manage
4 Train
Taxonom
y
Classify
Classification
Trigger Execute

1 2 3
Monitor Data Data Data
Flow Flow
Dashboa
rd
Portal
Reports
Information
Unification

Network Computing and Control Technologies Research Laboratory 33

M3Data as an IT Controller

Network Computing and Control Technologies Research Laboratory 34

 Implementations
 Secure Information Sharing
 Knowledge Management & Exploitation
 Interoperability
 Systems Integration

Network Computing and Control Technologies Research Laboratory 35

 A Recent Application
 Capability #1 – Joint
Intelligence Picture
 Correlate related information
into a common geospatially
based picture to enhance
Situational Awareness

 Capability #2 – Rapid &

Agile Integration
 Using M3Data graphical studio,
exploit new information
sources on the fly in support of
evolving mission requirements

Network Computing and Control Technologies Research Laboratory 36

 A Web Desktop Collaborative
Environment

Network Computing and Control Technologies Research Laboratory 37

 Converged Services
 Use the same device, media to perform
many tasks
 Some services already here
 IP Telephony (surf the web and talk on the
phone… using a single Internet connection)
 Telephony over cable… same idea
 Surfing the web on the cell phone
 Multimedia collaboration: video calls, IM, …
 Even video telephony is here, although only
isolated instances

Network Computing and Control Technologies Research Laboratory 38

 Collaborating Warriors

Network Computing and Control Technologies Research Laboratory 39

 Collaborative Web

url 1 url 2

Network Computing and Control Technologies Research Laboratory 40

 Conlcusions
 According to Forrester the market for
business intelligence reporting and
analysis exceeded $5.5 billion in 2004
 The information quality market will
pass the $1 billion mark in 2008.
 With multiple sources of data within
one organization; data management
tools are essential to understanding
the business process.
Network Computing and Control Technologies Research Laboratory 41
 Conclusions (2)
 There is need for distributed environment
which can interact, supervise, manage, and
control the enterprise IT infrastructure
 The major emphasize is now on how a
software environment can automatically
deal with data, information and knowledge
quality
 Computing concepts are in a time of big
changes: from a classical approach to a
holistic one

Network Computing and Control Technologies Research Laboratory 42

 Thank You

Network Computing and Control Technologies Research Laboratory 43