Scribd
Upload
201 views15 pages

Advanced Internet Monitoring System Guide

The document describes an Internet Monitoring System (IMS) capable of intercepting internet traffic at 100 Gbps speeds. The IMS has a hardware-based front-end for packet capture and a software-based back-end for control, monitoring, data warehousing, and business intelligence. It can capture each packet in real-time with nanosecond timestamping, recognize sources and destinations, filter duplicate packets, and slice packets to multiple CPUs for processing while recording to disk storage. Key challenges are processing millions of packets per second within nanoseconds and storing terabytes of daily data.

Uploaded by

tranceforge
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
201 views15 pages

Advanced Internet Monitoring System Guide

The document describes an Internet Monitoring System (IMS) capable of intercepting internet traffic at 100 Gbps speeds. The IMS has a hardware-based front-end for packet capture and a software-based back-end for control, monitoring, data warehousing, and business intelligence. It can capture each packet in real-time with nanosecond timestamping, recognize sources and destinations, filter duplicate packets, and slice packets to multiple CPUs for processing while recording to disk storage. Key challenges are processing millions of packets per second within nanoseconds and storing terabytes of daily data.

Uploaded by

tranceforge
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Internet Monitoring System (IMS)

Prepared by Betha Aris <betha.aris@IEEE.org>

Key Point

Intercepting Internet traffic with 100 Gbps speed Hardware-based interception (front-end) Software-based control and monitor (back-end) Realtime Deep Packet Capture/Inspection Data Warehouse Business Intelligence Data Mining

Criterias
Capturing each packet (real-time and without loss) Time-stamping each packet with nanosecond precision Recognizing the sources, destination and protocol Filtering for duplicating packet (relevant packets) Slicing packets to multiple CPUs for processing Recording each packet to disk

Challenges
With 30 million packets (@20Gbps), it means these tasks need to be completed for a packet every 30 nanoseconds. With 10 Gbps in both directions, at least 25 Giga Bytes of data per seconds. It means we need at least 2.160 Tera Bytes of data per day.

Processes

Capturing with passive probe Filtering Mediation and Handover Reconstruction Analysis Back-End Reporting (Software-based

Front-End (Hardware-based Intelligence System)

Intelligence System)

Filtering Criterias

Parameters for Identifying Username & Network Access Identifier L2 L7 Protocol (MAC, VPN, MPLS, IPV4/6, UDP/TCP, FTP, HTTP, DHCP etc) Dial-in Calling Number Identity Cable Modem Identity Generic Keyword Spotting (Regular Expression) RADIUS (Subscribers name) Email (POP, IMAP, SMTP) Webmail Sender and Its Contents Chat/IM Conversation VoIP Encrypted Protocols (Skype, Torrent, VPN and SSL)

Conceptual Framework

Capturing data from the source Filtering Data (Deep Packet Inspection) Decoding and processing data Indexing and data storage Refining data through multiple interactive analysis

Giga Interface Network Processing Units Data Mining & Text Mining Data Warehouse Business Intelligence

Front-End (Provider Edge)


Multiple passive taps (copper/fibre @10Gbps) Multiple network probe (@10Gbps) Mediation Server Network Elements (Switches, Routers, Racks, etc)

Front-End Capabilities

Passive interception IP probes with on-board DPI L2 L7 Protocol detection IP Protocol stack interception MPLS, VLAN, VPN interception

Back-End (LEA Operational Center)

Storage Monitoring Systems Analysis Result Reporting Other customization needs

How IMS Works

Proposed Network Topology


Can be changed based on further requirements/identifications

How NPUs Works

How Indexing Works

Proof of Concept

You might also like