Case Study — Telecom Mass Processing Skybase

Online Cloud Based Data store for a South American Telecom Company Big data Platform

Client—Tier 2 GSM operator in South America. Legal regulations and law enforcement in the country de-
mands the operator to maintain 6 Years of Subscriber Call Detail Records enriched with Business Intelli-
Summary gence for query and easy access. The client has raw data which needs data decoding, enrichment, trans-
formation and DB load to meet compliance requirement. This case study describes how Gamma’s solution
helped the client in compliance .

 Store data in cloud [Amazon s3] which could be accessible to Client-CSP

 Business intelligence to be applied on ASN.1 format MSC files and stored hence parsing, enrichment
& loading required
 Business logic on-the-fly. E.g. Dialed digits analysis to store call destination -country name deciphered
Scope
from B Number , IMSI analysis to derive HPLMN for in bound Roamers
 Massive data volume worth of six years of CDR-s

The retrieved MSC files were not segregated according to their respective zones therefore segregation
logic has to be implemented at the collector level before loading to Amazon S3 cloud environment. Ama-
Challenges zon S3 instances had to be bought for the exact requirement to minimize the cost and meet timeline to
complete the task. Hadoop eco system requires processed input files to be ‘big enough’ . Output files
have to be merged and compressed in LZO format for further processes

 Deploy Gamma’s Big data management platform– Skybase

 Files were segregated according to the Zones and were uploaded in Amazon S3 cloud environment.
 Skybase ETL-Collector collected the Raw MSC files segregated by specific Zones,
 Ericsson MSC De-serializer was customized and deployed to parse binary files, enrich required fields
and generate CSV files
Solution
 Skybase ETL-Loader to collect and load the transformed CSV files using Hadoop’s Map-Reduce func-
tionality and store in Hive DB and subsequently Impala for DB queries
 Processing time requirement - process entire data set in not more than 36 hours

 Customized solution addressing the exact need reducing the overall cost and implementation time
 Lead time on procuring H/W reduced considerably and saved an estimated amount of USD 30,000 on
hardware costs. Further saving for S/W not quantified here but found to be big
 Six years of backlog Data Collected, Enriched and Loaded in a less than 36 hours
 Database fine-tuned for querying desired output
Results  Distributed processing capability of Gamma’s Skybase ETL demonstrated effectively with files pro-
cessed in multiple servers from Amazon on a Timesharing basis
 Solution is fine-tuned and scaled down to handle the current on-line hourly processing of the re-
quired data
 Excess HW released for further savings

Gamma Analytics provides advanced data collection, analysis and management tools based 551 Wilmot Road
info@gammanalytics.com
on a new model of leveraging technology, data science and research methodologies. The New Rochelle,
New York 10804,
+1 914 740 4067
company is headquartered in New York, US with development center in Kolkata, India. Do- www.gammanalytics.com
main expertise includes Telecom, Finance, Social Media, Automobiles & Retail United States