STC CEM Customer Journey Metrics

Functional Requirement Specification

UCE9b: I can use my Mobile for Streaming

Linked to CJ group

UCx9b: I can use my Mobile for Streaming

Verified tmforum Conformant

Version 1.5 (2017-04-23) i
Abstract
STC CEM requires that a Customer Journey (CJ) ICT SQM UI, based upon objective and measurable Customer
Journey Touch-points should be developed for the CJ’s that an STC customer is likely to undergo in his day-
to-day interaction with STC services.
The CEM project focuses on Telecom Customer Experience assurance and aims to provide an end-to-end
Telecommunications Management Forum (TM Forum) based CJ quality monitoring and management
solution for STC.
The System should encompass features which can monitor, analyze and report the service quality and
customer experience of the Touch-points in the TM Forum Customer Experience Lifecycle model, Buying,
Using and Sharing and thus assure optimum Customer Experience on a per Journey, per User basis. (PJPU)

Audience:
This document is aimed primarily at:
• STC and Huawei CEM Engineers & Management
• Huawei product developing engineers

Modification record
Title Description Contact Information
Document Owner The person who wrote or owns this Keith Gardiner (KWX306548)
document
Change Log 1 27/10/2016 Draft 1.1 First draft

Change Log 2 19 /12/2016 V1.2 M. Zohairy

Peer Review comments.
Change Log 3 14/04/2014: V 1.3, Merged UC, Keith Gardiner (KWX306548)
UCP, UCPE, UCE. Added TCP & DNS
metrics
Change Log 4 Updated Chapter 1 with clear Alan Hao 00725379 & Feng
functional requirements Jianzhao 00230188
Change Log 5 23/04/2017 Vn 1.5 Updated Keith Gardiner (KWX306548)
Dimension MCC (Mobile Country
Code) to enable Inbound Roaming
monitoring.
Service Owner
Review TM Forum Initial Conformance Review by TM Alfred Anaya-Dubernard
V1R2 Forum aanaya@tmforum.org

Version 1.5 (2017-04-23) ii

Contents

1 Overview ...................................................................................................................................... 1
1.1 Business Value: Customer Journey Dashboard ............................................................................................................. 1
1.2 Objective: ...................................................................................................................................................................... 1
1.3 Functional Requirements Overview:............................................................................................................................. 2
1.4 Functional Requirements: ............................................................................................................................................. 3
1.5 Integration Requirements ............................................................................................................................................. 5
1.6 Abbreviations ................................................................................................................................................................ 5
1.7 FRS Document Methodology: ....................................................................................................................................... 6
2 UCx9b: I can use my Mobile for Streaming .......................................................................... 7
2.1 Functional Value: .......................................................................................................................................................... 7
2.2 Solution Description ..................................................................................................................................................... 8
2.2.1 Customer Flow ........................................................................................................................................................... 8
2.2.2 TM Forum Metrics proposed. .................................................................................................................................. 10
2.2.3 Metrics required. ..................................................................................................................................................... 14
2.3 Mobile Streaming system Chart .................................................................................................................................. 17
2.4 Data to deliver the metrics ......................................................................................................................................... 19
2.4.1 System Flow ............................................................................................................................................................. 19
2.4.2 Formula. ................................................................................................................................................................... 21
2.4.3 System example data ............................................................................................................................................... 23

Version 1.5 (2017-04-23) iii

1 Overview

1.1 Business Value: Customer Journey Dashboard

A Customer Journey (CJ) is comprised of multiple Touch-points, (TP) where a Customer has interactions
with a Telco Operator during their day-to-day transactions, usually across several of the Telco Operator’s
Business domains and Access Channels. (e.g. through retail outlets, the Web, Telco network and User
Equipment (UE))
In most Telco’s, there are structured Organizational silo’s where each Business unit controls his, and his-
only domain and what this means is that no single unit ever has total visibility or, more importantly,
accountability for the entire Customer Journey experience.
Looking solely at individual transactions makes it difficult for the Telco Operator to exactly identify where
to direct improvement efforts because performance data is also stored in silo’s and are monitored by
several, equally silo’d individuals making it difficult to reconcile the impact of failures upon other
domains.
Cumulative bad experiences within a CJ through any Access Channel will quickly lead to a poor Customer
Experience Index.
By introducing a way of monitoring and actioning a degraded CJ that a Customer may experience, by
means of defined threshold values for each Touchpoint within that CJ, the Operator is provided with an
End-to-end view of the Customer Experience and will discover more-effective ways to collaborate across
functions units (the silo’s), and provide a process that delivers gains throughout the Telco Operator’s
Organization, for example:
• Improved Customer Experience,
• Reduced Customer Complaints,
• Reduced churn,
• Increased revenue.
• More effective equipment utilization.

1.2 Objective:
• The objective is to cover the technology-related lifecycle of the customer included in the TM Forum
model of Buying, Using and Sharing and to measure the user experience at each touch point.
• Based on that, STC can ensure and improve the customer experience on all targeted journeys.

Version 1.5 (2017-04-23) 1

1.3 Functional Requirements Overview:
1. A customer’s interaction with STC has been defined by means of Customer journeys. Customer
journeys are a recognised industry wide technique used to measure moments of truth with
customers and how customers are treated.

2. The value in customer journeys is in ascertaining for each SMOT (Single Moment of Truth) what
the customer expects and what level of service quality is delivered to the customer. This is
recorded by means of metrics.

3. Many Customer Journeys provide customers experience in totality and a customer’s interactions
must be able to be assessed not only in one journey but across all journeys they are involved
within.

4. STC’s customer journey framework is based upon decomposition of TM Forums CEM structure.
This is displayed within TM FORUM’s Customer Experience Lifecycle Model, whereby solution
should be able to track the experience of each customer across all the phases of customer’s
typical lifecycle with a service provider i.e. Buying, Using and Sharing.

Reference: TM FORUM GB962-C

Version 1.5 (2017-04-23) 2

1.4 Functional Requirements:
1. Monitoring Capability

Service alarms should be raised when the performance is outside user specified norms of
tolerance. These should allow, if not acted upon by the team within a specified period, the auto
creation of trouble tickets which will be assigned to the correct remedy queue and escalated as
appropriate through the organization.

Provide real-time Consumer & Enterprise CX monitoring and tracking. Provide flexible and
configurable alarms and trouble ticket generation in case of CX breaches at various levels like
Customer, Service, Journey, Touchpoint, NE and Area.

2. Analysis Capability

Ability to query defined metrics based on required dimensions (such as channels, customer type,
service type and etc) to perform needed analysis. Capability of combination of certain dimensions
such as customer type, technology type and etc. All ICT and NW topology starting from channels till
all contributed NEs need to be covered. Comparison capability should be provided for historical
analysis on same metrics and similar objectives.

3. Filtering Capability

The creation of a capability to filter criteria within a dashboard screen to provide information to
Analysis use cases. This filter should be able to be saved for subsequent rerunning of such use case
queries. By adding this filter concept we are effectively creating a dynamic dashboard that analysis
can use to answer specific use case queries. The ability to engage extra filtering requirements,
such as those customers who don’t receive the service that they are provisioned for. This requires
alignment with use cases of analysis.

4. Drill Down Capability

All dash boards need to contain a drill down capability to the xDR and signaling by customer and so
provide in-context drill down to root causes of all CX issues enabling step-wise troubleshooting. The
ability to report summaries of root-causes that have distressed customers. Should provide message
flow schematics based upon direct NE integration.

Version 1.5 (2017-04-23) 3

 5. Reporting & Analytic Capability

The CJ should have Big Data, Real-time Processing, and Analytics as needed to process and analyse
customer experience and service quality data from a variety of high-volume data sources and allow
multi-dimensional analytical queries for analysis and reporting needs. Capability of xDR and SDR
extraction and correlation; storage to keep final results for historical analysis purpose; Mata-data
management; Flexibility of multi-dimension analytic and reporting; UI for end user to perform
direct analysis activities and Integration of other 3rd party reporting tools.

6. Dynamic Dashboard

Dashboards should have the ability to be user configurable and have the capability for the user to
dynamically select metrics from an available catalogue. These should be able to be amended as per
the user’s individual requirements so that the metrics can be designed by user interaction to
provide amended / new metrics using the source data that reflect the needs of the user.

7. Single view of customer Capability

Across- technology tracing capability to enable unified visibility to customers experience issues
between Fixed and Mobile. Provide End to End automatic data correlation required to cover the CJ
via all TPs from related sources. Ability to link multiple product/services to customer using unique
IDs, accounts, etc. Capability to let end user query with multiple type of user identification such as
MSISDN, IMSI, IMEI, Telephone number and etc.

8. Metrics Customization Capability

Allow extensive customization capabilities for introducing new services, metrics, KPIs/KQIs,
business/data flow changes, topology changes etc. without any development or changes needed in
the core product.

Provide a dynamic feature of building new CJ’s with customized TPs and related metrics and
dimensions.

9. Export Capability

Version 1.5 (2017-04-23) 4

 All information on the dashboard can be exported as required by end user per agreed format.

10. System Availability and Stability

System resource will be dedicated for GUI features which should be impacted by other DB related
activities.

11. Dashboard Time Granularity

The Dashboard should allow the user to choose various time-granularities as follows:

15 minutes, 1 hour, 24 Hours, 7 Days, 1 Month 3 Months.

1.5 Integration Requirements

The product should be fully integrated with any required STC systems/Tools in order to
demonstrate an E2E view of the service quality and customer experience of the services and
metrics included in the scope. These integrations should enable the solution to provide E2E
service quality visibility and correlation starting from Physical layer till Application layer, and
vendor should demonstrate how the solution along with existing STC systems can be used to find
the root cause of service experience issues with possible ICT faults and degradations in lower
layers including physical connectivity layer.

1.6 Abbreviations

Abbreviation Description

CEM Customer Experience Management

TM Forum/TM FORUM TM Forum (www.TM Forumorum.org)

SoW Scope/Statement of Work

HLD High Level Design

LLD Low Level Design

HW Hardware

Version 1.5 (2017-04-23) 5

 SW Software

KPI Key Performance Indicators

KQI Key Quality Indicators

QoS Quality of Service

CEI Customer Experience Index

CX Customer Experience

CSAT Customer Satisfaction

NPS Net Promoter Score

OSS Operations Support Systems

BSS Business Support Systems

DPI Deep Packet Inspection

1.7 FRS Document Methodology:

• The Functional Requirements Specification is derived from the development of a straw-man
customer journey discussed with STC, where the activities / steps that a customer will engage in are
documented.
• Metrics, based upon the TM FORUM Metric Catalogue are subsequently aligned to the journey, with
metrics attached to a customer step. These metrics display the efficiency that the customer has
experienced at each step.
• The various Technological functional units involved at each Touch-point are then identified and
captured in a Sequence-flow diagram.
• This then enables further analysis of the required STC metrics and functional units involved for each
identified Customer Journey by the downstream P&S (Huawei Product& Support) so that they can
determine, in conjunction with STC, how to capture and represent this information. This will allow
P&S to deliver HLD and LLD documents which will drive a customer experience management
product that manages the delivery of STC services based upon a an End-to-end Customer Journey
view.

Version 1.5 (2017-04-23) 6

2 UCx9b: I can use my Mobile for Streaming

Mobile for Streaming is an important journey for a customer. Apart from the Network that supports the
interaction, the IT systems which monitor and allow the dispatch, based on the Customer’s Top-up balance, also
need to be monitored for correct functionality.
By analyzing customers and not just pure network or IT transactions, we can understand how a customer is
treated end to end within the call experience regardless of the tools that are used to accomplish the journey.
Our use of dimensions allows us to drill down to understand issues that customers may have, for example by
region, by delay in Top-up activation and so-on.

Note: This document merges Prepaid, Postpaid, Consumer and Enterprise so a filter will be
necessary to enable a Dashboard user to be able to choose any single type, multiples of, or all
of the 4 CJ’s mentioned.
I. UC9b: I can use my Mobile for Streaming
II. UCP9b: I can use my Mobile for Streaming
III. UCE9b: I can use my Mobile for Streaming
IV. UCPE9b: I can use my Mobile for Streaming

Note b) The addition of the MCC (Mobile Country Code) to the Dimensions field enables the capture of Inbound
Roamers’ quantity and usage pattern although, since the Inbound Roamers’ profile is unknown to STC the
value is limited. However, it can be useful for monitoring the number of Inbound Roamers, the Data usage and
Destination ID data.
It is therefore necessary to enable filtering on the MCC.

2.1 Functional Value:

The Mobile for Streaming CJ will allow the CEM team to:
1. Monitor the performance of the Streaming delivery system, identify issues with the Service Provider and
interfacing IT systems.
2. Monitor the performance of the associated Network that delivers the Streaming Service to the Customer
and identify issues.
3. Monitor interactions within STC OSS and set suitable benchmarking.

Version 1.5 (2017-04-23) 7

 4. Monitor daily work against the benchmarking and resulting alerts.
5. Provide demarcation for identified issues and drive resolution thereof.
6. In conjunction with the Analysis team, provide insight to common problems.
7. Allow problem resolution before other customers are affected.

2.2 Solution Description

2.2.1 Customer Flow

The Customer Journey is measured from the time a mobile user presses “Play” on his UE. The sequence of
signaling messages that follow in order to set up the streaming flow between his UE and the Media Service
Provider are measured, followed by measurement of the streaming download performance.
PR1, PR2

TRIGGERS

Customer Informed
wants to unable to
access connect
Internet

Web access,
TCP Receives
Customer connect download
connect confirmation Disconnect
sends URL data,
request connect
interact.

DNS query
Handoff to Use

For each customer step we have recorded thinking regarding what is required to make the customer happy,
satisfied or unhappy. By the use of metrics, we should be able to measure success if the desired driver is being
provided to elicit customer happy responses.

Version 1.5 (2017-04-23) 8

 PR1, PR2

TRIGGERS

Informed
Customer unable to
wishes to connect
stream media

Web access,
Customer TCP Receives
connect download
presses connect confirmation Disconnect
data,
“Play” request connect
interact.

DNS query
Positive Experience

Handoff to Use

Connects Request sent I know there is a I receive connection My download is fast The experience was
instantly without delay reason why no instantly and error free. No satisfactory
service inc stall or disconnects
blocked, no credit on streaming
etc
Browser Request sent I know there is a I receive connection There is minimal The experience was
connects without delay reason why no instantly delay, and minimal acceptable
instantly service stalling or
Negative Experience

disconnects on
streaming
Issues The data seems to I’m left confused There is a delay on Delay is excessive, The experience was
connecting App connect slowly if at connection Stall and very poor
all disconnects are
frequent on
streaming

Thus, we identify what functional area owns the delivery of service to the customer at each step. Metrics
based upon TM FORUM can be attached to each stage.

TRIGGERS

Informed
Customer JOURNEYS
unable to
wishes to
connect
stream media

Web access,
Receives
Customer TCP download
connect confirmation Disconnect
presses connect data,
connect
“Play” request interact.

3)
DNS query CO-C-20 Streaming Stall Frequency
CO-C-20a # Streaming Stalls per Streaming Session
CO-C-21 % Time Streaming Stalled
CO-C-21a # Seconds per Streaming Stall
CO-C-22 % Streaming Sessions Disconnected
CO-C-22a # Kb/s Streaming Download Throughput

1) 2)
CO-C-15 % TCP Connect Success CO-C-16 % Get Response Success
CO-C-17 % Initial Buffer Success
CO-C-18 % Streaming Response Success
CO-C-19 # Seconds Streaming Response

System •- Network •Firewall/Net •Network •Network /

work//CRM/B Firewall
illing

KPI 1) None 2) 3)

Version 1.5 (2017-04-23) 9

 At this point, there are certain Pre-requisites that will determine whether-or-not the
mobile-user will be able to enjoy Mobile Streaming as follows:
PR1 • Firewall issues.
PR2 • Is there RF Coverage?
• Is there Network Congestion?
• Has the mobile-user sufficient pre-paid Credit to access the
Internet?
• Is the handset configured correctly?
• Does the Customer profile configuration allow such a call?
• Is the A-party attached?

These pre-requisites are beyond the scope of this CJ.

2.2.2 TM Forum Metrics proposed.

The following TM FORUM based metrics are proposed to enable the analysis of the Mobile Streaming customer
journey.

CO-C-15 % TCP Connect Success

# TCP_Ack messages/# TCP_Syn messages
Dimensions: Customer ID, Device ID, UE ID, MCC, Location, Originating Cell ID, Target IP address, Technology Ty
3G, TD-LTE, FD-LTE….}
Units: %
Capture Period: 15 minutes
Value: Used to measure media streaming experience
Comments: None

CO-C-16 % Get Response Success

# Initial HTTP Reply Data messages/ # Initial HTTP Get messages
Dimensions: Customer ID, Device ID, UE ID, MCC, Location, Originating Cell ID, Target IP address, Technology Ty
3G, TD-LTE, FD-LTE….}, Media Type {720P Video, 1080P Video…}
Units: %
Capture Period: 15 minutes
Value: Used to measure media streaming experience
Comments: Media Type may not be known

CO-C-17 % Initial Buffer Success

# TCP Media Data (Initial Buffer Fully Occupied) messages /# Associated HTTP Get messages
Dimensions: Customer ID, Device ID, UE ID, MCC, Location, Originating Cell ID, Target IP address, Technology Ty

Version 1.5 (2017-04-23) 10

 3G, TD-LTE, FD-LTE….}, Media Type {720P Video, 1080P Video…}
Units: %
Capture Period: 15 minutes
Value: Used to measure media streaming experience
Comments: “associated” means occurring in the same transaction sequence (thread). Media Type may not be
known

CO-C-18 % Streaming Response Success

[% TCP Connect Success]*[% GET Response Success]*[% Initial Buffer Success]
Dimensions: Customer ID, Device ID, UE ID, MCC, Location, Originating Cell ID, Target IP address, Technology Ty
3G, TD-LTE, FD-LTE….}, Media Type {720P Video, 1080P Video…}
Units: %
Capture Period: 15 minutes
Value: Used to measure media streaming experience
Comments: Media Type may not be known

CO-C-19 # Seconds Streaming Response

SUM [Time of TCP media data (initial buffer fully occupied) message – Time of associated TCP-SYN message]
/# TCP-SYN messages
Dimensions: Customer ID, Device ID, UE ID, MCC, Location, Originating Cell ID, Target IP address, Technology Ty
3G, TD-LTE, FD-LTE….}, Media Type {720P Video, 1080P Video…}
Units: Seconds
Capture Period: 15 minutes
Value: Used to measure media streaming experience
Comments: “associated” means occurring in the same transaction sequence (thread). Media Type may not be
known

CO-C-20 Streaming Stall Frequency

#TCP media (buffer empty) messages/ Sum [Time of TCP media (buffer empty) or RST/FIN message – Time
of preceding TCP media (buffer fully occupied) message]
Dimensions: Customer ID, Device ID, UE ID, MCC, Location, Originating Cell ID, Target IP address, Technology Ty
3G, TD-LTE, FD-LTE….}, Media Type {720P Video, 1080P Video…}
Units: Stalls/Hour
Capture Period: 15 minutes
Value: Used to measure media streaming experience
Comments: Equation represents the number of stalls divided by the total time that the media is actually playing.
The time when the media is stalled is not included in the denominator. Media Type may not be known

Version 1.5 (2017-04-23) 11

 CO-C-20a # Streaming Stalls per Streaming Session
#TCP media (buffer empty) messages/ # TCP Media Data (Initial Buffer Fully Occupied) messages
Dimensions: Customer ID, Device ID, UE ID, MCC, Location, Originating Cell ID, Target IP address, Technology Ty
3G, TD-LTE, FD-LTE….}, Media Type {720P Video, 1080P Video…}
Units: Stalls/Session
Capture Period: 15 minutes
Value: Used to measure media streaming experience
Comments: Equation represents the number of stalls divided by the total number of streaming sessions. Media
Type may not be known

CO-C-21 % Time Streaming Stalled

SUM [Time of TCP media (buffer fully occupied) message - Time of preceding TCP media (buffer empty)
message]/ SUM [Time of RST/FIN message – Time of TCP media data (initial buffer fully occupied) message]
Dimensions: Customer ID, Device ID, UE ID, MCC, Location, Originating Cell ID, Target IP address, Technology Ty
3G, TD-LTE, FD-LTE….}, Media Type {720P Video, 1080P Video…}
Units: %
Capture Period: 15 minutes
Value: Used to measure media streaming experience
Comments: Equation represents the proportion of total playing time that the playback is stalled. Media Type
may not be known
CO-C-21a # Seconds per Streaming Stall
SUM [Time of TCP media (buffer fully occupied) message - Time of preceding TCP media (buffer empty) message]
/ #TCP media (buffer empty) messages
Dimensions: Customer ID, Device ID, UE ID, MCC, Location, Originating Cell ID, Target IP address, Technology Ty
3G, TD-LTE, FD-LTE….}, Media Type {720P Video, 1080P Video…}
Units: Seconds
Capture Period: 15 minutes
Value: Used to measure media streaming experience
Comments: Equation represents the average stall time per playback stall event. Media Type may not be known

CO-C-22 % Streaming Sessions Disconnected

# Streaming Disconnection/ # TCP Media Data (Initial Buffer Fully Occupied) messages Dimensions: Customer
ID, Device ID, UE ID, MCC, Location, Originating Cell ID, Target IP address, Technology Type {GSM, 3G, TD-LTE, F
Media Type {720P Video, 1080P Video…}
Units: %
Capture Period: 15 minutes
Value: Used to measure media streaming experience

Version 1.5 (2017-04-23) 12

 Comments: Streaming Disconnection indicates the number of times for which a user stops playing a multimedia
file due to poor quality of the sound or image and the number of times for which the file stops unexpectedly due
to network disconnection. Media Type may not be known
CO-C-22a # Kb/s Streaming Download Throughput
SUM [HTTP Reply Data N message content]/SUM [Time of HTTP Data.last message – Time of Associated
TCP_SYN message]
Dimensions: Customer ID, Device ID, UE ID, MCC, Location, Originating Cell ID, Target IP address, Technology Ty
{GSM, 3G, TD-LTE, FD-LTE….}, Media Type {720P Video, 1080P Video…}
Units: Kb/s
Capture Period: 15 minutes
Value: Used to measure media streaming experience
Comments: “associated” means occurring in the same transaction sequence (thread). Media Type may
not be known

CO-F-15 # Seconds Streaming Response Standard Deviation

Standard Deviation of # Seconds Streaming Response
Dimensions: Device Type, Region ID, Technology Type {GSM, 3G, TD-LTE, FD-LTE….}, Media Type
{720P Video, 1080P Video…}
Units: Seconds
Capture Period: 1 month
Value: Used to measure the consistency of media streaming performance
Comments: Media Type may not be known

CO-F-16 Streaming Stall Frequency Standard Deviation

Standard Deviation of Streaming Stall Frequency
Dimensions: Device Type, Region ID, Technology Type {GSM, 3G, TD-LTE, FD-LTE….}, Media Type
{720P Video, 1080P Video…}
Units: Stalls/Hour
Capture Period: 1 month
Value: Used to measure the consistency of media streaming performance
Comments: Media Type may not be known
CO-F-16a # Kb/s Streaming Download Throughput Standard Deviation
Standard Deviation of # Kb/s Streaming Download Throughput
Dimensions: Device Type, Region ID, Technology Type {GSM, 3G, TD-LTE, FD-LTE….}, Media Type
{720P Video, 1080P Video…}
Units: Kb/s
Capture Period: 1 month

Version 1.5 (2017-04-23) 13

 Value: Used to measure the consistency of media streaming experience
Comments: Media Type may not be known

2.2.3 Metrics required.

The following metrics are required by STC to enable the analysis of the Mobile Streaming customer
journey; however, the Capture period has been revised to Near-real-time.

The table below shows system source and proposed capture period for metrics amended from TM
FORUM
Metric System Dimension Capture Period
CO-C-15 % TCP Connect SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Success Cell/BSC/RNC/SGSN/Region/Province/City/G
GSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-C-16 % Get Response SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Success Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-C-17 % Initial Buffer SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Success Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-C-18 % Streaming SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Response Success Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-C-19 # Seconds SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Streaming Response Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-C-20 Streaming Stall SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Frequency Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-C-20a # Streaming SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Stalls per Streaming Cell/BSC/RNC/SGSN/Region/Province/City/
Session GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE

Version 1.5 (2017-04-23) 14

 Metric System Dimension Capture Period
CO-C-21 % Time SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Streaming Stalled Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-C-21a # Seconds SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
per Streaming Stall Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-C-22 % Streaming SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Sessions Disconnected Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-C-22a # Kb/s SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Streaming Download Cell/BSC/RNC/SGSN/Region/Province/City/
Throughput GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-F-15 # Seconds SEQ Customer ID, Device ID, UE ID, MCC, Monthly
Streaming Response Cell/BSC/RNC/SGSN/Region/Province/City/
Standard Deviation GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-F-16 Streaming Stall SEQ Customer ID, Device ID, UE ID, MCC, Monthly
Frequency Standard Cell/BSC/RNC/SGSN/Region/Province/City/
Deviation GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
CO-F-16a # Kb/s SEQ Customer ID, Device ID, UE ID, MCC, Monthly
Streaming Download Cell/BSC/RNC/SGSN/Region/Province/City/
Throughput Standard GGSN, Target IP address, Technology Type
Deviation {GSM, 3G, TD-LTE, FD-LTE

UC13-01 # Seconds TCP SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Uplink RTT Cell/BSC/RNC/SGSN/Region/Province/City/GGS
N, Target IP address, Technology Type {GSM, 3G,
TD-LTE, FD-LTE
UC13-02 # Seconds SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
TCP Downlink RTT Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
UC13-03 % TCP SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Connect Success Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE

Version 1.5 (2017-04-23) 15

UC13-01 # Seconds TCP SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Uplink RTT Cell/BSC/RNC/SGSN/Region/Province/City/GGS
N, Target IP address, Technology Type {GSM, 3G,
TD-LTE, FD-LTE
UC13-04 # TCP SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Connect Delay Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
UC13-05 % TCP SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Uplink Packet Cell/BSC/RNC/SGSN/Region/Province/City/
Retransmission GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
UC13-06 % TCP SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Downlink Packet Cell/BSC/RNC/SGSN/Region/Province/City/
Retransmission GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
UC13-07 # DNS Query SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Delay Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
{GSM, 3G, TD-LTE, FD-LTE
UC13-08 # DNS Query SEQ Customer ID, Device ID, UE ID, MCC, Near Real-time
Success Rate Cell/BSC/RNC/SGSN/Region/Province/City/
GGSN, Target IP address, Technology Type
GSM, 3G, TD-LTE, FD-LTE

Version 1.5 (2017-04-23) 16

 The table below shows metrics required to assist with analysis tasks in this journey
Metric
Device (UE) Information
S1-MME Attach Success Rate
Initial Context Setup Success
Rate
DNS Query Success Rate
DNS Query Delay(MS)
PDP Activation Success Rate
Credit Control Success Rate for
Initial Request (%)
Credit Control Success Rate for
Update Request (%)
Perceived Call Success Rate
Inter SGSN RAU Success rate
Inter MME Data Forwarding
Tunnel Creation Success Rate
2.3 Mobile Streaming system Chart
The technology systems that manage this operation are provided below, along with the sequence of activities
that contribute to this journey being fulfilled. The systems involved being:-
Version 1.5 (2017-04-23)
Calculation Formula Dimension
N/A
N/A
[S1-MME Attach Success
eCGI, eNodeB, MME, region,
Times]/[S1-MME Attach
province/state
Requests]
[Initial Context Setup Success
eCGI, eNodeB, MME, region,
Times]/[Initial Context Setup
province/state
Requests]
[First DNS Query Cell/BSC/RNC/SGSN/Region/Entire
Successes(MS)]/[First DNS Network/Province/GGSN
Query Requests(MS)]
[DNS Query Total Cell/BSC/RNC/SGSN/Region/Entire
Delay(MS)]/[DNS Query Network/Province/GGSN
Successes(MS)]
[PDP Activation Success
Times]/[PDP Activation State, Region, SGSN, RNC, SAI,
Requests] State+APN
[Credit Control Initial Success
Times]/[Credit Control Initial
Request Times] GGSN/PGW, OCS
[Credit Control Update
Success Times]/[Credit
Control Update Request
Times] GGSN/PGW, OCS
[Perceived Successful State, Region, MSC pool, MSC,
Call]/[Call Attempts] BSC/RNC or CGI/SAI
Inter SGSN RAU Success/Inter
SGSN attempts SGSN
Inter MME Data Forwarding
Tunnel Creation successes/
Inter MME Data Forwarding
Tunnel Creation attempts SGW/MME
17
 • SEQ Analyst Platform, data collection by both Active and Passive probes deployed on
the Mobile network. For the Mobile Streaming scenario, the probes are identified
under “PS” marked in Blue and LTE in Yellow whilst the actual mobile data is marked
in Red.
• The RAN and its associated Network.

Version 1.5 (2017-04-23) 18

2.4 Data to deliver the metrics

2.4.1 System Flow

The Mobile Streaming service system flow:
The Streaming service process in a mobile network includes four processes:
 Access process: the signaling establishment process shown in Figure 2-1
This process is the same as that for the Web Browsing service.
 Video main page opening process: the video main page process shown in Figure 2-1
Details of this process are the same as those for the Web Browsing service.
 Initial video streaming buffering: the process from A to C shown in Figure 2-1
This process indicates connection setup between UEs and the service server. Video
download starts, including DNS connection setup (optional), TCP connection setup, and
video request, and video streaming data is downloaded until the buffering is complete.
 Video streaming playing: the process from C to G shown in Figure 2-1
 This process begins when the buffering is complete and ends when the video download
is complete.
Points for measuring KQIs of Video Streaming services are included in the processes
from point (1) to point (7). These KQIs are classified into accessibility and integrity
categories to reflect user experience for video streaming services.

Version 1.5 (2017-04-23) 19

 Figure 2.1: Flowchart of Mobile Streaming services
.

Table 2.1: Measurement items of Mobile Streaming services

Trigger Description of User Protocol or Technical Description

ID Experience

(1) A user presses the play button. Time when a UE sends a DNS Query
message

If a DNS Query message is unavailable, use the

time when the TCP SYN message is sent.
If the TCP SYN message is unavailable, use the
time when the HTTP GET message is sent.

(2) The video responds and data Time when the UE receives the HTTP 200 OK
download starts. message

Version 1.5 (2017-04-23) 20

 Trigger Description of User Protocol or Technical Description
ID Experience

(3) Initial buffering is complete and Time when the received data volume
video playing begins. reaches the threshold and video playing
begins

(4) The video pauses. Time when no data is in the cache area and
therefore the video pauses

(5) The video pause ends. Time when the data buffer size reaches the
threshold and the video playing begins again

(6) All video data is downloaded. Time when the UE receives the last video
data packet with payload

(7) The video playing is complete Time when the TCP link for transmitting
or the user closes the video. video data is released

Before the video playing is complete or the user

closes the video, the server or UE has released all
TCP links because there is no data to download.

2.4.2 Formula.
Metric Calculation Comments
CO-C-15 % TCP Connect # TCP_Ack messages/# A user presses the play
Success TCP_Syn messages button and DNS
querymessage is sent. (1)
CO-C-16 % Get Response # Initial HTTP Reply Data The video responds and
Success messages/ # Initial HTTP Get data download starts. (2)
messages
CO-C-17 % Initial Buffer [% TCP Connect Success]*[% The video responds and
Success GET Response Success]*[% data download starts. (1, 2)
Initial Buffer Success]
CO-C-18 % Streaming
Response Success
CO-C-19 # Seconds SUM [Time of TCP media Initial buffering is complete
Streaming Response data (initial buffer fully and video playing begins. (4)
occupied) message – Time of
associated TCP-SYN
message]/# TCP-SYN
messages
CO-C-20 Streaming Stall #TCP media (buffer empty) The video pauses. (5)
Frequency messages/ Sum [Time of TCP
media (buffer empty) or
RST/FIN message – Time of
preceding TCP media (buffer

Version 1.5 (2017-04-23) 21

 fully occupied) message]
CO-C-20a # Streaming
Stalls per Streaming Session

CO-C-21 % Time Streaming SUM [Time of TCP media The video pauses. (5)
Stalled (buffer fully occupied)
message - Time of preceding
TCP media (buffer empty)
message]/ SUM [Time of
RST/FIN message – Time of
TCP media data (initial
buffer fully occupied)
message]
CO-C-21a # Seconds per The video pauses. (5)
Streaming Stall
CO-C-22 % Streaming The video ends or the user
Sessions Disconnected aborts.

CO-C-22a # Kb/s Streaming

Download Throughput
CO-F-15 # Seconds Streaming
Response Standard Deviation
CO-F-16 Streaming Stall
Frequency Standard
Deviation
CO-F-16a # Kb/s Streaming
Download Throughput
Standard Deviation
UC13-01 # Seconds TCP
Uplink RTT
UC13-02 # Seconds TCP
Downlink RTT
UC13-03 % TCP
Connect Success
UC13-04 # TCP Connect
Delay
UC13-05 % TCP Uplink
Packet Retransmission
UC13-06 % TCP
Downlink Packet
Retransmission
UC13-07 # DNS Query
Delay
UC13-08 # DNS Query
Success Rate

Version 1.5 (2017-04-23) 22

2.4.3 System example data

Attached is the sample data received for this journey and other Mobile journey based on SEQ probes.

CT Data
Source.docx

Version 1.5 (2017-04-23) 23