[ ]

ITU-TQSDG - DUBAI
ASCOM PRESENTATION:
QOS AND QOE IN LTE MOBILE NETWORKS

Dr. Irina Cotanis

ITU-
T QSDG, Dubai,
November 2014
Who We Are
Independent Auditor

+40
11
patents
worldwide
Next-generation
technology
650 Customers Globally

More than
30 patents ITU-T
ITU-T
worldwide ETSI
ETSI
Innovative VQEG
VQEG
Leader Wireless
network Formative Player
performance

Dr. Irina Cotanis ITU-T QSDG, Dubai,

November 2014
AGENDA

LTE QoE- QoS- KPI MAPPING RELATIONSHIPS

CHALLENGES EMERGING FROM LTE
ASCOM LTE QoS/QoE TESTING APPROACHES
SHARING EXPERIENCE: SOME USE CASES
CONCLUSIONS

Dr. Irina Cotanis

ITU-
T QSDG, Dubai,
November 2014
[ ]

LTE QOE-QOS KPI MAPPING RELATIONSHIPS

Dr. Irina Cotanis

ITU-
T QSDG, Dubai,
November 2014
ASCOMs APPROACH ON QoS/QoE CYCLE => SAME AS ITUs
(ITU-T G.1000)

Subscriber MNO
Alignment
QoE Requirements of QoS Targeted by MNO
gap
Subscriber (Target SLA)

Execution
Value gap

QoE/QoS KPIs,

gap
QoS TEMS
Cycle

QoS Delivered by MNO

QoE Perceived by Subscriber Perception gap
(Achieved SLA)

QoE
Subscriber-centric Network-centric
Quality (QoE) Quality (QoS)
TEMS
Dr. Irina Cotanis ITU-T QSDG, Dubai,
November 2014
LTE QoE- QoS - KPIs MAPPING
Network
Customer Performance
Experience

QoS KPIs
QoE Upper Layers
(IP/UDP, IP/TCP)
(throughput, delay
Application Physical layer
Layer
Layer 2&3 MAC, (coverage,
(accessibility, interference)
retainability, RLC, LLC (reports
related to QoE
integrity) dimensions = PDP
context, HO info,
codecs)
Dr. Irina Cotanis ITU-T QSDG, Dubai,
November 2014
COST EFFICIENT TOP DOWN APPROACH ENABLED BY PRE-
DEFINED LTE QoE-QoS-KPIs MAPPINGS
Increased Top-down customer
operational efficiency experience centric approach

Application Layer
Voice Service: 97
(accessibility,
Auryst, 03 PESQ, 12 retainability, integrity)
POLQA
Video-Audio Services:
08 VSQI/MTQI, 12 Upper Layers (IP/User Data
PEVQ, 12 VQmon Protocol , IP/Transport Control
Protocol): Throughput, Delay,
Packet Loss, Rebuffering

Layers 2 & 3 messaging reports

related to QoE dimensions like
PDP context, HO info, codec
usage

Physical layer details like

coverage and interference

Reduce troubleshooting time by using QoE centric mapping to QoS and KPIs per service type
Dr. Irina Cotanis ITU-T QSDG, Dubai,
November 2014 7
[ ]

CHALLENGES EMERGING FROM LTE

Dr. Irina Cotanis

ITU-
T QSDG, Dubai,
November 2014
VoLTE-ViLTE-RCS EXPERIENCE

Call Quality as Perceived by Subscribers

MOS / ITU-T P.863; J.247 & P.120x.x based (video)
On device VoLTE client: re-buffering/time scaling for
voice; error concealment type for video
Voice path delay (mouth to ear), echo, video voice
lip sync
RTP Packet loss, latency, jitter
HOIT (LTE HO, eSRVCC)
Throughput
Voice /video codec type and bit rates

Call control performance

Session set-up: SIP signaling statistics / IMS Registration, RTT
Session Accessibility, Retainability
QCI allocation verification
LTE RRC connection and HO statistics
Dr. Irina Cotanis ITU-T QSDG, Dubai, November 2014
MOBILE VIDEO STREAMING EXPERIENCE

Codec types: Devices & Clients

High Different form factors
Low Different 2D/3D displays
Adaptive bit rates Different adaptive error concealment
schemes

Network centric:
Service-Centric
Packet loss
Jitter Wide variety of content & bit rates
Allocated GBR and QCI
HTTP/TCP vs. RTP/RTSP

Dr. Irina Cotanis

ITU-
T QSDG, Dubai,
November 2014
VoLTE QoE AND ITS ROOT CAUSES (QoS, KPIs SOURCES)

Interruptions Perceived call

Perceived Mouth to ear Noisiness (QoE)
(QoE) session
frequency delay (QoE) (incl. musical
(incl. time performance
Spectrum (QoE) noise)
clipping) (QoE)
Network (QoS): Network Network (QoS) Network (QoS):
IMS network
Limited (QoS/KPIs) (IMS path can be Limited BWD with
(QoS/KPIs):
Bandwidth (IP/IMS loss, jitter, key contributor) noisy speech
RAN erroneous bits, SIP statistics,
(speech
RAN HO) IMS registration
contamination)
Device (KPIs): Device based Codec/device
Spectral signal process- signal Device (KPIs):
shaping ing (KPIs) (e.g. processing Imperfect NR
Reverberations NR, EC) (KPIs) (musical noise)
Loudness
Codec/client during silent
(KPIs):PL ITU-T on going work:
concealment periods G.VoLTE,
schemes Non-optimal P.TCA (Technical
loudness levels Cause Analysis)
Aggressive
VAD schemes Codec/client
(KPIs)
Dr. Irina Cotanis
ITU-
PLC interpolation
T QSDG, Dubai, based (additive
November 2014 artifacts)
 MOBILE VIDEO STREAMING QoE AND ITS ROOT CAUSES (QoS, KPIs
SOURCES)

Low throughput (limited bandwidth) QoS

Packet loss, discard, late arrival

IP delays / jitter

KPIs Video resolutions/re-scaling Possible transcoding KPIs

Encoding/transcoding rates
Compression schemes Limited coverage, interference

Bandwidth estimation
algorithms impact the bit rate
selection
Visual impairments: QoE
Content server load,
(competing video streams) Blockiness,
Bluryness,
Content complexity KPIs Client under/over flow; Jerkiness,
(variable movement) improper buffer Freezing with and/or without
lengths/adaptation length skipping

Initial buffering settings Perceived service accessibility

/ access time
Display resolutions, form
factors
Dr. Irina Cotanis ITU-T QSDG, Dubai,
November 2014
[ ]

ASCOM LTE QOE/QOE TESTING APPROACHES

Dr. Irina Cotanis

ITU-
T QSDG, Dubai,
November 2014
UNDERSTAND AND USE ITU-T BASED QoE METRICS
BEST FITTED TO THE SERVICE BEING TESTED
LTE VOICE AND MOBILE VIDEO SERVICES

LTE Broadcast Streaming server

Video stream
(YouTube, eMBMS) Encoding
Encoding
Cellular Voice &
IP
IP recordings
recordings (RTP
(RTP Data Network
KPIs/QoS) synchronized
KPIs/QoS) synchronized Decoding
Decoding
with
with RAN
RAN KPIs
KPIs Conversational
Video
Solutions
Solutions IP
Voice:
Voice: MOS
MOS P.OLQA
P.OLQA Voice
transport/payload
Video
Video:
Video: MOS
MOS PEVQ
PEVQ Voice/Conversational Video
stream parameters

Full
Full Reference
Reference listening
listening media
media quality
quality evaluation
evaluation
(intrusive, perceptual) measurement
(intrusive, perceptual) measurement
Direct
Direct RTP
RTP KPIs/QoS
KPIs/QoS
MOS,
MOS, media
media based
based KPIs
KPIs
Solutions
Solutions
Voice:
Voice: MOS
MOS -- ITU
ITU P.564
P.564 based
based
Video-audio: MOS P.120x.x
Video-audio: MOS P.120x.x Non
Non Reference
Reference listening
listening media
media quality
quality evaluation
evaluation
based
based (VQmon)
(VQmon) (non-intrusive, parametric)
(non-intrusive, parametric)
Dr. Irina Cotanis
ITU-
T QSDG, Dubai, MOS,
MOS, network
network based
based KPIs
KPIs
November 2014
[ ]

SHARING EXPERIENCE: SOME USE CASES

Dr. Irina Cotanis

ITU-
T QSDG, Dubai,
November 2014
VoLTE MOS SCORE and VoLTE KPIs/QoS

QoE Evaluation &

TEMS Discovery Troubleshooting

:K PIs
sure
Mea

TEMS Investigation
Deliver VoLTE
TEMS Pocket Stack
Optimization
POLQA;
POLQA; speech
speech path
path delay,
delay, volume,
volume, echo
echo
Client
Client Information
Information (re-buffering,
(re-buffering, codec)
codec)
ESM configuration (QCI, RoHC)
Real time IP trace & L3 logging
IP recording (RTP stats), SIP stats

RAN (HOIT, Scheduling, RSRP, CINR,

CQI, PMI/RI, UE category, MTU Size,
Protocol stack configuration)
Dr. Irina Cotanis
ITU-
T QSDG, Dubai,
November 2014
ADAPTIVE STREAMING QoE AND QoS/KPIs

Dynamically
adaptive
HTTP/TCP

ETSI TS 102.250-2,
Dr. Irina Cotanis ITU-T QSDG, Dubai,
November 2014 ETSI TR 101.578 Measurement Guidance for TCP based video
services
[ ]

CONCLUSIONS

Dr. Irina Cotanis

ITU-
T QSDG, Dubai,
November 2014
A FULL QoE/QOS CYCLE SOLUTION FOR LTE NETWORK AND
SERVICES

My Customer
Experience Automated data
Data correlation
and presentation

My competitors
customers
(Benchmark)

Providing network and

My customers, customer experience
real field agents -centric diagnoses for voice
(VoLTE, OTT, VoHSPA),
and video-audio services
(OTT, RCS-e/Joyn)
Built-in best practices
scripted data analytics
provide automated root
cause analysis

Automated
what, why
and engineer-
trusted how
scenarios

Dr. Irina Cotanis ITU-T QSDG, Dubai,

November 2014
CONCLUSIONS

PLEASE VISIT OUR WEBSITE:

http://
www.ascom.com/nt/en/index-nt/about-us-network-testing/nt-about-us-resources.htm/
White papers: VoLTE, Video Streaming, HetNets, Carrieir Aggregation
and...watch the space: LTE Broadcast testing to come soon

Informa Webinar: Advanced testing with Ascom in LTE networks

Webinars: VoLTE, Video Streaming

Dr. Irina Cotanis ITU-T QSDG, Dubai,

November 2014
[ ]

THANK YOU

Dr. Irina Cotanis

ITU-
T QSDG, Dubai,
November 2014