Professional Documents
Culture Documents
Amr Efr Mos
Amr Efr Mos
Monitoring the End User Quality of Experience using the derivative Mean Opinion Score (MOS) KPI
Objectives:
Indicate the importance of MOS to the end user perceived speech quality. Introduce the MOS derivative Key Performance Indicator (KPI) which comes from Drive Tests Data and Network Statistics. Implement a mathematical formalism in order systemize the MOS calculation procedure. Compare different speech codecs implementations in BSC areas using MOS distribution patterns.
Effect of transmission performance on Multimedia Quality of Service 17 - 19 June 2008 - Prague, Czech Republic
Contents:
MOS concept & dependencies. Average MOS calculation in Cell & BSC level combining drive tests data & OSS measurements. Cell MOS distribution patterns for different speech codecs. EFR vs AMR codec comparison. Effect of MS technology on different speech codecs share. Evaluating speech codec effectiveness using quality and speech codec distribution patterns
Effect of transmission performance on Multimedia Quality of Service 17 - 19 June 2008 - Prague, Czech Republic
Mean Opinion Score (MOS) concept description: A common benchmark used to determine sound quality. In voice telephony, represents a measure of human speech quality, at the destination end of the circuit. It is an important factor in determining the QoS. MOS Measurement: 1. A pool of listeners rate a series of audio files using a five-grade scale ranging from 1 to 5. 2. The average or Mean Opinion Score (MOS) for each audio file is calculated. Conditions: 16 or more listeners (adequate sample). Quiet environment.
To reduce variability, tests commonly include reference files that have "industry accepted" MOS scores.
Speech Codec
Full Rate (FR) Half Rate (HR) Enhanced Full Rate (EFR) Adaptive Multi Rate (AMR)
Poor
MOS concept & dependencies. Average MOS calculation in Cell & BSC level combining drive tests data & OSS measurements. Cell MOS distribution patterns for different speech codecs. EFR vs AMR codec comparison. Effect of MS technology on different speech codecs share. Evaluating speech codec effectiveness using quality and speech codec distribution patterns
Effect of transmission performance on Multimedia Quality of Service 17 - 19 June 2008 - Prague, Czech Republic
Process results & create MOS Quality & Speech Codec Matrix
Export Speech Codec usage Matrix Export Quality class statistics Matrix
Data Merge
Perform Matrix Operations to calculate Cell/BSC MOS Effect of transmission performance on Multimedia Quality of Service value
OSS Derived Tables Quality Class Share of BSC Speech Share of BSC Table
Speech Codec Share AMR FR AMR HR EFR FR HR
DL Quality QUAL_0 QUAL_1 QUAL_2 QUAL_3 QUAL_4 QUAL_5 QUAL_6 QUAL_7 Samples Share
Dimension: 5X8
Samples Share
Speech Codec Weighted MOS Value 3.55 AMR FR MOS AMR HR MOS 3.24 EFR MOS 3.59 FR MOS 3.08 HR MOS 2.82
Dimension: 5X1
Dimension: 8X1
BSC MOS =
Speech Codec Weighted MOS Value AMR FR MOS 3.55 AMR HR MOS 3.24 EFR MOS 3.59 FR MOS 3.08 HR MOS 2.82
HR
= 3.43
Dimension: 1X5
Dimension: 5X1
(1)
The product of (1), (m x 1) is further multiplied with the (1 x m) speech codecs matrix to obtain a single average MOS value, as in (2):
SC 0 ... ... [C 0 ... SC m
(2)
Key issues:
Process can be easily implemented in a performance monitoring system. Can be applied in Cell, BSC or any other network structure. Cells with Poor or Bad MOS can be isolated and be further investigated. A single average MOS value of a cell group (i.e. BSC area) may not be essential. The comparison of different network areas provides QoS indicative results.
MOS concept & dependencies. Average MOS calculation in Cell & BSC level combining drive tests data & OSS measurements. Cell MOS distribution patterns for different speech codecs. EFR vs AMR codec comparison. Effect of MS technology on different speech codecs share. Evaluating speech codec effectiveness using quality and speech codec distribution patterns
Effect of transmission performance on Multimedia Quality of Service 17 - 19 June 2008 - Prague, Czech Republic
MOS distribution:
Derives from cell level MOS calculation of a group of cells Gives a brief description of the end user perceived speech quality in a specific network area.
70
0 1.5 Bad Poor 2.1 2.7 Fair 3.5 Good Excellent 5
60 Number of cells 50 40 30 20 10 0
58 74 06 5 3 46 98 22
62
66
14
38
78
82
54
2,
2,
3,
2,
2,
3,
3,
2,
2,
2,
3,
3,
3,
3,
3,
3,
3,
3,
86
MOS classification
Effect of transmission performance on Multimedia Quality of Service 17 - 19 June 2008 - Prague, Czech Republic
3,
94
40 30 20 10 0
2, 5
2, 58
2, 66
2, 74
2, 82
2, 9
2, 98
3, 06
3, 14
3, 22
3, 3
3, 38
3, 46
3, 54
3, 62
3, 94
3, 7
3, 78
3, 86
MOS classification
Effect of transmission performance on Multimedia Quality of Service 17 - 19 June 2008 - Prague, Czech Republic
>4
Poor
MOS classification
Effect of transmission performance on Multimedia Quality of Service 17 - 19 June 2008 - Prague, Czech Republic
MOS concept & dependencies. Average MOS calculation in Cell & BSC level combining drive tests data & OSS measurements. Cell MOS distribution patterns for different speech codecs. EFR vs AMR codec comparison. Effect of MS technology on different speech codecs share. Evaluating speech codec effectiveness using quality and speech codec distribution patterns
Effect of transmission performance on Multimedia Quality of Service 17 - 19 June 2008 - Prague, Czech Republic
Promotion of latest technology MSs to existing subscribers could be a key issue of marketing strategy in order to optimize speech quality.
Effect of transmission performance on Multimedia Quality of Service 17 - 19 June 2008 - Prague, Czech Republic
MOS concept & dependencies. Average MOS calculation in Cell & BSC level combining drive tests data & OSS measurements. Cell MOS distribution patterns for different speech codecs. EFR vs AMR codec comparison. Effect of MS technology on different speech codecs share. Evaluating speech codec effectiveness using quality and speech codec distribution patterns
35%
25%
20%
15%
10%
5%
To achieve this aim drive tests data should be post-processed differently with regard to the original data.
Effect of transmission performance on Multimedia Quality of Service 17 - 19 June 2008 - Prague, Czech Republic
80%
Samples Share
70% 60% 50% 40% 30% 20% 10% 0% Excellent Good Fair Poor Bad
Samples percentage
MOS Class
100%
R/EFR/FR/HR
90% 80%
EFR/FR/HR
FR/HR
Samples percentage
70% 60% 50% 40% 30% 20% 10% 0% Excellent Good Fair Poor Bad
MOS Class