Fourth International Conference on Networked Computing and Advanced Information Management

The QoE Evaluation Method through the QoS-QoE Correlation Model

Hyun-Jong Kim1, Dong-Hyeon Lee1, Jong-Min Lee1, Kyoung-Hee Lee2,

Won Lyu2, Seong-Gon Choi1
1
School of Electrical & Computer Engineering, Chungbuk National University (CBNU),
12 Gaeshin-dong, Heungduk-gu, Cheongju-city, Chungbuk, 361 – 763, Republic of Korea
2
Electronics and Telecommunications Research Institute (ETRI),
161 Gajeong-dong, Yuseong-gu, Daejeon, 305-350, Republic of Korea
1
{hjkim78, dhlee, ljm80, sgchoi}@cbnu.ac.kr, 2{leekhe, wlyu}@etri.re.kr

Abstract of the quality of the service.

Currently, the satisfaction of user is becoming one
of the most important topics concerned by the service
providers. So we propose the approach method for the
objective QoE measurement through the QoS
parameters. The measurements and provisioning of the
quality of service (QoS) are generally defined in terms
of network delivery capacity and resource availability,
not in terms of satisfaction to the end-user. The
fundamental assumption behind such traditional
provisioning is that the measured quality of service is
closely related to the quality of experience (QoE) for
the end-user. In the paper, we describe the QoS and
QoE correlation model, and the QoE evaluation
method using QoS parameter in the converged network
environment is studied.
1. Introduction
The increasing deployment of broadband IP
networks has enabled integration and convergence of
voice service (e.g. VoIP), video service (e.g. VoD) and
data service, which is referred as a Triply Play. Like
this, as various services were provided in the integrated
network environment, QoS and QoE concept was
introduced in the IP network.
Currently, the satisfaction of users is becoming one
of the most import topics concerned by the service and
network providers. However, existing methods used for
evaluation the quality of experience (QoE) mostly rely
on user survey and scores from the user, which are too
subjective and need mush processing time and cost. So,
our investigation is trying to relate the objective
network service conditions with the human perception
Moreover, because existing QoE measurement
schemes measure QoE by using a part QoS
measurement parameters, they are difficult to reflect of
the various services. The quality measurement scheme
that uses the preexistence QoS measurement parameter
can provide the objective quality information in a
network-level. However, this quality measurement
scheme is difficult to expect the quality which the
service end-user experiences. So, in this paper, by
using the quality of service information measured in a
network-level, we explain the proposed QoS-QoE
correlation model for the objective QoE, and describe
the example of the service QoE evaluation.
The paper proposed a method to evaluate the QoE
of user in convergence network. The remainder of the
paper is organized as follows. Section II outlines the
current status of QoE evaluation method and related
works. Section III introduces the QoS parameters
related with QoE evaluation and the service classes.
Section IV describes the QoS-QoE correlation model
for the QoE evaluation and section V gives the
example of the QoE evaluation by using this
correlation model. Finally, section VI presents the
conclusion.
2. Related works
2.1. The voice and video service quality
measurement method
Existing quality assessment technologies can be
classified into two categories: subjective quality
assessment schemes and objective ones. Current

978-0-7695-3322-3/08 $25.00 © 2008 IEEE 719

DOI 10.1109/NCM.2008.202

Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on July 15,2010 at 23:14:43 UTC from IEEE Xplore. Restrictions apply.
 research status of these two kinds of evaluation
methods are illustrated as followed paragraphs:
Currently, the evaluation methods for the speech
service are mature. For subjective evaluation methods,
opinion rating (MOS) based on customer’s satisfaction
has been studied to assess the perceptual QoS. It is
specified in ITU-T recommendations E.800 initially [1].
On another hand, several objective quality assessed
methods has been proposed in ITU-T, such as P.861[2]
PSQM (Perceptual Speech Quality Measure), P.862[3]
PESQ (Perceptual Evaluation of Speech Quality) and
G.107 E-Model[4].
For the video service evaluation, subjective video
quality evaluation method is the most reliable video
quality measurement method. A group of viewers is
selected and gathered in a room, the measurement
environment is specified in the ITU-T Recommend-
ation P.910[5]. For the research of objective video
quality method, some estimation software has been
developed which can analyze the video signals and
produce the quality evaluation results. One traditional
objective video quality measurement, Peak Signal to
Noise Ratio (PSNR), has been widely used in many
applications to assess video quality.
PSNR does not take the visual masking phenomenon
into consideration. In other words, every single pixel
error contributes to the decrease of the PSNR, even if
this error is not perceived. So, MPQM (Moving
Pictures Quality Metric) was proposed for the objective
the video quality measurement[6][10]. MPQM is an
objective quality metric for moving picture which
incorporates human vision characteristics. MPQM
represents the typical image quality assessment models
based on the error sensitivity. The widely adopted
assumption of these models is that the loss of
perceptual quality is directly related to the visibility of
the error signal.
From current research status of evaluation method,
we can see that the subjective method based on user
survey can reflect the experience of user more directly
and match well to the feeling of user. However, this
kind of method has several problems, such as, it
required special environment and equipments, needs a
mass of people to participate the test. In conclusion,
subjective video quality measurement cannot provide
real-time and in-service quality monitoring for real-
time video applications. So the application of the
method is limited.
720
Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on July 15,2010 at 23:14:43 UTC from IEEE Xplore. Restrictions apply.
2.2. The study of relationship between QoS
and QoE
Currently, the investigation of QoS and QoE
correlation is continued. Khirman and Henriksen were
trying to relate the objective network service conditions
with the human perception of the quality of the service.
Their subject has been widely investigated for voice
delivery and it is widely acknowledged that the
relationship between voice transmission conditions and
the human perception of quality is far from linear[7].
They discuss in detail how the human satisfaction of
HTTP service is affected by the two main network QoS
parameters, namely network delivery speed and latency.
However, it is difficult to represent the feature of the
provided and various services from only the bandwidth
and latency time in the integrated network environment.
In [8] the authors thought that pervasive computing
environment brings the method of evidence context
related to QoE. They studied the QoE evaluation
method in pervasive computing environment, and
proposed the enhanced QoE evaluation parameter
model. In [8] rough-set based algorithm is proposed to
reduce context attributes and determine the weight of
each attribute, the algorithm has been validated on
video streaming service, and the architecture of QoE
evaluation system is described. As a mass of evidence
information related to the experience of users can be
gathered through the context-awareness computing, the
calculation results of QoE evaluation method can
highly match the real feeling of users. However, the
method needs to be enhanced along with the
development of pervasive computing.
3. The QoS parameters related with end-
user
This clause addresses the QoS quality parameters
which can be considered for the QoE evaluation.
Transfer Capacity is a fundamental QoS parameter
having primary influence on the performance perceived
by end-users. Many user applications have minimum
capacity requirements; these requirements should be
considered when entering into service agreements. And
lost bits or octets can be subtracted from the total sent
in order to provisionally determine network capacity.
An independent definition of capacity is for further
study.
It is assumed that the user and network provider
have agreed on the maximum access capacity that will
 be available to one or more packet flows in a specific 3.1. Delay
QoS class (except the Unspecified class in Table Ⅰ). A
Delay manifests itself in a number of ways,
packet flow is the traffic associated with a given
including the time taken to establish a particular service
connection or connectionless stream having the same
from the initial user request and the time to receive
source host (SRC), destination host (DST), class of specific information once the service is established.
service, and session identification. Other documents Delay has a very direct impact on user satisfaction
may use the terms microflow or subflow when referring depending on the application, and includes delays in
to traffic streams with this degree of classification. the terminal, network, and any servers. Note that from a
Initially, the agreeing parties may use whatever user point of view, delay also takes into account the
capacity specifications they consider appropriate, so effect of other network parameters such as throughput.
long as they allow both network provider enforcement
and user verification. For example, specifying the peak 3.2. Delay variation
bit rate on an access link (including lower layer
overhead) may be sufficient. The network provider Delay variation is generally included as a
agrees to transfer packets at the specified capacity in performance parameter since it is very important at the
accordance with the agreed QoS class. transport layer in packetized data systems due to the
The network performance objectives may no longer inherent variability in arrival times of individual
be applicable when there is packets submitted n excess packets. However, services that are highly intolerant of
of the capacity agreement or the negotiated traffic delay variation will usually take steps to remove (or at
contract. If excess packets are observed, the network is least significantly reduce) the delay variation by means
allowed to discard a number of packets equal to the of buffering, effectively eliminating delay variation as
number of excess packets. Such discarded packets must perceived at the user level (although at the expense of
not be included in the population of interest, which is adding additional fixed delay).
the set of packets evaluated using the network
performance parameters. In particular, discarded 3.3. Information loss
packets must not be counted as lost packets in assessing
Information loss has a very direct effect on the
the network's IPLR performance. A discarded packet
quality of the information finally presented to the user,
might be retransmitted, but then it must be considered
whether it be voice, image, video or data. In this
as a new packet in assessing network performance. context, information loss is not limited to the effects of
Each network QoS class creates a specific bit errors or packet loss during transmission, but also
combination of bounds on the performance values. This includes the effects of any degradation introduced by
clause includes guidance as to when each network QoS media coding for more efficient transmission (e.g. the
class might be used, but it does not mandate the use of use of low bit-rate speech codecs for voice).
any particular network QoS class in any particular
context. Table 2. Guidance for IP QoS classes [12]
Table 1. IP network QoS class definitions and QoS
network performance objectives [12] Applications (ex) Network techniques
class
Real-time, jitter sensitive,
Network QoS Classes Constrained routing and
0 high interaction (VoIP,
performance Class Class Class Class Class Class distance
VTC)
Parameter 0 1 2 3 4 5 Real-time, jitter sensitive, Less constrained routing
100 400 100 400 1
IPTD 1s U interactive (VoIP, VTC) and distances
ms ms ms ms Transaction data, Highly Constrained routing and
IPDV 50ms 50ms U U U U 2
interactive (Signalling) distance
1x 1x 1x 1x 1x Transaction data, Less constrained routing
IPLR U 3
10-3 10-3 10-3 10-3 10-3 interactive and distances
-4
IPER 1x 10 U Low loss only (short
4 transactions, bulk data, Any route/path
video streaming)
Traditional applications of
5 Any route/path
default IP networks

721

Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on July 15,2010 at 23:14:43 UTC from IEEE Xplore. Restrictions apply.

4. The proposed QoE measurement scheme
using the QoS parameters
The provided QoS-QoE correlation of the service
can variously show up in the integrated network
environment by QoS quality parameters considered for
a kind of service and QoE. We propose the correlation
model approach method that can reflect all elements
influencing on QoE in order to solve these problem.
The relation between the QoE and QoS cannot be
easily mapped to the service which can be provided in
the converged network. According to the traffic
characteristic of each service, the QoS level which is
required in order to satisfy the QoE class is different.
Moreover, the QoE influence element at the terminal
layer which is required in order to evaluate QoE need
to reflect. The normalized QoS score of the service
which can be provided can be obtained through the
next equation from the integrated network.
(1)
By using quality indexes of a network-level, the QoS
score can obtain. It becomes with application so that
there are delay, jitter, loss rate, error rate, bandwidth,
the signal success rate, and etc, that is the objective
quality parameter, and the weighted value of these
quality parameters be different from the used service.
For example, the service sensitive to delay time
assigns the weighted value 10 to the delay time 100ms
like VoIP and video conference service and the service
which is a bit less sensitive to the delay time like VoD
assigns the weighted value 5. In the same condition, the
bandwidth proffer of over required is meaningless if
VoIP and video conference service are satisfied by the
minimum guaranteed bandwidth. In another hand, the
case where the user experience quality increases as the
bandwidth provided like VoD is big will be able to
assign the large-scale weighted value.
(2)
Here, the used variable and constant are determined
by elements influencing on QoE including an
environment and the using service kind, terminal
position, using codec, and etc. The QoS of this
equation is the normalized QoS score which is
722
Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on July 15,2010 at 23:14:43 UTC from IEEE Xplore. Restrictions apply.
calculated by the equation (1). The QoE class measured
by the QoS quality parameter of a network-level is
mapped like the existing MOS grade with 5 classes.
The α represents the QoS quality class of the network-
level which has to be provided as the QoS class
constant in order to satisfy the required QoE class. The
minimum range of the QoE quality standard of the
provided service is determined by this value.
Next, β is determined according to the class of
service as the grade of service constant. Finally, K is
the Scale constant for showing the satisfaction about
the use service. This generalizes the subjective quality
about the using service. Although the same QoS quality
is provided in the wire-network environment, this
variable considers elements influencing on the
user experience quality according to the radio channel
and terminal. According to the class of service, the next
paragraph shows the QoE measurement scheme that
uses QoE and QoS correlation model.
4.1. The Guaranteed service
The services provided in the integrated network
environment can classify as the Guaranteed service, the
Premium service, and the Best Effort service, and the
Guaranteed service has the top priority. The data
priority processing and bandwidth guarantee, and etc.
are required according to the service priority in order to
provide the real-time multimedia service like VoIP,
IPTV and Video Conference, and etc.
QoE
5
Guaranteed Service
4
3
2
1
10 20 30 40 50 60 70
QoS
Figure 1. The QoS-QoE correlation model of the
Guaranteed service
The most of guaranteed services is the real-time
voice / video service like the VoIP, Video Phone and
Video Conference service. The stable bandwidth has to
be provided for the seamless usage of this service and
the minimum connection delay time has to be
guaranteed. The quality parameter needed for the
 quality measure of this service is delay, jitter and
packet Loss rate, and etc. And the call connection yield,
and the use of service duration and audio / image MOS
value are needed when the quality measure of this
service considers QoE.
The QoS-QoE correlation model of the Guaranteed
service can represent the on / off model. If the
QoS score which is required in order to guarantee the
QoE of this service is unable to be satisfied, the
Guaranteed service cannot be provided. As shown in
Figure 1, the QoS score has to be provided over the
minimum 50 in order to guarantee the QoE of the
Guaranteed service. As to this model, the β-value is
small compared with the other model and this means
that the minimum guaranteed class of the QoS which
has to be supported is very sensitive.
Here, the shown numerical value is one example and
actually evaluates the quality customer satisfaction
measurement about the service user and need to
calculate the average distribution of QoE in order to
establish the exact relation.
4.2. The Premium service
In the case of the premium service, it is more flexible
than the Guaranteed service about the real-timeness.
The target service kinds are the Movies streaming
service, IPTV, VoD and Interactive Gaming, and etc.
As to the traffic characteristic of these services, the
minimum bandwidth has to be guaranteed as the real-
time streaming service and the multicasting service.
The major quality parameters are similar to the case of
the Guaranteed service and are however a feature to be
less sensitive to a delay than the Guaranteed service.
QoE
5
4 the grade of service. Moreover, QoS and QoE relation
Premium Service A
3 Premium Service B
Premium Service C
2
1 various network-levels in the integrated network
10 20 30 40 50 60 70
QoS
Figure 2. The QoS-QoE correlation model of the
Premium service
The QoS-QoE correlation model of the premium
service has the feature that is a linear, more than the
723
Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on July 15,2010 at 23:14:43 UTC from IEEE Xplore. Restrictions apply.
Guaranteed service and this can be controlled by the
grade of service constant (B). The premium service can
divide the quality of service according to this linearity.
This accommodates the various class of the AF PHB of
DiffServ. According to select the appropriate QoE
class constant, a network and service providers can
organize the various premium service QoS-QoE
correlation models.
4.3. The Best Effort service
The best-effort service can include preexistence
internet services and the quality level is similar to it.
This says to be the more sensitive QoS-QoE correlation
model the QoS parameters than QoE. This means that
although the QoS parameter performance is improved,
there is a limit in the QoE grade improvement which
the end user experiences.
QoE
5
4
3
2 Best Effort Service
1
10 20 30 40 50 60 70
-1 QoS
Figure 3. The QoS-QoE correlation model of the
Best Effort service
In this way, we propose the QoS-QoE correlation
model according to three service classes. The network
performance (QoS class) which has to be guaranteed at
least can be expected through this model according to
of the provided service can be analyzed through this
model in the integrated network environment.
In the future, the research for setting up the
quantified QoS class toward QoS parameter of the
environment is needed. Moreover, the research about
which how will set up the value of α, β and K in
consideration of the real time multimedia service kind
and service usage environment is required.
 Quality VSWR(Voltage Standing
5. The example of the QoE measurement Indicator Wave Ratio),
using the QoS-QoE correlation model (CQI) CINR (Carrier to
Interference plus Noise
The process of measuring the QoE of the Premium Ratio)
service by using the QoS-QoE correlation model is as
follows. The Table 1 shows the weighted value of the
each QoS parameters needed to the QoE evaluation of The QoS normalized by the equation (1) is
the Premium service. substituted for the equation (2), and then the QoE class
If quality data measured about the video conference can be evaluated. As shown in Figure 4, we can predict
service of the Premium service are as follows, we can e the class of Premium service B for
xpect the QoE of a user through the QoS-QoE correlati the experience quality of an end-user in the
on model. above environment. We can confirm whether it is
satisfied the class of service which the first user joins
from this measurement result or not.
• Delay (D) = 125ms Because of being the result that the QoE is
• Jitter (J) = 45ms evaluated by using the objective network quality
• Packet loss rate (L) = 2 x 10-5 parameters, it can be regarded as the more objective
QoE than the other measurement schemes.
• Packet error rate (E) = 1 x 10-6
Here, the used in order to digitize the QoE class
• Bandwidth (B) = 2.5Mbps network quality index and the standards aren't perfect,
• Call success rate(S) = 99.0% but are one example. The investigation of the weighted
• QoS = F(D, J, L, E, B, S) = 8+10+7+10+10+8=53 value about the network performance index has to be
separately executed. The proper quality parameter
Table 3. QoS parameter and range related with score need to be selected against service users through
the question survey.
QoE

QoS parameter and range QoE

Note 5
Parameter Range Score
70ms ~ 100ms 10 4
Y.1541,
Delay (D) 100ms ~ 150ms 8
G.1010
150ms ~ 200ms 6 3
Premium Service B
30ms ~ 50ms 10
Jitter (J) 50ms ~ 60ms 5 Y.1541 2
60ms ~ 70ms 3
1
~ 10-5 10 Y.1541
Packet loss
10-5 ~ 10-4 7 DSL-
rate (L)
10-4 ~ 10-3 3 Forum 10 20 30 40 50 60 70
~ 10-6 10 Y.1541 QoS
Packet error
10-6 ~ 10-5 7 DSL-
rate (E)
10-5 ~ 10-4 4 Forum
Figure 4. The example of the QoE evaluation using
256 ~ 128Kbps 10
QoS-QoE correlation model
Voice 128 ~ 80Kbps 8 G.1010
Bandwidth 80 ~ 64Kbps 6 DSL-
(B) 3 ~ 2Mbps 10 Forum
Video 2 ~ 1Mbps 8 WiMAX 6. Conclusion
1Mbps~512Kbps 5

Call success
100 ~ 99.9% 10 Because existing QoE measurement schemes
99.9% ~ 99.0% 9 Undefined measure QoE by using a part QoS measurement
rate (S)
99.0% ~ 98.0% 8
parameters, they are difficult to reflect of the various
HO within the cell N/A
services. The quality measurement scheme that uses the
Handover
HO between the cells N/A Undefined preexistence QoS measurement parameter can provide
(H)
HO between the frequencies N/A
the objective quality information in a network-level.
However, this quality measurement scheme is difficult
Channel RF Power, RSSI, N/A Undefined

724

Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on July 15,2010 at 23:14:43 UTC from IEEE Xplore. Restrictions apply.
 to expect the quality which the service end-user
experiences. So, in this paper, by using the quality of
service information measured in a network-level, we
explain the proposed QoS-QoE correlation model for
the objective QoE, and describe the example of the
service QoE evaluation.
In the future, the research for setting up the
quantified QoS class toward QoS parameter of the
various network-levels in the integrated network
environment is needed. Moreover, the research about
which how will set up the value of α, β and K in
consideration of the real time multimedia service kind
and service usage environment is required.
Acknowledgment
This work was supported by the IT R&D program
of MKE/IITA. [2008-S-006-01, Development of Open-
IPTV (IPTV2.0) Technologies for Wired and Wireless
Networks]
*Corresponding Author: Seong Gon Choi (sgchoi@cbnu.ac.kr)
References
[1] ITU-T P.800, “Mean Opinion Score(MOS) terminology,”
Mar 2003.
[2] ITU-T P.862, “Objective quality measurement of
telephone-band (300-3400Hz) speech codec,” Feb 1998.
725
Authorized licensed use limited to: UNIVERSIDAD POLITECNICA DE VALENCIA. Downloaded on July 15,2010 at 23:14:43 UTC from IEEE Xplore. Restrictions apply.
[3] ITU-T P.862, “Perceptual evaluation of speech quality
(PESQ),” Feb 2001.
[4] ITU-T G.107, “The E-model, a computational model for
use in transmission planning,” Mar 2005.
[5] ITU-T P.910, “Subjective video quality assessment
methods for multimedia applications,” Sep 1999.
[6] Wang, Y. “Survey of Objective Video Quality
Measurements,” Tech report, Worcester Polytechnic Institute,
June 2006.
[7] Stas Khirman, Peter Henriksen, “Relationship between
Quality-of-Service and Quality-of-Experience for Public
Internet Service,” PAM 2002, March 2002.
[8] Liu Li-yuan, Zhou Wen-an, Song Jun-de, “The Research
of Quality of Experience Evaluation Method in Pervasive
Computing Environment,” PCA 2006, Aug 2006.
[9] Hideaki YAMADA, Norihiro FUKUMOTO, Manabu
ISOMURA, “A QoE based service control scheme for RACF
in IP –based FMC networks,” CEC/EEE 2007, July 2007.
[10] NetPredic, Inc. White Paper, “Performance Analysis for
Video Streams across Networks,” December 2003.
[11] Mukundan Venkataraman, Shamik Sengupta, Mainak
Chatterjee, “Towards a Video Measuring Video Quality,”
IEEE Transactions on Broadcasting, Vol 50, No. 3, Sep 2004.
[12] ITU-T Recommendation Y.1541, “Network performance
objectives for IP-based service,” Feb 2006.