Measurements and QoS Analysis of Live-World
Mobile Telecommunication Networks
Charalampos N. Pitas, Eleni D. Avgeri, Marilena P. Kallenou, and Philip Constantinou
Mobile Radiocommunications Laboratory
School of Electrical and Computer Engineering, National Technical University of Athens
e-mail : {chpitas, lena, marilena, fkonst}@mobile.ntua.gr
Abstract– The scope of this paper is to study the performance of
contemporary live-world mobile telecommunication networks.
The survey is focused on High Speed Packet Access technology
which is based on Wideband CDMA. The measurement platform
allows QoS assessment, radio access networks audit and
benchmarking. Specifically, services are evaluated via extensive
mobile and static measurements campaigns in both indoor and
various outdoor environments. Emphasis is given to QoS
categorization and characterization. Accessibility, retainability
and integrity are thoroughly examined by Key Performance
Indicators. Enhanced drill down analysis, measurements filtering
and reporting is executed on a spatial relational database
management system.
Index Terms–Mobile Telecommunication Systems (GSM / GPRS /
UMTS / HSPA), Quality of Service (QoS), Key Performance
Indicators (KPIs).
I. INTRODUCTION
Nowadays 3rd generation wireless cellular networks have
already met a great success and have been spread to several
new services. Some of these services require higher bit rates,
reduced latency, and increased spectral efficiency.
High Speed Packet Access represents a new high-speed
data transfer feature released by the 3rd Generation
Partnership Project (3GPP) with the aim of empower UMTS
downlink and uplink data rates and improve support of high
dada-rate packet-switched services. Commercial mobile
telecommuni-cation networks in Greece have already
introduced HSPA technology in coexistence with UMTS R.99
and GSM/GPRS radio access networks.
GSM GPRS WCDMA HSDPA HSUPA
R.99 Release 5 Release 6
Fig. 1 Mobile Telecommunication Systems evolution up to 3.5G.
At the beginning of this paper, SECTION II, a short
introduction of the HSPA technology is presented with new
channel types details. Then, in SECTION III, QoS KPIs (Key
Performance Indicators) are defined for network evaluation.
Afterwards, in SECTION IV, the measurement equipment is
presented as well as the tests which are carried out. Analysis
and measurements post-processing follow in Section V and
finally, SECTION VI, several conclusions are exposed.
II. HIGH SPEED PACKET ACCESS NETWORKS.
HSPA, which is known as 3.5G, is composed of two
technologies, High Speed Downlink Packet Access (HSDPA)
and High Speed Uplink Packet Access (HSUPA) [1]. Services
of 3G wireless cellular networks, such as video telephony,
voice over IP (VoIP), gaming, file downloading and
video/picture sharing, require different Quality of Service
(QoS) with respect to data rate, delay, error rate, capacity and
coverage. HSDPA is targeting increased capacity, reduced
round trip delay, higher peak data rates up to 10 Mbps, higher
user/cell throughput and lower latency.
For this scope, the 3GPP has developed the HSDPA
technology, an evolution of UMTS based on Wideband Code
Division Multiple Access (WCDMA). It is important to note
that HSDPA is a pure access evolution without any core
networks impacts, except for minor changes due to the higher
bandwidth access. For example, in the 3GPP Rel. 5 the
maximum throughput set into the signaling protocol has been
increased from 2 Mbps to 16 Mbps in order to support the
theoretical maximum limit of HSDPA data rate (14.4 Mbps).
It follows that the deployment of HSDPA is very cost
effective since the incremental cost is mainly due to Node B
and Radio Network Controller (RNC) hardware/software
upgrade while the operator cost to provide data services is
significantly reduced. This is due to the high improvements in
spectral efficiency introduced by HSDPA.
In order to achieve these goals, HSDPA technology has
introduced new channels and new techniques. HSDPA
optimizes downlink performance. The HSDPA functionalities
define three new channel types: High-Speed Downlink Shared
Channel (HS-DSCH), High-Speed Shared Control Channel
(HS-SCCH) and High-Speed Dedicated Physical Control
Channel (HS-DPCCH). Three new features of HSDPA are
Adaptive Modulation and Coding (AMC), Hybrid Automatic
Repeat Request (HARQ) and Fast Scheduling.
High Speed Uplink Packet Access (HSUPA) uses the
Enhanced Dedicated Channel (E-DCH) and constitutes a set
of improvements that optimize uplink performance. Among
these improvements are higher throughputs, reduced latency, received. The KPI does not include the duration of
and increased spectral efficiency. HSUPA is standardized in PDP Context activation.
Release 6. Performance gains of HSUPA are achieved through 3. Setup Failure Rate: Tests percentage that failed at the
the following approaches an enhanced dedicated physical setup phase.
channel, a short TTI, as low as 2 ms, which allows faster
responses to changing radio conditions and error conditions, B. Retainability
fast Node-B-based scheduling, which allows the base station 1. Download Time: The interval of time between the first
to efficiently allocate radio resources and fast Hybrid ARQ, and the last byte.
which improves the efficiency of error processing. 2. RTT (Round Trip Time): The interval of time between
a packet is sent and received.
TABLE I
3. Drop Rate: Tests percentage that failed and had already
2G TO 3G SUPPORTED BIT RATES EVOLUTION
UMTS
started normally.
GSM GPRS HSPA
(R.99)
C. Integrity
Theoritical 7.2 Mbps
14.4 171 1. Throughput: The sum of bytes that had been sent or
peak data 2 Mbps Up to
Kbps Kbps
rate 14.4Mbps received to the duration it took.
Typical 2. Overall Service Success: Tests percentage that have
downlink 9.6 384
40 Kbps 5.8 Mbps been successfully completed. It is calculated by
peak data Kbps Kbps
rate excluding from the sum of tests the ones that failed
Typical either at the setup phase either they dropped.
uplink 9.6
20 Kbps 64 Kbps 2Mbps
peak data Kbps
rate Furthermore, voice quality is of the first consideration for
packet switched mobile telephony. Speech Quality indicator is
III. QUALITY OF SERVICE KEY PERFORMANCE INDICATORS the major factor determining customer satisfaction, since it
represents the quality of listener’s reception. It is generally
According to ETSI’s standards [2] QoS KPIs can be expressed as a Mean Opinion Score (MOS) which is valued
categorized in five phases during service use from the between 1 (bad level) and 5 (excellent level). Hereupon
customer's point of view: speech quality is measured and assessed by relative algorithm
1. Network Availability: Probability that the mobile services as well as statistics are presented.
are offered to a user.
2. Network Accessibility: Probability that the user performs IV. MEASUREMENT SYSTEM
a successful registration on the PLMN.
3. Service Accessibility: If the user wants to use a service, A. System Configuration
the network operator should provide him as fast as The measurements and analysis have been conducted using
possible access to the service. Includes KIP’s that SwissQual Diversity equipment.
describe how successful is the service access (call setup
rate) as well as KPI’s that describe setup time (call setup
time).
4. Service Integrity: This describes the Quality of Service Control &
Analysis
during service use. Includes KPI’s that describe the SQL Server
Test Manager NQDI
quality of the service (MOS for voice quality).
5. Service Retainability: Service retainability describes the
Backend
termination of services (in accordance with or against the
will of the user).Includes KPI’s that count the time that a CS Network IP
Network
service can retain (Drop Call Rate or call holding time). Media Server
Voice Server Mobile Network

Analysis is concentrated on QoS for PS data. Crucial KPIs

are comprehensively studied: Probes

A. Accessibility
1. PDP Context Activation Duration and Success Rate:
indicates the time required for the service access as
well as the success rate.
2. Setup Time: The interval of time from the moment the
user service requests until the first byte is sent or
Diversity
Benchmarker
Fig. 2. Measurement System Architecture
Diversity is a portable, highly scalable platform for drive-
test systems that allows the user to measure more than one
cellular network of different technology. The system is mobile
in automotive setting or carried by a pedestrian. The interface The session of voice tests contains both mobile phone to
to the mobile network is through special models of cellular mobile phone test as well as mobile to PSTN phone test. In
phones that give access to internal network parameters [4],[5]. both cases A-side and B-side are defined and exchange short
The measurement system sets up calls, records cellular voice recordings alternately in order to emulate a call
network parameters, measures voice quality calls using Voice conversation. The conversation schema is presented at Fig.2.
Server as well as the quality of other payload such as data,
email, ping and video using Media Server. Diversity provides
QoS evaluation, benchmarking, and RF and network
optimization.
It is designed for both wireless and fixed networks. The
quality of service in voice and data tests is performed through
related speed and audio quality algorithms. Tests execution is
configured and controlled by Test Manager and also
measurements data can be displayed live on NQView. Test
Manager and NQView are software applications running on
the laptop during the measurements. In this scenario, laptop
user in the vehicle monitors the process of measurements.
B. Measurement Campaign
The measurement system is placed in a car while executing a
measurement campaign [6]. In the campaign a session of tests
is consecutively repeated.
Fig. 4 Quality of voice measurement tasks configuration.
The data session contains the following actions and tests:
1. Activation Packet Data Protocol (PDP Context
V. ANALYSIS – POST PROCESSING RESULTS
Activation)
2. Reception of 10 MB file (FTP Download) from a A. Analysis Tool
public server broadband connected to the internet.
3. Sent a 3 MB file to the public server (FTP Upload). Measurements are imported in the post processing application
4. Web browsing at NTUA (www.ntua.gr) website of size based on SwissQual’s NQDI (NetQual Database Interface)
of 1 MB. which is connected with a relational time-spatial database.
5. 10 Ping with 400 Bytes Packet Size. NQDI offers enhanced drill down analysis, filtering and data
mining, KPI statistics, map plotting, database querying/mining
6. Deactivation Packet Data Protocol (PDP Context and reporting.
Deactivation). Using NQDI the measurement data can be filtered by using
complicated queries in order to analyze specific part of all the
data located in the database.
A typical structure of a query based filter applied in the data
base aims to show up performance parameter values and
compute statistics from a set of well organized measurements
or/and network information tables where crucial criteria and
combined conditions take place. A general query form is
presented:
SELECT <List_of_Parameters>, <Statistics_Functions>
FROM <List_of_Measurements_Tables> OR/AND
<List_of_Radio_Network_Parameters>
WHERE <List_of_Combined_Criteria/Conditions>

The QoS statistics analysis is focused on significant KPIs,

especially, achieved download and upload throughput, overall
http success rate and round-trip-time [7].

Fig. 3 Measurements Scenario

 In Table II, a statistics analysis of the achieved download
bitrates is presented in details. A percentage of 42.59% is
remarked between 1.8Mbps and 3.6Mbps when a worthy
28.71% is provided up to 7.2Mbps. It is obvious that
14.4Mbps technology has not introduced yet.
Download throughput statistics functions are plotted at the
following figure. It is remarked that only 8,33% of download
measurements are below 384kbps while 91,67% are fine
supported by HSDPA.

Fig. 6. FTP Download Throughput PDF and CDF graph.

In Table III, a statistics analysis of the achieved upload

bitrates is presented in details. A worthy percentage of
77.19% is remarked between 384kbps and 2Mbps. It is clear
Fig. 5. Analysis process diagram. that HSUPA technology is well introduced and is traced in the
majority of the measurement samples. Besides, 21,02% of the
B. Results relevant samples are detected between 128kbps and 384kbps.
1) Packet Switched Services So, broadband access is well supported for demanding upload
It is known that the maximum typical throughput for use.
HSDPA 7.2Mbps technology is 5.8Mbps while for HSDPA TABLE III
FTP UPLOAD THROUGHPUT STATISTICS
3.6Mbps technology is 3Mbps. As it is arisen from the
following diagram, HSDPA 7.2Mbps is applied in the UL throughput
FUNCTION PERCENTAGE
network even though 3.6Mbps technology is used in more (kbps)
samples. >2000 0,00%
384<UL<=2000 77,19%
TABLE II
FTP DOWNLOAD THROUGHPUT STATISTICS PDF 128<UL<=384 21,02%
DL throughput 64<UL<=128 1,66%
FUNCTION PERCENTAGE UL<=64 0,13%
(kbps)
>7200 0,00% >2000 0,00%
3600<DL<=7200 28,70% 384<UL<=2000 77,19%
PDF 1800<DL<=3600 42,59% CDF 128<UL<=384 98,21%
384<DL<=1800 20,37% 64<UL<=128 99,87%
DL<=384 8,33% UL<=64 100,00%
>7200 0,00%
3600<DL<=7200 28,70% Upload throughput statistics functions are plotted at the
CDF 1800<DL<=3600 71,30% following figure. It is mentioned that 98,21% of the upload
384<DL<=1800 91,67% measurement samples are located above 128kbps and up to
DL<=384 100,00% 2Mbps owing to HSUPA technology. Less than 2% of the
samples are lower than 128kbps.
 following diagram (Fig.8) we present a total time distribution
of each technology.

Fig. 7. FTP Upload Throughput PDF and CDF graph.

Download time is used to evaluate HTTP performance,
which is considered to be more efficient than throughput. In
the diagram, the average and minimum download time is
presented, as well as the Overall Service Success, tests that
have successfully completed without drop or setup failure.
The average download time 7,1sec is capable for loading a
website of 1 MB.
Fig. 9. Data Serving Technology distribution.
2) Circuit Switched Services – Telephony
Speech Quality indicator is the major factor determining
customer satisfaction, since it represents the quality of
listener’s reception. It is generally expressed as a Mean
Opinion Score (MOS) which is valued between 1, bad level,
and 5, excellent level. Most objective algorithms [8][9][10]
are based on a comparison between a reference sample and a
coded version of the reference. The most widely used models
of algorithms are ITU-T P.862 standardized in 2001. In
contrast with subjective experiments, this model enables
extensive testing to be performed over short periods. The
results of ITU-T P.862 algorithm are displayed in the
following figures.

Fig. 8. Total HTTP session time and Success Rate.

Fig. 10 CDF of Listening Quality (P.862.1)
Based on the results of the Round Trip Time KPI and the
following table, the network performance considered to be
very good with average RTT 194msec.

TABLE III
PING RTT STATISTICS
Count 528
% 99,62%
Avg [ms] 194
Min [ms] 101
Max [ms] 3454
StDev [ms] 165
95% Cut-Off 345
Fig. 11 Listening Quality distribution.

For each test, Diversity stored the duration of the provided 3) KPIs Map Plotting
data service technology. Combining all types of tests, in the Finally, Fig.12 and Fig.13 display crucial QoS parameters –
Channel Quality Indicator and Listening Quality - having
applied a spatial filter on a polygon which contains a route of
the NTUA Campus. The result of the spatial query has been
plotted on a GIS Viewer (Google Earth) giving us an
opportunity to locate problematic areas regarding performance
or RF coverage.
Fig. 12 Channel Quality Indicator (CQI) plotted on Google Earth
Fig. 13 Listening Quality (LQ P.862.1) plotted on Google Earth.
VI. CONCLUSIONS
In this paper, it is concluded by performance measurements
that nowadays mobile telecommunication networks meet a
great evolution, integration and interoperability in Greece.
Mobile broadband access supports ubiquitously QoS
demanding services which are well described by KPIs.
Consequently, it is necessary and obligatory (by independent
national regulatory authority [11]) for mobile operators to
perform continuously measurements so as to audit and
optimize the efficiency of radio access networks as well as for
benchmarking reasons. Open and frequent publication of
crucial KPIs will lead mobile operators to robust competition
introducing innovative and qualitative services. Contemporary
technologies like Long Term Evolution UMTS, also known as
HSPA+, will explicitly uplift the provided bitrates. Future
work may be performing large scale measurements campaigns
in differentiated environments of interest in order to
extensively examine QoS parameters.
REFERENCES
[1] H.Holma and A.Toskala, “HSDPA/HSUPA FOR UMTS, High Speed
Radio Access for Mobile Communications”, Ed. John Wiley & Sons,
LTD, 2006.
[2] ETSI TS 102 250-1, “Speech Processing, Transmission and Quality
Aspects (STQ); QoS aspects for popular services in GSM and 3G
networks; Part 1: Identification of Quality of Service aspects”, V1.2.1,
Technical Specification, 2007.
[3] ETSI TS 102 250-2, “Speech Processing, Transmission and Quality
Aspects (STQ); QoS aspects for popular services in GSM and 3G
networks; Part 2: Definition of Quality of Service parameters and their
computation”, V1.6.2, Technical Specification, 2008.
[4] ETSI TS 102 250-3, "Speech Processing, Transmission and Quality
Aspects (STQ); QoS aspects for popular services in GSM and 3G
networks; Part 3: Typical procedures for Quality of Service
measurement equipment", V1.3.2, Technical Specification, 2005.
[5] ETSI TS 102 250-4, “Speech Processing, Transmission and Quality
Aspects (STQ); QoS aspects for popular services in GSM and 3G
networks; Part 4: Requirements for Quality of Service measurement
equipment”, V1.2.1, Technical Specification, 2007.
[6] ETSI TS 102 250-5, “Speech Processing, Transmission and Quality
Aspects (STQ); QoS aspects for popular services in GSM and 3G
networks; Part 5: Definition of typical measurement profile”, V1.5.1,
Technical Specification, 2008.
[7] ETSI TS 102 250-6, "Speech Processing, Transmission and Quality
Aspects (STQ); QoS aspects for popular services in GSM and 3G
networks; Part 6: Post processing and statistical methods", V1.2.1,
Technical Specification, 2004.
[8] ITU-T Rec. P.862, “Perceptual evaluation of speech quality (PESQ): An
objective method for end-to-end speech quality assessment of narrow-
band telephone networks and speech codecs”, 2/2001.
[9] ITU-T Rec. P.862.1, “Mapping function for transforming P.862 raw
result scores to MOS-LQO”, 11/2003.
[10] ITU-T Rec. P.862.2, “Wideband extension to Recommendation P.862
for the assessment of wideband telephone networks and speech codecs”,
11/2007.
[11] Hellenic Telecommunications and Post Commission (EETT), Desicion,
No.480/017, "Definition of quality indicators of public provided
electronic communication services and specification of the content and
the form of the reported information as well as the method and the time
of the publication by the electronic communication services providers",
Official Government Gazette of Hellenic Republic, OGG/1153/B/2008,
2008.