GSM Radio Network Performance

Evaluation Delivery Report (Template)

For internal use only

Huawei Technologies Co., Ltd.

All Rights Reserved
2012-7-21

Revision Record
Date

Version Description

2010-9-28

2.2

2012-07-21

Initial

HUAWEI Confidential

Review

Author

Lishucheng

2010-9-28

Page2, Total25

For internal use only

Content
1 Introduction...................................................................7
2 Workflow.......................................................................8
3 Evaluation information collection....................................9
1.1 Complaint Information Collection............................................................................................9
1.2 Statistic data Collection...........................................................................................................10

4 Subscriber complaint ...................................................11

1.3 Various complaints percentages...............................................................................................11

5 DT preparation.............................................................12
1.4 Specific drive test measurement tools.....................................................................................12
1.5 External Antenna and Attenuator Requirement.......................................................................12
1.6 Cluster definition.....................................................................................................................13
1.7 Route definition.......................................................................................................................15
1.8 Test Methodology....................................................................................................................16

6 Drive Test.....................................................................17
7 DT data procession.......................................................18
1.9 Coverage KPI..........................................................................................................................18
1.9.1 Rx-lev log file....................................................................................................................18
1.9.2 Rx-lev Distribution............................................................................................................18
1.9.3 Rx-lev Coverage................................................................................................................18
1.9.4 Rx-Qual log file.................................................................................................................19
1.9.5 Rx-Qual Distribution.........................................................................................................19
1.9.6 Rx-Qual Coverage.............................................................................................................19
1.10 Accessibility KPI...................................................................................................................19
1.10.1 Call Setup Success Rate of MS Origination log file.......................................................19
1.10.2 Origination Call Setup Success Rate...............................................................................20
1.10.3 Call Setup Success Rate log file......................................................................................20
2012-07-21

HUAWEI Confidential

Page3, Total25

For internal use only

1.10.4 Call Setup Success Rate .................................................................................................20

1.11 Retainability KPI...................................................................................................................21
1.11.1 Call-Drop Rate log file....................................................................................................21
1.11.2 Call-Drop Rate.................................................................................................................21
1.12 Handover...............................................................................................................................24
1.12.1 Handover Statistics log file.............................................................................................24
1.12.2 Handover Statistics..........................................................................................................24

8 Summary......................................................................24

2012-07-21

HUAWEI Confidential

Page4, Total25

For internal use only

Figures
Workflow of Network Evaluation........................................9
Complaints percentage....................................................11
Top view for install Ext ANT..............................................13
Cluster border ................................................................14
Test Route.......................................................................16
Rx-lev Coverage..............................................................18
Rx-Qual Coverage............................................................19
Drop Call Location-1.......................................................23
Drop Call Reason-2.........................................................24

2012-07-21

HUAWEI Confidential

Page5, Total25

For internal use only

Tables
Timelines of Performance Evaluation..................................7
Complaint Information.......................................................9
Pre-swap statistic data....................................................10
Cluster sites list..............................................................14
Origination Call Setup Success Rate.................................20
Call Setup Success Rate..................................................20
Call Drop Rate.................................................................21
Handover Statistics.........................................................24
Drive Test KPI summary...................................................24

2012-07-21

HUAWEI Confidential

Page6, Total25

For internal use only

Introduction

Before a radio network swapping, a well evaluation of the original network should
be carried out. The evaluation result will be the benchmark of the swapped network.
This report will discuss about methodology of evaluation and give an example to
represent real status of the network.
Evaluation will be done in 3 aspects as follows:

Complain status evaluation

Traffic Statistic evaluation

Drive Test evaluation

For the previous 2 tasks the main work are collect information from operator and
classify them into tables that can be reference after the swapping. Thus the key task
in this report is the last one Drive Test evaluation, it will be discuss in detail.
Scope of This Report:

Evaluation information collection

DT preparation

Drive Test

DT data procession

Summary

Performance Evaluation Schedule:

Table 1.1 Timelines of Performance Evaluation
No.

Items

Duration

1
2
3
4
5
6
7

Update Radio Engineering Parameter

Collect Complaint Information
Collect Performance Statistic before Swap
Review Existing Sites Coordinates
Drive Test
DT Data Post Process
Generate Evaluation Report

0.2 week
0.4 week
0.4 week
0.4 week
7 week
6 week
0.6week

2012-07-21

HUAWEI Confidential

weekly progress
1st 2nd 3rd 4th 5th 6th 7th 8th

Page7, Total25

For internal use only

Workflow

The workflow of Performance Evaluation as below:

Existing Radio
Network Information

Legend
START

Data-In-001

Huawei
001
Operator

Collect Existing
Radio Network
Information

Huawei &
Operator

Doc-Out-001
002

NOC

Review Existing Sites

Coordinates

OS

003
Drive Test

OS/
RD

004
DT Data Post Process

005
Generate Evaluation
Report
Doc-Out-002

END

2012-07-21

OS

Out Sourcing

RD

Remote Delivery

HUAWEI Confidential

Page8, Total25

For internal use only

Figure 1.1 Workflow of Network Evaluation

Evaluation information collection

Before commencing swap process, there is a group of information need to be

collected from the original radio network. These information will become the
baseline for evaluate the swapped network performance in the future.

1.1 Complaint Information Collection

Complaint information can be get from the operator call centre, it record the
complaint history information before the swap. This information can indicate
existent problems in the original network that can be the reference for swapped
network. It used to evaluate the feeling of subscriber in the network before and after
the swap.
It should include recent subscribers cases which are still open.(problems, areas,
analysis and follow up action items)
The recent period means during last 2 weeks or 1 month before the swap. If swap is
delayed and overdue past 1 month, this information needed to re-collect.
Complaint information showing as the table below:
Table 1.1 Complaint Information
NO

Problem

Problem

Problem

Related

Related

Related

Related

Trouble

Analysis

Solution

REMAR

Detail

Area

time

RNC ID

CELL ID

SITE

CLUSTER

shooting

conclusion

& result

DT data &

Missing

Adding

Trace

neighbor

neighb

NAME

Call

xx

hh:mm,

drop/Ca

buildin

dd-

INTHRA

ll setup

g,

mm-

EXPRESS

xx

2012-07-21

10

1001

RAM

data

HUAWEI Confidential

or

Page9, Total25

For internal use only

fail/Low

street,

through

xx

yyyy

WAY

cells

cells,
the

put

district,

proble

xx

city/xx

disapp

cross

eared.

road,
xx
district,
xx city

This table need keep same with the Complaint Information sheet of the <
GRFSDTXXXXX-Radio Network complaint Information>

1.2 Statistic data Collection

Pre-swap statistic data need to be collected as the benchmark criterion for swapped
network. The statistic data should include all the items that negotiated in KPI
Mapping phase.
The data should be collected in the last 2 weeks or 1 month before the swap. If swap
is delay and overdue past 1 month, this data need re-collect to make sure the statistic
will represent real time performance of the original network.
Statistic data for example as the table below:
Table 1.2 Pre-swap statistic data
2010-10-11.
Drop
Rate

Call

2010-10-12

2010-10-13

2010-10-14

2010-10-15

2010-10-16

2010-10-17

1%

Call
Setup
Success Rate

97%

Soft
Hand
Over Success
Rate

98%

2012-07-21

HUAWEI Confidential

Page10, Total25

For internal use only

The KPI items here are just for example, the exact ones need negotiated with
operator in the KPI Mapping phase.

Subscriber complaint

1.3 Various complaints percentages

Figure 1.1 Complaints percentage

2012-07-21

HUAWEI Confidential

Page11, Total25

For internal use only

DT preparation

1.4 Specific drive test measurement tools

Probe V2R3 for voice and data tests

External Antenna 0dB gain with cable and connector

GPS with external antenna

Laptop loaded with Probe V2R3 or TEMS

2 Mobiles and 1 data card

In case of some KPI acceptance items are special in other vendors test and analyze
tool, validate whether them can be realized in Huaweis tool also. If not TEMS/Pilot
Fleet will be recommended or need choose other tool, that means DT tool
purchasing flow need to be started. Pay attention the operation guide of the test tools
from other vendor should be considered also.
Setup:
MS: CS Calls using a National Roaming disabled SIM. MS is set as GSM mode
only and locked on operator network (2G->3G cell reselection is not allowed)
Data Card for PS, SIM card special requirement:

Using a National Roaming disabled SIM and locked on operator GSM

Support EGPRS

SIM cards provided and prepared by operator

FTP Download Server provided by operator

CS Calls Answering Machine provided by operator

Vehicle

DC to AC converter

1.5 External Antenna and Attenuator Requirement

The external antenna of MS will be used in Drive Test, and the antenna will be
mounted on rooftop of drive test car.
The following figure shows the external antenna installation.
It is necessary that 1 Lambda displacement between Antenna and the roof edge.
2012-07-21

HUAWEI Confidential

Page12, Total25

For internal use only

Roof luggage rack is not allowed.

S P

TM

S P
+

A N

A MN

TS

Figure 1.1 Top view for install Ext ANT

1.6 Cluster definition

The clusters shall be defined based on the principles determined both huawei and
customer. Principles please refer to <GRFSTG00209-GSM RF Optimization
Technical Guide-V2R2-E> section 2.2.
There is a cluster definition example below:

2012-07-21

HUAWEI Confidential

Page13, Total25

For internal use only

Figure 1.2 Cluster border

List of cluster sites:
Table 2.1 Cluster sites list
SiteID

NodeB Name

Cell Name

10350

IESSW08

IESSX

10357

25

10350

IESSW08

IESSY

10358

06

10350

IESSW08

IESSZ

10359

57

10360

MALLSURW08

MALLSURX

10367

18

10360

MALLSURW08

MALLSURY

10368

09

10360

MALLSURW08

MALLSURZ

10369

20

10390

SAIBAW08

SAIBAX

10397

38

10390

SAIBAW08

SAIBAY

10398

10390

SAIBAW08

SAIBAZ

10399

17

11070

CGUEVARAW08

CGUEVARAX

11077

60

11070

CGUEVARAW08

CGUEVARAY

11078

11070

CGUEVARAW08

CGUEVARAZ

11079

52

2012-07-21

HUAWEI Confidential

Cluster

Cell ID

Frequenc
y

Page14, Total25

For internal use only

Total number of sites for the cluster is: 10

Percentage Sites Radiating: 100%

1.7 Route definition

The route will be defined by Huawei and sent to customer for confirmation. If
customer agree on it, Huawei will use it for the cluster route otherwise changes will
be made.
All sectors in every site in the cluster should be covered by the drive route.
Routes should cover Business centers, shopping centers, tourist attractions and
railway stations.
Major roads: Motorways in both directions.
The routes should be laid out to gain a clear footprint of each cell.
Routes should avoid roads with restricted access, pedestrian areas or secure areas or
those requiring special access conditions.
Routes should be captured in MapInfo format or something similar, i.e. Google
Earth.
At least 200 samples for PS and CS have to be collected.
Below is a sample of test route.

2012-07-21

HUAWEI Confidential

Page15, Total25

For internal use only

Figure 1.3 Test Route

1.8 Test Methodology

Cluster drive tests shall perform 2 call scenarios: Voice short calls, VP short calls
with 120 sec duration with 10 sec interval.
PS functionality tests were performed by downloading a huge file from an FTP
server with CQT (Call Quality Test).

2012-07-21

HUAWEI Confidential

Page16, Total25

For internal use only

Drive Test

During the DT a Protocol-File should be written which captures all the problems
like broken cable, loose cable, mobile error and any other equipment error, unusual
behaviour or lunch break.
The protocol shall as well capture the DT-engineers name and vehicle number.
Pre and Post DT shall be done using the same vehicle and measurement equipment.
The DT-Engineer has to check the stability and functionality of the equipment
during the DT.
In case of major problems during the DTs the DT Engineers should contact Huawei
technical assistance personnel.
It is important to check if the right network is measured.
Information: Tests will be done by using National Roaming disabled SIM-Cards.
The Log-Files should not be too big. A proper splitting of files length is necessary.
Recommend every log file less than 20M.

2012-07-21

HUAWEI Confidential

Page17, Total25

For internal use only

DT data procession

1.9 Coverage KPI

1.9.1 Rx-lev log file
PSDL_070620_Toapayoh_1.gen

1.9.2 Rx-lev Distribution

Figure 1.1 Rx-lev Coverage

1.9.3 Rx-lev Coverage

Rx-lev (>=-95dBm) Ratio = 99.24%
2012-07-21

HUAWEI Confidential

Page18, Total25

For internal use only

RSCP (>=-85dBm) Ratio = 96.17%

1.9.4 Rx-Qual log file

PSDL_070620_Toapayoh_1.gen

1.9.5 Rx-Qual Distribution

Figure 1.2 Rx-Qual Coverage

1.9.6 Rx-Qual Coverage

Rx-Qual (>=4dB) Ratio = 96.54%

1.10 Accessibility KPI

1.10.1 Call Setup Success Rate of MS
Origination log file
PSDL_070620_Toapayoh_1.gen

2012-07-21

HUAWEI Confidential

Page19, Total25

For internal use only

1.10.2 Origination Call Setup Success Rate

Table 2.1 Origination Call Setup Success Rate
Call Statistics

Result

Rate

Total Number of Calls

26

100.0%

Total Drops

7.7%

Total Completed Calls

23

88.5%

Total Call Setup Success

26

100.0%

Total Call Setup Failure

0.0%

Total Number of Outgoing Call

26

100.0%

Total Drops of Outgoing Call

7.7%

Outgoing Call Setup Success

26

100.0%

Outgoing Call Setup Failure

0.0%

Origination Call Setup Success Rate

= Total Call Setup Success / Total Number of Calls = 26 / 26 * 100% = 100.0%

1.10.3 Call Setup Success Rate log file

PSUL_070620_Toapayoh_1.gen

1.10.4 Call Setup Success Rate

Table 2.2 Call Setup Success Rate
Call Statistics

Result

Rate

Total Number of Calls

23

100.0%

Total Drops

0.0%

Total Completed Calls

22

95.7%

2012-07-21

HUAWEI Confidential

Page20, Total25

For internal use only

Total Call Setup Success

23

100.0%

Total Call Setup Failure

0.0%

Total Number of Outgoing Call

23

100.0%

Total Drops of Outgoing Call

0.0%

Outgoing Call Setup Success

23

100.0%

Outgoing Call Setup Failure

0.0%

Call Setup Success Rate of VP

= Total Call Setup Success / Total Number of Calls = 23 / 23 * 100% = 100.00%

1.11 Retainability KPI

1.11.1 Call-Drop Rate log file
PSDL_070620_Toapayoh_1.gen

1.11.2 Call-Drop Rate

Table 2.3 Call Drop Rate
Call Statistics

Result

Rate

Total Number of Calls

26

100.0%

Total Drops

7.7%

Total Completed Calls

23

88.5%

Total Call Setup Success

26

100.0%

Total Call Setup Failure

0.0%

Total Number of Outgoing Call

26

100.0%

Total Drops of Outgoing Call

7.7%

2012-07-21

HUAWEI Confidential

Page21, Total25

For internal use only

Outgoing Call Setup Success

26

100.0%

Outgoing Call Setup Failure

0.0%

One drop call was not only caused by the normal release from DL_DCCH but
also traffic time was 120sec should be excluded from the KPI

The other drop call was caused by the Abis interface problems between BSC
501 and BSC 403.

Origination Call Drop Rate

= Total Drops/ Total Number of Calls = 0 / 26 * 100% = 0.00 %

2012-07-21

HUAWEI Confidential

Page22, Total25

For internal use only

Figure 1.3 Drop Call Location-1

2012-07-21

HUAWEI Confidential

Page23, Total25

For internal use only

Figure 1.4 Drop Call Reason-2

1.12 Handover
1.12.1 Handover Statistics log file
PSDL_070620_Toapayoh_1.gen

1.12.2 Handover Statistics

Table 4.1 Handover Statistics
Handover
Statistics

Total

490

Total Handover
Performed
Total Handover
Success

489

Total Handover Failure

99.80%

Handover Success
Rate

Summary

Below is the Drive Test KPI summary:

Table 1.1 Drive Test KPI summary
KPI

2012-07-21

Original network Test

Results

Contractual KPI
criteria

HUAWEI Confidential

Acceptance
Results

Page24, Total25

For internal use only

Rx-lev

97.62%@-100dBm

>=96%@-95dBm

Rx-Qual

97.52%@-12dB

>=96%@-12dB

Call Setup Success

Rate

100.00%

4.17%

1%

100.00%

Call Drop Rate

HO Success Rate

KPI: Means Drive Test KPI items agreed by both Huawei and operator.
Original network Test Results: Means the original network KPI test values.
Contractual KPI criteria: Means Drive Test KPI criteria agreed by both Huawei and
operator.
Acceptance Results: It will fill with the KPI test values that test from the swapped
network.
Please note when Acceptance Results of one KPI item achieves any one of the
Original network Test Results or Contractual KPI criteria, that KPI item is passed.

2012-07-21

HUAWEI Confidential

Page25, Total25