Professional Documents
Culture Documents
Version: V1.1 Security Category: Secret Serviceable range: Radio Planning Section
ZTE Corporation
Secret Secret
Version Introduction
Version V1.0 Date 2008-01-21 Writer Chen Chun Assessor Zheng Hao None 1V3 KPI and EDGE KPI were added. 2Amendment: mainly directed at V2.97 Zheng Hao, V1.1 2008-12-07 Yang Jiazhi Chen Chun, Fei Aiping and V6.0. 3Some classic KPI cases and comments were added. 4Principles for KPI risk analysis were added. Amendment Records
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements.
Secret Secret
CONTENTS
1 2 2.1 2.1.1 2.1.2 2.1.3 2.2 2.2.1 2.2.2 2.2.3 2.2.4 2.2.5 2.2.6 2.2.7 2.2.8 2.3 2.3.1 2.4 2.4.1 2.4.2 2.4.3 2.4.4 2.4.5 2.5 2.5.1 2.5.2 3 3.1 3.2 3.3 Overview...........................................................................................................................1 CS Radio Service Statistical KPI....................................................................................4 Availability ........................................................................................................................4 SDCCH availability/SDCCH ....................................................................................4 TCH availability/TCH ...............................................................................................4 Cell utilization ...........................................................................................................4 Accessibility ......................................................................................................................5 Random Access Success rate.....................................................................................5 SDCCH blocking rate................................................................................................5 TCH blocking rate .....................................................................................................6 Radio connection ratio...............................................................................................6 SDCCH drop rate ......................................................................................................7 SDCCH assignment success rate...............................................................................7 TCH assignment success rate ....................................................................................7 Call setup success rate ...............................................................................................8 Mobility .............................................................................................................................9 Handover success rate ...............................................................................................9 Reliability ..........................................................................................................................9 TCH Call Drop Rate..................................................................................................9 Traffic Call Drop .....................................................................................................10 Call Completion rate................................................................................................10 UL RxQual 0-4 ........................................................................................................11 DL RxQual 0-4 ........................................................................................................11 Call Setup time ................................................................................................................12 Call setup time (GSM to PSTN)..............................................................................12 Call setup time (GSM to GSM)...............................................................................12 CS Radio Service Test KPI ...........................................................................................13 Call Setup Success Rate ..................................................................................................13 Handover Success Rate ...................................................................................................13 Call Drop Rate.................................................................................................................13
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements.
Secret Secret
3.4 3.4.1 3.4.2 3.4.3 4 4.1 4.1.1 4.1.2 4.1.3 4.1.4 4.1.5 4.1.6 5 5.1 5.1.1 5.1.2 5.1.3 5.1.4 5.1.5 5.1.6 5.2 5.2.1 5.3 5.3.1 5.3.2 5.3.3 5.4 5.4.1 6 6.1 6.2 6.3 6.4 6.5 Coverage (Drive Test)......................................................................................................13 SQI...........................................................................................................................14 MOS ........................................................................................................................14 MOS & PSEQ .........................................................................................................15 PS Radio Service Statistical KPI ..................................................................................18 Accessibility ....................................................................................................................18 DL TBF establishment success rate .........................................................................18 UL TBF establishment success rate .........................................................................18 GPRS RLC ..............................................................................................................19 GPRS RLC ..............................................................................................................20 EGPRS RLC............................................................................................................20 EGPRS RLC............................................................................................................20 PS Radio Service Test KPI............................................................................................22 Accessibility ....................................................................................................................22 Attach success rate...................................................................................................22 Average GPRS attach time ......................................................................................22 PDP context activation success rate ........................................................................22 PDP context activation time ....................................................................................23 PING test success rate .............................................................................................23 Average PING test time ...........................................................................................23 Mobility ...........................................................................................................................24 Routing Area Update Successful rate ......................................................................24 Data Throughput..............................................................................................................24 Average FTP download rate ....................................................................................24 Average FTP upload rate .........................................................................................25 Throughput ..............................................................................................................25 Reliability ........................................................................................................................25 PDTCH Drop Rate ..................................................................................................25 Introduction to CN Statistical KPI...............................................................................27 Paging success rate ..........................................................................................................27 Location update success rate ...........................................................................................27 Attach success rate ..........................................................................................................27 GPRS average attach time ...............................................................................................28 PDP context activation success rate ................................................................................28
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements.
Secret Secret
6.6 7 7.1 7.2 7.3 7.4 7.5 7.6 Routing Area Update Success rate ..................................................................................28 Comments on KPI Cases...............................................................................................30 Pengshui (Chongqing) PS Test Result.............................................................................30 Jishou (Hunan) China Mobile EDGE Test Result ...........................................................31 Acceptance requirements for EDGE KPI of Data Service of a network .........................33 Operation & Maintenance KPI of a Network ..................................................................34 Operation & Maintenance KPI of a network in Southeast Asia ......................................36 KPI requirements for Indonesia Telkomsel network .......................................................40
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements.
Secret Secret
1 Overview
This guidebook collected the Key Performance Indicators (KPI) for CS and PS services, which are now commonly adopted in presales bidding projects; and the definitions of different KPIs and counter formulas for V2/V3 are explained here as 5 well. Meanwhile, with the aim to unify response on KPIs and avoid the possible market risks, after analyzing the performance indicators of current networks of both domestic and overseas key projects, the writer provides the reference values and risk values of different KPIs in this guide. If the KPIs listed in the bidding documents go beyond the reference values and risk values (suppose risk value of 10 the KPI is provided; otherwise, suggested values shall be applied.), we need to conduct risk analysis and evaluation, meanwhile, try to avoid the risk and send out pre-warning. The KPIs can be quite different due to the differences in network structure and dimension. To make sure that our KPI can meet requirements in bidding documents, we need not only support from theories, but accumulation of 15 work experience as well. The suggested values and risk values of KPI mentioned in this guide are reference suggestions concluded from the current conditions of GSM network. As GSM techniques and skills are developing and equipment performance is improving constantly, the reference standards for KPI will change accordingly. The key requirement for engineers engaged in presales network planning is to 20 thoroughly understand and master the related KPIs, which is also the basic but highest demand for radio network planning engineers. Why network operators set up KPIs and what are they aimed at? How to define KPI and what can be resulted from deviation of KPI? How much influence can the value of KPI impose on presales bidding? 25 How to clarify KPI and how to achieve favorable negotiation results during KPI clarification? How to avoid and pre-warn KPI risks? What is the development direction of KPI assessment? Each presales radio network planning engineer shall possess related techniques of 30 and up-to-date answers to the six questions above. For lack of space, the six This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 1Total 45 pages
Secret Secret
questions will not be further discussed in this guidebook. The engineers will be able to experience and get to know them in presales planning work in future. Generally, KPI can be classified into four types: statistical KPI, testing KPI, equipment performance KPI, customer perception KPI. Only statistical KPI and 5 testing KPI are elaborated here. As for equipment performance KPI and customer perception KPI, they are focused on in another guidebook. Nowadays, network operators are attaching more and more importance to KPI during negotiations. Besides the four types mentioned above, a lot more aspects are involved in KPI. KPI is also closely related to the basic network environment. Therefore, KPI is 10 affected by various factors. That is why operators are now regarding KPI as a very important leverage to shift risk and decrease network investment and maintenance cost. Equipment manufacturers must commit themselves to suitable, reasonable and timely KPI, so as to bring down contract risk and ensure profits. Rationalize commitment, share risks is an important principle in presales planning work. 15 The following are the factors that shall be taken into consideration when we work on KPI: 1. network dimension; 2. network type; 3. degree of subscriber(number) growth; 20 4. geographical distribution of network(urban, countryside, village) 5. equipment types; there is difference in KPI for V2 and V3. Functional principles for this guidebook (mainly directed at KPI): 1. This guide is only available to the radio network planning engineers engaged in presales bidding. No one is entitled to spread it outside or extensively spread 25 it inside the company. The right to interpret the serviceable range of this guide remains with GSM Technical Research Section & Resources Section. 2. It is suggested to release the guidebook in encrypted document. It can not be procured unless logging into document security software. 3. 30 The guide is a reference to radio network planning staff engaged in presales bidding. It can not replace the presales KPI project evaluation.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 2Total 45 pages
Secret Secret
4. The KPI values enclosed in this guide are valid within only half a year since the release of it. After half a year, it needs to be updated and reassessed. 5. The guide is mostly applicable to V2.97 of V2 and V6.0 (ISMG) and V6.1 (Minos) of V3. 5 Notes: V6.0 (ISMG), V6.1 (Minos), sic passim
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 3Total 45 pages
Secret Secret
The ratio of the SDCCH usable total to the total SDCCH allocated in network is mainly used to examine equipment performance, especially to inspect the utilization of SDCCH in Notes the cells. It is directly related to the stability of equipmenttransmission and power supply, etc.. It reflects the utilization ratio of network equipment and the stability of initial network operation.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 4Total 45 pages
Secret Secret
value Indicator definition Counter formula This is mainly to compare performance of preplanning network with that of actual Notes network. The prerequisite is that precise traffic model traffic dispersion network capacity and redundancy rate shall be provided in preplanning period. This KPI is also related with network configuration and traffic distribution. None Cells, in which ratio of actual traffic / desired traffic<1
2.2 Accessibility
2.2.1 Random Access Success rate
KPI KPI reference value Indicator definition V2 Counter formula V3 V6.1 Notes V6.0 (C11626+C11696-C11697)*100%/(C11625+C11696) (C100030004+C100030010+C100030038-(C100030011 *100%/(C100030003+C100030010+C100030038) (C900060004+C900060010+C900060038-(C900060011 *100%/(C900060003+C900060010+C900060038) +C900060039)) +C100030039)) Random Access Success rate Suggested value Risk value 99% 99.5%
Random access success times *100% / Random access call attempt times
SDCCH congestion rate shall be taken into consideration, because of its direct influence on this KPI.
This is mainly to evaluate the fitness between preplanned network and actual network. It will be affected by the following factors: Notes 1. the fitness between network desired capacity and actual capacity; fitness between preset traffic model and the actual one; 2. burst traffic; 3. lack of SDCCH capacity; 4. internal & external interference; 5. preplanned redundancy rate; 6. operators special requests; 7. FR and HR
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 5Total 45 pages
Secret Secret
used as signaling channel.
V2
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 6Total 45 pages
Secret Secret
C900060021+C900060032+C900060044+C900060048))
This is mainly to compare performance of preplanned network with that of actual network. The prerequisite is that Notes precise traffic modeltraffic dispersionnetwork capacity and redundancy rate shall be provided in preplanning period. This KPI is in direct tie-in constraint with SDCCH blocking rate and TCH blocking rate.
SDCCH call drop total*100%/(random access call attempts + voice channel call attempts total (handover excluded)signalling) V2 V3 V6.0 V6.1 C11605*100%/(C11625+C11696) C100030053*100%/(C100030003+C100030010+C100030038) C900060053*100%/(C900060003+C900060010+C900060038)
C11605 accumulates SDCCH call drops resulted from all reasons, including radio link failure LAPD link breakshandover failure, etc., but except the following four reasons: lost connection with SMM, forced OMC-R disconnection, forced disconnection by other calls, and other causes. If Notes we compute according to the formula above, SDCCH call drop rate will be rather low. If counter 116135 is included (BSS call drop times on SDCCH), the suggested KPI value is 0.6%, and suggested risk value 0.4%. C100030053/C900060053 is used to find out the frequency of call drops resulted from radio link failureLAPD link breakshandover failure. This KPI value is relatively lower.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 7Total 45 pages
Secret Secret
Risk value Indicator definition failure times) V2 C11657*100%/(C11657+C11658) (C100030017+C100030028+C100030036+C100030235 + C100030199+C100030210)*100%/(C100030017+C100030028 V6.0 Counter formula V3 +C100030036+C100030018+C100030029+C100030037+ C100030235+C100030199+C100030210+C100030200+ C100030211+C100030135) (C900060017+C900060028+C900060036+C900060199+ C900060210+C900060235) *100%/(C900060018+C900060029+ V6.1 C900060037+C900060200+C900060211+C900060135+ C900060017+C900060028+C900060036+C900060199+ C900060210+C900060235) Notes This KPI is directly related to radio environment frequency carrier problems assignment methodsapplication of data service, etc.. 99%
TCH assignment success times*100/( TCH assignment success times +TCH assignment
(1-SDCCH blocking rate)* (1-SDCCH assignment failure rate)*(1-SDCCH drop rate)*(1TCH blocking rate)*(1-TCH assignment failure rate) The definition of this CSSR is relatively strict because too many factors are included (LACSMS, etc.). The definition is not rational from the view of voice call, so that the CSSR used in MOTO excluded SMS factor. Handover is excluded.
93% 95%
((MOC success + MTC success/ (MOC attempt + MTC attempt)) *(1-SDCCH drop rate)*(1-TCH blocking rate)*(1-TCH assignment failure rate) According to common understanding, CSSR is mostly applied to inspect the establishment of voice call. It is rather reasonable to exclude LAC and SMS. Handover is excluded. Suggested value Risk value 97% 95%
((MOC success + MTC success/ (MOC attempt + MTC attempt)) *(1-SDCCH drop rate)*(1-TCH blocking rate)*(1-TCH assignment failure rate) The CSSR is mainly measured in the view of congestion. It is similar to Hutchisons definition of CSSR. Handover excluded. CSSR is usually obtained from the combination of original KPI. Different manufacturers have different definitions for CSSR. The formula above is
Explanation
regarded too strict. Some may define it to be TCH assignment success rate or radio access completion rate. We need to make clear the formula with customers. It is regarded a
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 8Total 45 pages
Secret Secret
favorable way to exclude location update indicators when we define this KPI.
2.3 Mobility
2.3.1 Handover success rate
KPI Handover success rate V2 Suggested value Risk value Suggested value Risk value 94% 96% 95% 97%
KPI value
reference
V3
Indicator definition
Handover success times in busy hours*100% / handover request times in busy hours (C11618+C11669-C11675)*100%/(C11617+C11668V2 C11675) (C100030098+C100030102+C100030120+C100030094+ C100030096)*100 V6.0 C100030214+ +C100030095) V3 (C900060098+C900060102+C900060120+C900060094+ C900060096) V6.1 *100%/(C900060097+C900060213+C900060214+ C900060215+C900060099+C900060100+C900060101+ C900060216+C900060119+C900060093+C900060095) This formula includes both incoming and outgoing handovers, while some operators may take into consideration only one of them. As for V3 equipment, 1-2% can be appropriately increased %/ (C100030097+C100030213+ C100030215+C100030099+C100030100
Counter formula
Notes
according to real situations. This KPI can differ a lot from each other due to different types of network. In many presales bidding, the operators require a handover success rate of around 98%, which involves high risks to be reached, though several networks have reached 98%.
2.4 Reliability
2.4.1 TCH Call Drop Rate
KPI TCH drop rate / Call drop rate V2 Suggested value Risk value Suggested value Risk value 1.0% 0.7% 0.7 0.5
V3
Indicator definition
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 9Total 45 pages
Secret Secret
C11615*100%/(C11611-C11698-(C11612+C11656+C11658+ C11617-C11618-C11699-C116103-C116133)) V6. 0 V3 V6. 1 (C100030054+C100030055)*100%/(C100030028+ C100030199+C100030210+ C100030098+C100030102+C100030120) (C900060054+C900060055) *100%/(C900060028+C900060036+ C900060199+C900060210+C900060098+C900060102+ C900060120) C100030036+
V2 Counter formula
Voice channel call drop rate falls into two types: one is with handover included; the other is with handover excluded. The formula above takes handover in it. If handover is not included, the indicator Notes will be about 0.2% higher than the one with handover included. This discrepancy is not universally definite yet, for it can be bigger in dual-band network. Generally, call drop rate of V3 equipment is a bit lower than the one of V2. This KPI differs a lot in varied types of network. The discrepancy can reach 0.3%.
KPI value
reference
V3
Indicator definition
Traffic /TCH call drop times*60 V2 (C11616/3600)*60/C11615 V6.0 V6.1 ((C100030129+C100030127)/3600)*60/ (C100030054+C100030055) ((C900060129+C900060127)/3600)*60/(C900060054+C900060055)
Counter formula V3
Notes
This KPI of different networks varies to a large extent. Generally, its difficult for to reach this KPIs standards, if the call drop (with handover included) rate is higher than 0.5%.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 10Total 45 pages
Notes
None
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 11Total 45 pages
Secret Secret
Obtain from test the time duration from starting call to ringing out
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 12Total 45 pages
Secret Secret
Illustration of coverage
85% sample
of
the values
85% sample
of
the values
85% sample
of
the values
85% sample
of
the values
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 13Total 45 pages
Secret Secret
requirement should have a value equal to or higher than 75 dBm. Influencing factors should have a value equal to or higher than 80 dBm. should have a value equal to or higher than 85 dBm. should have a value equal to or higher than 90 dBm.
1. landform; 2. inconsistency between project implementation and project plans; 3. test equipment; 4. test routes Detailed coverage requirements (as shown above) are delivered in the bid documents. Estimation of network dimension is also based on the requirements. Nowadays, requirements for network coverage are higher, so the accordingly estimated number of sites will meet the needs. If the network dimension has to be reduced due to market strategies needs, this KPI will cause problems in the acceptance inspection. Further more, we need to make sure that our customers are aware of regional assignmentDT routes and test methods. Coverage KPI is one of the most important KPIs, because it directly reflects not only our radio network planning ability and the performance of our equipment, but also our core competitive strength in network planning. Due to lack of space, there is no more illustration of coverage KPI hereafter in this guide.
Notes
3.4.1 SQI
KPI KPI value Indicator definition This KPI is for inspecting the sound distortion degree resulted from incapable link transmission. The formula is Ericssons patent. It takes into consideration many factors, including BERFERhandover timeshandover conditionsDTX activated (or not) Notes and the encoder in use, etc., to finally get the SQI. There are different SQI top limits for different types of encoders. For half-rate encoder, the SQI top limit is 19; for full-rate encoder, 21; for enhanced full-rate encoder, 30. SQI ranges from 18 to 30. SQI value will increase when EFR is applied. reference SQI Suggested value Risk value TEMS test value 95% sample >=25 95% sample >=18
3.4.2 MOS
KPI KPI value Indicator definition Notes The commonly used MOS assessment method consists of subjective and objective scoring parts. In the subjective part, engineers subjectively compare the original voice material and the deteriorated voice material according to ITU-T Suggestion book P.800 and P.830, and give their MOS scores respectively, then average the scores from different engineers. While in the objective part, PESQ method from ITU-T Suggestion book P.862 is employed, that is special equipment or software (e.g. the VQT test equipment of Agilent) is used to proceed Test value reference Mean Opinion Score ( MOS) Refer to the form attached bellow.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 14Total 45 pages
Secret Secret
the assessment. In MOS, 0 means the worst quality, 5 stands for the highest. Standard PCM quality shall reach around 4.4. The MOS values for commonly used coding formats are as follows: G.711MOS 4.8 G.721G.723G.726MOS 4.2 G.728 MOS4.2 G.729 MOS4.2 G.723.15.3MOS3.5 G.723.16.3MOS3.98 MOS can be affected greatly by the performance of test equipment. The test results from different manufacturers are different. Currently, the test equipment from Actena performs the best.
Average MOS(Including FR &HR Voice version combination MOS) Frequency hopping 100% FR 100% EFR 100% AMR FR 60% HR (+EFR) 30% HR (+EFR) 10% HR (+EFR) 60% AMR HR (+AMR FR) 30% AMR HR (+AMR FR) 10% AMR HR (+AMR FR) 3.04 3.63 3.56 No hopping 3.29 3.57 3.54
Secret Secret
can be used to compute voice samples MOS-LQO value (Mean Opinion ScoreListening Quality Objective). The computed result and the subjective scoring result are basically the same. PESQ compares the output signal (collected after the signal is transmitted through equipment) and the reference signal, then works out the 5 difference. In the ordinary course of event, the bigger the difference between the output signal and the reference signal, the smaller the reference value of MOS will be. PESQ is testified to be a reliableeconomic and fast measuring tool for voice encoder. Compare PESQs predicted value and the result from subjective scoring method, we can see the correlation between the two values is 0.935(refer to 10 P862E.docfor correlation factors). For 22 kinds of ITU benchmark tests, the absolute error is within 0.5 MOS, 69.2% among which is within 0.25 MOS. It is worth our attention that PESQ value ranges from 0.5 to 4.5, while MOS value ranges from 1 to 5.
15
Fig 1 MOS/PESQ Correlation Diagram
At the present time, there are two types of QOS test method-subjective and objective methods. MOS subjective test method is defined in ITU-T P.800. The objective test methods includes PSQMPSQM+ and PESQ etc.. ITU-T P.862 20 PESQ is the method currently recommended by ITU to test end-to-end
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 16Total 45 pages
Secret Secret
network voice quality. It would be ideal if MOS and PESQ are simultaneously supported in the QOS test.
Form 1 Definition of MOS Values
Level Excellent Good Fair Poor Bad MOS 5 4 3 2 1
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 17Total 45 pages
Secret Secret
Statistical value GPRS UL signalling TBF link setup times + GPRS UL data TBF link setup times +EGPRS UL signalling TBF link setup times +EGPRS UL data TBF link setup times
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 18Total 45 pages
Secret Secret
*100/ UL packet channel request timesV3 V2 (C70042+C70055+C70069)*100%/C70040 (C100010025+C100010033+C100010026+C100010034) Counter formula V3 V6.1 V6.0 *100% /(C100010159 +C100010160+C100010161+C100010168 +C100010163+C100010164+ C100010165+C100010166) (C900040025+C900040033 +C900040026 +C900040034) *100%/ (C900040159+ C900040160 +C900040164 +C900040161+C900040168+C900040163 +C900040165+C900040166) This KPI reflects the request fulfilled status of PS uplink resources. Its TBF includes UL signaling TBFGPRS UL data TBF and EGPRS UL data TBF. UL TBF link setup success rate can reflect the UL PS quality of a cell. Its major influencing factors are as follows: Notes 1. 2. 3. 4. Interference: internal & external interference; Parameters setting: eg. errors in Timer and functional parameters Stability of equipment and transmission: eg. instant breakdown resulted from high error rate or unstable link; Usability of data service functions.
Statistical value GPRS UL TBF NACK blocks *100% / valid data statistics received by GPRS UL TBF
This KPI reveals UL GPRS RLC retransmission rate, which is the most important indicator to reflect the sensibility of GPRS UL service and to show whether instability congestion and interference exist in the system. Its major influencing factors are as Notes follows: 1. 2. Interference: internal & external interference; Stability of equipment and transmission: eg. instant breakdown resulted from high error rate or unstable link;
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 19Total 45 pages
This KPI reveals DL GPRS RLC retransmission rate, which is the most important indicator to reflect the sensibility of GPRS DL service and to show whether instability congestion and interference exist in the system. Its major influencing factors are as Notes follows: 1. 2. Interference: internal & external interference; Stability of equipment and transmission: eg. instant breakdown resulted from high error rate or unstable link;
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 20Total 45 pages
Secret Secret
KPI reference value Indicatory type Indicator definition V2 Counter formula V3 V6.1 Notes V6.0 C70082*100%/ C70081 C100010073*100%/C100010072 C900040073*100%/C900040072 EGPRS DL TBF NACK blocks *100% / valid data statistics received by EGPRS DL TBF Suggested value Risk value Statistical value 5% 10%
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 21Total 45 pages
Secret Secret
Sum of attach success duration/ Attach success times. It is the Indicator definition Indicatory type Counter formula Notes Normally, this KPI is the same as the CQT value. Authentication included. duration from the mobile sends out the first Attach Request to it receives the Attach Accept signaling. It is required to perform the test ten times at each test site. Test value None
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 22Total 45 pages
Secret Secret
Risk value Indicator definition 98%
PDP activation success times/attempt times 100. PDP activation success means that the mobile phone receives the Activate PDP Context Accept signaling 15 seconds after it sends the Activate PDP Context Request.
Test None Normally, this KPI is the same as the CQT value. It is required to perform the PDP activation test ten times at each test site. Test methods for GPRS and EDGE are the same.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 23Total 45 pages
Secret Secret
Suggested value Risk value 1 sec 0.6 sec
EDGE Indicator definition Indicatory type Counter formula Notes Test value None
Authentication is included in this KPI. The Ping package size will have influence on the test result. Generally, the smaller the Ping package is, the better the test result will be. It is sometimes referred as Round Trip Time.
5.2 Mobility
5.2.1 Routing Area Update Successful rate
KPI KPI reference Indicator definition Indicatory type Counter formula Notes test None Usually it can be statistical value. It also can be obtained from DT too. Routing Area Update Success rate Suggested value Risk value 90% 95%
Actual download data quantity (byte)/actual download duration Test value None Normally, this KPI is the CQT value. Class 4 mobile. Size of the downloaded document will affect the test result. Usually, the smaller the document size is, the better the test result will be. It is related to the number of PS channelsterminal typeencoding method and test software, etc.. The test result can also be affected
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 24Total 45 pages
Secret Secret
by the adoption of encoding method for CS3 and CS4. 3static time slots are configured.
EDGE
Actual upload data quantity (byte)/actual upload duration Test value None Normally, this KPI is the same as the CQT value. Class 4 mobile. Size of the uploaded document will affect the test result. Usually, the smaller the document size is, the better the test result will be. It is related to the number of PS channelsterminal typeencoding method and test software, etc.. The test result can also be affected by the adoption of encoding method for CS3 and CS4. This KPI is the result of single time slot.
5.3.3 Throughput
Mobile phone with multi-timeslot Level 8; single Preconditions subscriber; 4 DL static signaling channels, 1 UL static signaling channel Indicators (EN) GPRS: Max data throughput-DL GPRS: Max data throughput-UL GPRS: Average data throughput-DL GPRS: Average data throughput-UL EDGE: Max data throughput-DL EDGE: Max data throughput-UL EDGE: Average data throughput-DL EDGE: Average data throughput-UL Indicators (CN) GPRS DL maximum transmission rate GPRS UL maximum transmission rate GPRS DL average transmission rate GPRS UL average transmission rate EDGE DL maximum transmission rate EDGE UL maximum transmission rate EDGE DL average transmission rate EDGE UL average transmission rate Suggested value 9.8kbyte/s 2.4kbyte/s 5kbyte/s 1.3kbyte/s 26.5kbyte/s 7kbyte/s 14kbyte/s 3.5kbyte/s
5.4 Reliability
5.4.1 PDTCH Drop Rate
KPI KPI value Indicatory type reference PDTCH Drop Rate Suggested value Risk value DT test value 5% 2%
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 25Total 45 pages
Secret Secret
Indicator definition Counter formula Notes None PDTCH drop times/ FTP download attempt times *100%
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 26Total 45 pages
Secret Secret
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 27Total 45 pages
Secret Secret
Customer attach establishment duration *100% / customer number of attach establishment success Statistics Attach establishment time refers to the time duration from the receipt of GPRS ATTACH REQUEST by MS to the delivery of ATTACH ACCEPT by SGSN. If the attach time exceeds the timer, the attach duration is only computed within the attach accepted period.
Notes
success times of PDP context activation started by MS*100%/ attempt times of PDP context activation started by MS Indicator definition The attempt times of PDP context activation started by MS is the times GGSN received Create PDP Context Request from SGSN. The success times of PDP context activation started by MS is the statistic of Create PDP Context Response sent from GGSN to SGSN when Cause value is Request Accepted. Indicatory type Counter formula Notes The performance evaluation in China Mobile in 2003 (success times of PDP context activation started by GPRS terminal/ attempt times of PDP context activation started by GPRS) results are as follows: Excellency index: 94% Reach-the-mark index: 90% Lowest Threshold index: 85% Test or statistics Statistics at GGSN side
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 28Total 45 pages
Secret Secret
Indicatory type Counter formula Notes From statistical value or DT. Statistics at SGSN side, provided by GPRS core network Update success rate at routing area crossing SGSN is lower than that at the SGSN routing area.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 29Total 45 pages
Secret Secret
Test state
Test serial 1 2 3 4 5 6 7 8 9 10 11
FTP type
FTP Kbyte
size
Download speed (Kbyte/s) 8.99 8.10 2.97 7.21 9.05 8.66 15.97 13.34 15.81 15.74 14.33
Download speed (kbps) 71.93 64.80 23.78 57.72 72.42 69.29 127.73 106.69 126.51 125.88 114.67
Pre-optimization Pre-optimization Pre-optimization Pre-optimization Pre-optimization Pre-optimization Pre-optimization Pre-optimization Pre-optimization Pre-optimization Pre-optimization Ping Ping interval package size byte 0 5 2 2 5 32 32 100 500 500 839 816 571 839 862 1
Download Download Download Download Download Download Download Download Download Download Download
102.381 48.18 204.8 2188.817 102.381 102.381 512.304 512.304 512.304 512.304 512.304
23.98 21.60 7.93 19.24 24.14 23.10 42.58 35.56 42.17 41.96 38.22
Ping timelagms 2 3 8 9 10
4 61 2 77 9 75 6 82 1 75
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 30Total 45 pages
Secret Secret
5 2 2 2 800 1000 1200 959 115 7 107 8 8 840 910 115 869 880 897 86 9 90 7 95 7 7 8 101 845 102 823 918 959 781 105 4 119 1 0 883 881 102 8 860 108 2 101 2 7 104 846 103 865 1010 1002 100% 100% 100%
Comments on case:
1. When C/I is in good condition, equipment can reach above 40kbps. 5 2. The Ping success rate and Ping timelag can vary very much due to different Ping intervals different Ping package sizes and different Ping servers. the EDGE download speed of current
Test Result
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 31Total 45 pages
Secret Secret
Test Branch office Test sites document size KB Fenghuang County Jishou prefectural government office building Cell 1 Fenghuang County Jishou prefectural government office building Cell 2 Fenghuang County Jishou prefectural government office building Cell 3 Jishou Old district Cell 1 1000 -49 2 Basketball field of Jiandaoping Primary School Basketball field of Jiandaoping Primary School Kindergarten next to Jiandaoping Primary School Opposite Wanmingta River Opposite Wanmingta River Square in front of TianXiaFengHuang Hotel Square in front of TianXiaFengHuang Hotel Gate of Linlinlou Drug Store Gate of Linlinlou Drug Store Gate of Linlinlou Drug Store Lobby of Xinhua Hotel Lobby of Xinhua Hotel Parking lot of Xinhua Hotel Lane, west to ChinaMobile business hall 197.74 1000 -51 2 Opposite MingDu Hotel 197.09 1000 -40 2 Lobby of MingDu Hotel 192.6 1000 -44 2 Gate of Prefectural government office 201.19 181.3 Test electrical level dBm Download times Test location RLC average rate Kbps RLC average rate
Kbps
198.98
206.26
201.79
Jishou
1000
-32
207.29
193.91
Old district Cell 3 Rainbow Bridge Cell 1 Rainbow Bridge Cell 2 TianXia Feng Huang Cell 1
2 2 2 2
196.38 191.1
181.36
186.55
TianXia Feng Huang Cell 2 BaoQing market Cell 1 BaoQing market Cell 2 BaoQing market Cell 3 Xinhua Bookstore Cell 1 Xinhua Bookstore Cell 2 Xinhua Bookstore Cell 3 Comprehensive Building Cell 1 Comprehensive Building Cell 2 Comprehensive Building Cell 2 Comprehensive Building Cell 1800M1 Comprehensive Building
2 2 2 2 2 2 2 2
206.26
198.02
185.34
180.09 191.5
188.38
190.16
157.44
Jishou
1000
-45
185.46
187.98
Jishou
1000
-60
184.07
184.45
Jishou Jishou
1000 1000
-50 -62
2 2
193.98 176.22
184.51
185.21
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 32Total 45 pages
Secret Secret
Cell 1800M2 Jishou Jishou Jishou Jishou Jishou Jishou Telecom Bureau Cell 1 Telecom Bureau Cell 2 Telecom Bureau Cell 3 Telecom Bureau Cell 1800M1 Telecom Bureau Cell 1800M2 Telecom Bureau Cell 1800M3 1000 1000 1000 1000 1000 1000 -51 -64 -57 -59 -73 -84 2 2 2 2 2 2 Business Hall of Telecom Bureau Business Hall of Telecom Bureau Backdoor of Supermarket (next to Telecom Bureau) Gate of furniture store (next to Telecom Bureau) Business Hall of Telecom Bureau Business Hall of Telecom Bureau 180.75 171.85 182.7 105.44 181.85 159.20
197.88
199.19
179.49
106.48
185.85
166.92
Comments on case:
1. When C/I is in good condition, the average EDGE download speed can reach above 180kbps with a peak value of above 200kbps; 2. This test is the CQT. Cell reselection and RAC renewal can be neglected, that 5 is they are not necessary to the test conditions.
receives the Attach Accept signaling 15 seconds after it sends the Attach Request.
means the
the mobile
<3s
Attach receives
Request GPRS
attach accept signaling. 3 PDP activation success rate PDP activation success means that the mobile phone receives the Activate PDP context Accept signaling 15 seconds after it sends the Activate PDP Context Request. 4 PDP time average PDP average activation time means the duration from the delivery Context receipt phone. 5 WAP website WAP website log-in success WAP website log-in success >95% of of Activate to Activate PDP the PDP Request PDP average activation time sum success of PDP activation duration/PDP <3s activation PDP activation success >95% ratePDP activation success times/attempt times100
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 33Total 45 pages
Secret Secret
log-in success rate means that the homepage rate WAP website log-in
fully and correctly displays within 60seconds after the mobile phone started the WAP website log-in request.
WAP
WAP homepage displaying time refers to the duration from the the WAP website fully log-in and request is sent out until homepage correctly displays.
WAP
<10s
WAP
webpage
WAP webpage refresh success means webpage that can the be selected refreshed
>95%
times100 8 Ping success rate ping success times/ping attempt times100 9 Ping average timelagsum of each Ping success duration/ping success times 10 FTP speed download FTP download speed actual download time(sec)9 11 FTP speed upload FTP upload speed actual >20kbps upload data quantityByte /actual upload time(sec) 9 data quantity download Byte /actual >80kbps <2s timelag >95%
Comments on case:
1. Requirements of operation and maintenance KPI are not as strict as those of acceptance check KPI. 2. Usually these KPI requirements can be satisfied.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 34Total 45 pages
Secret Secret
attempt Number 2 TCH Drop Ratio % 2 2.2 Base on system performance report of OMCR, Indication Definition: TCH Drop Ratio = TCH call drops/ Seized TCHs (include handover ) 3 Call Ratio Success % 98 97 Base on system performance report of OMCR, failure rate 4 SDCCH Blocking Ratio % 0.5 0.75 Base on system performance report of OMCR, Indication Definition: SDCCH Blocking Ratio=SDCCH overflows/ SDCCH call attempts 5 TCH Blocking Ratio % 2 2.5 Base on system performance report of OMCR, Indication Definition: TCH Blocking Ratio=TCH overflows (include handover)/ TCH call attempts (include handover) 6 TCH Assignment Success Ratio % 98 97 Base on system performance report of OMCR, Indication Definition: TCH assignment Success Ratio=Number of TCH assignment successful / Number of TCH call attempts (include handover) 7 GPRS per TBF 8 EDGE per TBF 9 Attach Success Rate (GPRS/ EDGE) 10 Handover Success Ratio 11 Call Blocking Ratio % < 2 for 98% of all GSM cells 12 Call Success (Drive Test) 13 Call Rate Test) 14 Speech MOS >= 3 for >= 3 for Depend on the DT route and method Drop (Drive % 2 3 Depend on the DT route and method Setup Rate % 97 < 2 for 90% of all GSM cells 90 Depend on the DT route and method Based on OMC-R statistics % 94 93 % 99 97 24-Hour measurement average Kbps 45 40 24-Hour measurement average Kbps 28 25 24-Hour measurement Indication Definition: Call Success Rate = 1 TCH assignment
DL throughout
DL throughout
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 35Total 45 pages
Secret Secret
Quality Test) (Drive 95% samples 15 Coverage (Drive Test) dBm >= -85 for 95% samples 16 UL Throughput 17 DL Throughput 18 BSC/RNC down time (unscheduled) 19 TRX/channel element failure (unscheduled) 20 SDCCH Mean Time 21 Radio Access Success Ratio 22 Worst GOS Cells Less than Less than per BSC/RNC / BH of a week % >99.5 >99 per BSC/RNC / BH of a week Holding s <2 <3 per BSC/RNC / BH of a week % < 1 of all active TRX < 2 of all active TRX per BSC/RNC / week % >99.999 (< 6 sec) >99.999 (< 6 sec) per BSC/RNC / week Kbps >30 >30 Based on DT Kbps >10 of measurement of 90% samples >= -85 for 90% samples >10 Based on DT of measurement Depend on the DT route and method of
measurement
measurement
1.5% cells of Network with GOS 5% or above >95 <1 per BSC/RNC / BH of a week per BSC/RNC / BH of a week
23 24
% %
>97 <1
Comments on case:
1. This KPI comprises statistical KPItesting KPI and equipment performance KPI, which features detailed classification and all-round evaluation; 2. Requirements for KPIs like GPRS average DL throughout per TBF and EDGE average 5 DL throughout per TBF are higher. 3. Requirements for evaluation of CS voice quality are higher.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 36Total 45 pages
Secret Secret
interval % of outdoor area with 2G signal level > 104 dBm Call Setup Success Rate (exclude blocking)
>90%
>97%
>99%
95%
>98%
>99%
per city / BH of a week (drive test monthly) per city / BH of week. With condition that Core Network is achived the same value of Level 1 and Level 2 per city / BH of a week (drive test monthly) per city / BH of a week (drive test monthly) per city / BH of a week (drive test monthly),per city / BH of week. With condition that Core Network is achived the same value of Level 1 and Level 2 Average value per city / BH of a week
NMS
>90%
>95%
>97%
pre-defined locations and Drive test route/NMS Pre Defined Drive Test Route/NMS
<0.5%
<0.5%
<1%
GSM: <3%
GSM: <2%
GSM: <1%
>90%
>97%
>97%
SQI (TEMS) Call setup delay (PSTN to MS, time from number dialed to ring) Call setup delay (MS to PSTN, time from send pressed to ring)
>20(95%)
> 25(95%)
> 25(95%)
10
<6s
< 6s
<6s
11
<5s
< 5s
<5s
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 37Total 45 pages
Secret Secret
Call Setup Successful Rate SDCCH Access Success SDCCH Blocking Rate TCH Blocking Rate per BSC/RNC / BH of a week per BSC/RNC / BH of a week per BSC/RNC / BH of a week (drive test monthly) per BSC/RNC / BH of a week (drive test monthly) per BSC/RNC / BH of a week (drive test monthly) per BSC/RNC / BH of a week (drive test monthly) With condition that Core Network is achived the same value of Level 1 and Level 2 per BSC/RNC / BH of a week (drive test monthly) per BSC/RNC / BH of a week (drive test monthly) per city / BH of a week per city / BH of a week
12
NMS
>95%
>98%
>99%
13
NMS Pre Defined Drive Test Route/NMS Pre Defined Drive Test Route/NMS Pre Defined Drive Test Route/NMS
>99%
>99%
>99.5%
15
<0.5%
<0.5%
<1%
16
<0.5%
<0.5%
<1%
18
<5%
<2%
<1.5%
19
20
RXQUAL 0 to 3
Pre Defined Drive Test Route/NMS Pre Defined Drive Test Route/NMS NMS
>80%
>85%
>90%
21
AMR voice FER Successful PS Attachment Successful PDP context activation Data session Setup Success Rate (PDP context already exist)
<1%
<1%
< 1%;
23
>98%
>98%
>98%
24
NMS
>98%
>98%
>98%
25
>95%
>98%
>98%
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 38Total 45 pages
Secret Secret
26
GPRS>30kbps
27
Moving MS FTP upload 1MB file Stationary MS near site, FTP download 1MB file Stationary GPRS MS in cell border, FTP download 1MB file Stationary UMTS MS in cell border (SHO), FTP download 1MB file PDTCH blocking rate 60 second data transfer drop Rate (MS moving) Routing Area Update Successful rate MS initiated service setup time MS roundtrip Ping time (20 ping averaged) PDP update
GPRS>8kbps
28
Pre-defined locations
29
Pre-defined locations
30
Pre-defined locations
N/A
31
NMS
<0.5%
<0.5%
<1%;
32
<3%
<2%
<1.5%
33
NMS
> 95%
> 95%
>97%
34
Pre Defined Drive Test Route Pre Defined Drive Test Route NMS
<5s
<5s
<5s
35
<1
<1
<0.8s
36
>98%
>98%
>99%
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 39Total 45 pages
Secret Secret
success rate
Comments on case:
1. This OM KPI just got some of the indicators, excluding the weighing of all KPIs and 2/3G interoperate KPI, etc.. Due to market factors, the OM KPI of this network is set unreasonable, and there is risk in some of the KPIs. 5 2. The evaluation of this KPI falls into three grades. The requirements for handover success rate and paging success rate are very high, which are difficult to reach. But the attached prerequisites can lower the risk to some extent. Besides, the definition of handover success rate varies with different manufacturers, which we need to make clear with customers in technical 10 clarification. 3. The requirements for data service and voice KPI are higher.
Comments on case
15 1. The evaluation scale of KPI is relatively large, mainly dealing with statistical KPI and testing KPI; 2. It ranges from 2G to 3G network. Its service includes voiceGPRS and 3G data service, etc.. 3. 20 The division of KPI evaluation scenes and most KPI values are relatively reasonable.
This document contains proprietary information of ZTE Corporation and is not to be disclosed or used except in accordance with applicable agreements. Page 40Total 45 pages