ASPIRATION

Acceptance Criteria
Rev3.a

By Tarek Nasr

Jul 19 2019

Date Revision Author Comments

21/1/2019 Rev1 Tarek Nasr Draft
23/6/2019 Rev2 Tarek Nasr MIMO KPI Update, 5G Beamforming and

1
 TDD for smart phones
10/7/2019 Rev3 Osman Taher Higher Order MIMO KPI
18/7/2019 Rev3.A Tarek Nasr Editing some comments, Fixed Ranks

2
Table of Contents
INTRODUCTION...........................................................................................................................................5
PROJECT1: LTE.............................................................................................................................................6
PART 1: NEW SITE...................................................................................................................................6
1.1 PHYSICAL SITE INSPECTION...........................................................................................................6
1.2 PERFORMANCE.............................................................................................................................6
1.2.1 LTE TECHNOLOGY SINGLE CARRIER NEW SITE.......................................................................7
1.2.1.1 OSS KPI............................................................................................................................7
1.2.1.2 Static or Fixed Testing (SSD)............................................................................................8
1.2.1.3 Drive Test Based KPI (SSV)..............................................................................................9
1.2.2 MULTI CARRIER NEW LTE SITE.............................................................................................11
1.2.2.1 OSS KPI..........................................................................................................................11
1.2.2.2 Static or Fixed Testing KPI.............................................................................................12
1.2.3 TDD Testing..........................................................................................................................14
1.2.3.1 OSS KPI..........................................................................................................................14
1.2.3.2 TDD Fixed Testing..........................................................................................................15
1.2.2.3 Drive Testing KPI...........................................................................................................16
1.2.4 TDD Testing for Smart Phones.............................................................................................17
1.2.4.1 OSS KPI..........................................................................................................................17
1.2.4.2 Fixed testing..................................................................................................................18
1.2.4.3 Drive Testing.................................................................................................................19
1.2.5 TESTING OF NEW SITE WITH HIGHER ORDER MIMO SOLUTION.........................................20
1.2.5.1 OSS KPI..........................................................................................................................20
1.2.5.2 Fixed Testing.................................................................................................................21
1.2.5.3 Drive Testing.................................................................................................................22
PART 2 CO-LOCATION...........................................................................................................................23
2.1 PHYSICAL INSPECTION...........................................................................................................23
2.2 PERFORMANCE......................................................................................................................23
2.2.1 Single LTE Carrier Upgrade...................................................................................................24
2.2.1.1 OSS KPI..........................................................................................................................24
2.2.1.2 Fixed Testing.................................................................................................................25
2.2.1.3 Drive Testing.................................................................................................................26
2.2.2 Multi LTE Carrier Upgrade....................................................................................................27

3
 2.2.2.1 OSS KPI..........................................................................................................................27
2.2.2.2 Static Testing.................................................................................................................28
2.2.2.3 Drive Testing.................................................................................................................29
2.2.3 TDD for Smart Phone Carrier Upgrade................................................................................30
2.2.4 Site Upgrade with Higher Order MIMO...............................................................................31
3 REPORT..............................................................................................................................................31
PROJECT 2: 5G...........................................................................................................................................32
2.1 PHYSICAL INSPECTION....................................................................................................................32
2.2 PERFORMANCE...............................................................................................................................33
2.2.1 OSS KPI..............................................................................................................................33
2.2.2 Fixed Testing............................................................................................................................34
2.2.3 Mobility Testing................................................................................................................35
3.0 FINAL REPORT.....................................................................................................................................39
4 ASPIRATION TESTING........................................................................................................................40
5 REGIONAL ASPIRATION ACCEPTANCE..............................................................................................40
6 CONCLUSION.....................................................................................................................................41

4
ASPIRATION Acceptance Criteria
INTRODUCTION

ASPIRATION is a STC initiative to improve customer experience. The target of this

project is to increase user speed from the current experience of about 38 Mbps to
50 Mbps in the last quarter of 2019 and 65 Mbps for entire 2020 all over the KSA.
In order to achieve the above targets, STC is going to upgrade existing sites with
features such as 4x4, and massive MIMO, in addition to new carriers on some
sites and new sites if required. All those solutions are to achieve the 2019 and
2020 goals.

This acceptance is to ensure that all new solutions are going to perform
accordingly and the most important is that will be delivering throughput targets
for 2019 and 2020 as stipulated in the Aspiration agreement.
Regardless of the achieved KPIs under this acceptance document, vendors are
liable for all penalties/corrective actions if the targeted footprint throughput
levels are not achieved by the end of the measurement period for 2019 and
2020.
Vendor’s failure to achieve the desired throughputs as stipulated in contract
and SoC article 4.2.1 will be dealt with as per the contract agreement.

This document is divided into 2 Aspiration Project:

1. LTE project
2. 5G coverage project
The LTE project consisted of 4 separate acceptance criteria:
1. New LTE site acceptance
2. New LTE carrier addition on existing site acceptance.
3. New antenna configuration such as 4x4 or Massive MIMO
4. Aspiration Acceptance
The 5G project consisted of new 5G site acceptance only

5
There are no upgrades for 2G or 3G technology included in this project. STC
Aspiration acceptance will require the passing functionality and performance of
all layers and technologies on new and upgraded sites.
This document is intended for engineer and technical staff that are going to be
involved in the testing and launching of new sites or upgrade of existing sites for
the ASPIRATION projects.

PROJECT1: LTE
PART 1: NEW SITE

1.1 PHYSICAL SITE INSPECTION

Part of the acceptance of a new site is passing the physical inspection. This
inspection will include: antenna height, antennas and RRUs models, and antenna
azimuth and down tilt if mechanical tilt is present. Below is a table that need to
be filled:
Table 1
Physical Inspection Type Proposed Actual Pass/Fail
Antenna Height
Antenna Models
Antenna Azimuth
Antenna Mec DT
RRU/BBU Model
Table 1: Physical inspection verification

1.2 PERFORMANCE

Note: All new sites and layers must PASS the functionality testing.

Note: Any KPI which is an indicator for certain feature or service and it is not
authorized to be activated, that KPI will not be required.
The below clauses should be considered for the WBB projects sites where there
no service affecting alarms are present:

6
 1. For the drop calls, if the number of drops is less than 10% and at the same time, it is
less than 10 drops per day, the site pass.
2. For the accessibility, if the number of attempts is more than 100 per day, the
accessibility passing rate values in the below table will be considered; otherwise, it is
passed.
3. For the HO, if the number of attempts is more than 300 per day, then the HO values in
the below tables will be considered to pass; otherwise, it is passed

1.2.1 LTE TECHNOLOGY SINGLE CARRIER NEW SITE

1.2.1.1 OSS KPI

Below is an OSS KPI table for a site with single LTE carrier (L700):
Name of KPI Value
Availability (%) >=99%
Accessibility (%) >=98%
Drop Calls Rate (%) <=2%
Intra Freq HO Success (X2, S1) (%) >=97%
Inter Freq HO Success (%) >=96%
IRAT Success Rate (%) >=96%
CSFB Success Rate (%) >=98%
Reselection Time xx
DL Traffic Volume (GB) xx
UL Traffic Volume (GB) xx
RANKs 1,2, 3 (IA), 4 (IA) % Distribution
Number of Users
Ave DL User throughput (Mbps)
Ave UL User throughput (Mbps)
DL Cell Throughput
UL Cell Throughput
Cell Max DL Throughput
2CC Inter Site CA Throughput DL (IA)
VoLTE Acc Succ Rate (IA) >=99%
VoLTE Retainability Succ Rate (IA) >=98%
SRVCC Succ Rate (IA) >=96%
DL PRB Utilization (%) xx
UL PRB Utilization (%) xx
UL Interference <=-100 dBm
RACH Succ Rate >=98%

7
 NB IoT Acc Succ Rate
NB IoT Retainability Rate
NPRACH Succ Rate >=92%
NB IoT DL Traffic
NB IoT UL Traffic
Table 2 LTE OSS KPI

Note: RANK3 and 4 percentage distribution are required when 4x4 antenna configuration is
deployed. There is no passing or failing thresholds for the RANKs distribution.

1.2.1.2 Static or Fixed Testing (SSD)

Below is a table for the KPIs that need to be collected during static test for 20
MHz channel:

Site Name Value

Sector Name Alpha, Beta or Gamma
PCI xx
Device Name and Model
Date
Test point Lat xx
Test point Long xx
EARFCN (DL) and (UL) xx
RSRP
RSRQ Xx dB
SINR
RANK >=2
MCS DL xx
MCS UL xx
Latency <35 msec
TX1
TX2
DL User Throughput (Mbps) >= 110
UL User Throughput (Mbps) >= 35
2CC CA DL User Throughput (Mbps) (IA) >=150
3CC CA DL User Throughput (Mbps) (IA) >=190
4CC CA DL User Throughput (Mbps) (IA) >=230

8
 VoLTE MO Succ Rate >=98%
VoLTE TM Succ Rate >=98%
SRVCC Succ Rate >=96%
CSFB >=98%
NB IoT Accessibility >=98%
NB IoT Retainability >=96%
NB IoT DL THP
NB IoT UL THP
Table 3: Site Static test KPI

Carrier Size Expected DL Throughput (Mbps)

2CC CA >150
3CC CA >190
4CC CA >230
Table 4: CA expected throughput
CA fixed testing with multiple CC should be done on 10% of sites in a cluster
Note: 256 QAM is going to be activated on all new sites. Therefore, SINR value at about 20 or
better will contribute to the 256 QAM modulation.

Table 10
Carrier Size Expected Throughput DL (Mbps) Expected UL Throughput (Mbps)
20 MHz >110 >35
15 MHz >90 >27
10 MHz >60 >18
Table 6 Throughput Table with 256 QAM Modulation

1.2.1.3 Drive Test Based KPI (SSV)

Note, All Max user throughput from drive test KPI has to be selected and averaged from the outer
loop

Below is a list of KPIs that need to be generated from drive testing:

KPI Name Events Count

PS Accessibility RRC Connection Estab SR >=98%

PS Retainability Drop Rate <=2%

9
 CS Accessibility Successes >=98%
CS Retainability Total Call Established >=98%
Complete >=98%
CSFB Succ Rate >=98%
VoLTE Accessibility (IA) Successes >=99 %
VoLTE Retainability (IA) >=98%
VoLTE MO Succ Rate >=98%
VoLTE MT Succ Rate >=98%
SRVCC Succ Rate >=96%
CS Call Setup Time (Sec) xx
CA DL Throughput (Mbps)
Ave Number of CC for CA
Ave throughput per CC
Latency xx
Time on LTE (%) xx
RANK1,2,3, 4 Distribution (%)
LTE Reselect Time (Sec) (CSFB) xx
Intra Freq HO Succ Rate >=97%
Inter Freq HO Succ Rate >=96%
IRAT Succ Rate xx >=92%
RACH Succ Rate >=94%
NB IoT Acc Succ Rate >=98%
NB IoT Retainability >=97%
NB IoT DL THP
NB IoT UL THP
NPRACH Succ Rate >=92%
Table 7 DT KPI

Note: In case the site is an island with single carrier, the only required mobility functional
tests are intra site and intra frequency and no inter eNodeB functional and carrier
aggregation testing is required.

Below is a table with expected throughput KPI for both 2x2 and 4x4 antennas
configurations (MIMO) during drive test for a FDD carriers that should be used as
a reference:

  2x2 MIMO 4x4 MIMO Proposed

RF Condition Ave DL User THP Ave UL User Max DL User Ave DL User Ave UL User Max DL User
Target (Mbps) THP Target THP (Mbps) THP Target THP Target THP (Mbps)

10
 (Mbps) (Mbps) (Mbps)

>75 @ 20 MHz >18 @ 20 >120 @ 20 MHz

>35 @ 20 MHz >15 @ 20 MHz >56 @ 20 MHz MHz
DLUT Quality High (Near Cell): >60 @ 15 MHz >90 @ 15 MHz
DL_SINR ≥ 15.5dB & >27 @ 15 MHz >12 @ 15 MHz >42 @ 15 MHz >14 @ 15 MHz
>40 @ 10 MHz >60 @ 10 MHz
DL_RSRP>=-85 dBm
>18 @ 10 MHz >8 @ 10 MHz >28 @ 10 MHz >10 @ 10
MHz

>10 @ 20 MHz >5 @ 20 MHz >20 @ 20 MHz >14 @ 20 MHz >6 @ 20 MHz >30 @ 20 MHz
DLUT Quality Low (Cell edge):
0dB ≤ DL_SINR < 7dB & >8 @ 15 MHz >4 @ 15 MHz >15 @ 15 MHz >11 @ 15 MHz >5 @ 15 MHz >23 @ 15 MHz
DL_RSRP>=-105dBm
>6 @ 10 MHz >3 @ 10 MHz >12 @ 10 MHz >7 @ 10 MHz >3 @ 10 MHz >15 @ 10 MHz

Table 8: Expected DT throughput KPI for FDD Carriers

1.2.2 MULTI CARRIER NEW LTE SITE

1.2.2.1 OSS KPI

Name of KPI F1 (700) Values for F2 Values for Fx
Availability (%) >=99% >=99% >=99%
Accessibility (%) >=98% >=98% >=98%
Drop Calls Rate (%) <=2% <=2% <=2%
Intra Freq HO Success (X2, S1) (%) >=97% >=97% >=97%
Inter Freq HO Success (%) >=96% >=96% >=96%
IRAT Success Rate (%) >=96% >=96% >=96%
CSFB Success Rate (%) >=98% >=98% >=98%
Reselection Time xx xx xx
DL Traffic Volume (GB) xx xx xx
UL Traffic Volume (GB) xx xx xx
RANKs 1,2,3,4.. % Distribution
Number of Users
Ave DL User throughput (Mbps)
DL Cell Throughput (Mbps)
UL Cell Throughput (Mbps)
Max Cell DL Throughput (Mbps)
2CC CA User Throughput DL (Mbps)
3CC CA User Throughput DL (Mbps) (IA)
4CC CA User Throughput DL (Mbps) (IA)
VoLTE Acc Succ Rate (IA) >=99% >=99% >=99%
VoLTE Retainability Succ Rate (IA) >=98% >=98% >=98%
11
SRVCC Succ Rate (IA) >=96% >=96% >=96%
DL PRB Utilization (%) xx xx xx
UL PRB Utilization (%) xx xx xx
UL Interference <=-100 dBm <=-105 dBm <=-105 dBm
RACH Succ Rate >=98% >=98% >=98%
NB IoT Acc Succ Rate
NB IoT Retainability Rate
NPRACH Succ Rate >=92% >=92% >=92%
NB IoT DL Traffic
NB IoT UL Traffic
Table 9: Multilayer LTE new site OSS KPI per layer

Carrier Size Expected DL Throughput (Mbps)

2CC CA >150
3CC CA >190
4CC CA >230
Table 10: CA expected throughput
CA fixed testing with multiple CC should be done on 10% of sites in a cluster

1.2.2.2 Static or Fixed Testing KPI

Site Name Values for F1 Values for Fx

Sector Name A, B or G A, B or G
PCI xx xx
Device Name and Model Name and Model
Date
Test point Lat xx xx
Test point Long xx xx
EARFCN (DL) and (UL) xx xx
RSRP
RSRQ Xx dB Xx dB
SINR
RANK >=2 >=2
MCS DL xx xx
MCS UL xx xx

12
 Latency <35 msec <35 msec
TX1
TX2
DL User Throughput (Mbps) >= 110 >= 110
UL User Throughput (Mbps) >= 35 >= 35
VoLTE MO Succ Rate >=98% >=98%
2CC CA DL User Throughput (Mbps) >=150 >=150
(IA)
3CC CA DL User Throughput (Mbps) >=190 >=190
(IA)
4CC CA DL User Throughput (Mbps) >=230 >=230
(IA)
VoLTE TM Succ Rate >=98% >=98%
SRVCC Succ Rate >=96% >=96%
CSFB >= 98% >= 98%
NB IoT Accessibility >=98%
NB IoT Retainability >=96%
NB IoT DL THP
NB IoT UL THP
Table 11 Static testing KPI for site

1.2.2.3 Drive Testing KPI

Note, All Max user throughput from drive test KPI has to be selected and averaged from the outer
loop

KPI Name Events Count

PS Accessibility RRC Connection Estab SR >=98%

PS Retainability Drop Rate <=2%
CS Accessibility Successes >=98%
CS Retainability Total Call Established >=98%
Complete >=98%
CSFB Succ Rate >=98%
VoLTE Accessibility (IA) Successes >=99 %
VoLTE Retainability (IA) >=98%
VoLTE MO Succ Rate >=98%
VoLTE MT Succ Rate >=98%
SRVCC Succ Rate >=96%
13
 CS Call Setup Time (Sec) xx
Mean DL Throughput (Mbps) >= 25
Mean UL Throughput (Mbps) >= 10
Max User DL Throughput (Mbps) >= 35
CA DL Throughput (Mbps)
Ave Number of CC for CA
Ave throughput per CC
Latency xx
Time on LTE (%) xx
RANK1,2,3, 4 Distribution (%)
LTE Reselect Time (Sec) (CSFB) xx
Intra Freq HO Succ Rate >=97%
Inter Freq HO Succ Rate >=96%
IRAT Succ Rate xx >=92%
RACH Succ Rate >=94%
NB IoT Acc Succ Rate >=98%
NB IoT Retainability >=97%
NB IoT DL THP
NB IoT UL THP
NPRACH Succ Rate >=92%
Table 13: Multilayer LTE new site SSV/Cluster DT KPI

1.2.3 TDD Testing

1.2.3.1 OSS KPI

Name of KPI Value for 20 MHz
Availability (%) >=99%
Accessibility (%) >=98%
Drop Calls Rate (%) <=2%
IRAT to UMTS Success Rate (%) >=96%
Intra Freq HO Success Rate >=97%
Inter Freq HO Success Rate >=96%
DL Cell Throughput xx
UL Cell Throughput xx
DL User Throughput xx
2CC CA DL User Throughput (IA) xx
3CC CA DL User Throughput (IA) xx
Number of Active Users
DL PRB Utilization (%)

14
 UL PRB Utilization (%)
UL Interference -105 dBm
RACH SR >=98%
Table 14: OSS KPI for single 20 MHz TDD Carrier

1.2.3.2 TDD Fixed Testing

Site Name KPI of TDD 20 MHz

Sector Name
PCI
Device Name and Model
Date
Test point Lat xx
Test point Long xx
EARFCN (DL) and (UL) xx
RSRP Xx dBm
RSRQ
SINR
RANK >=2
MCS DL xx
MCS UL xx
Latency <35 msec
TX1
TX2
DL User Throughput (Mbps) >= 55
UL User Throughput (Mbps) >= 7
2CC CA DL User Throughput (Mbps) >=90
3CC CA User Throughput (Mbps) (IA) >=110
Table 15: Site 20 MHz TDD fixed testing KPI

Carrier Size Expected DL Throughput (Mbps)

2CC CA > 90
3CC CA > 110
Table 16: TDD CA expected throughput

CA fixed testing with multiple CC should be done on 10% of sites in a cluster

1.2.2.3 Drive Testing KPI

Note, All Max user throughput from drive test KPI has to be selected and averaged from the outer
loop

15
 KPI Events @TDD 20 MHz

PS Accessibility RRC Connection Estab SR >=98%

PS Retainability Drop Rate <=2%
Intra Freq HO SR % >=97%
Inter Freq HO SR % >=96%
Mean User DL Throughput >=12
(Mbps)
Mean User UL Throughput >=4
(Mbps)
Max User DL Throughput (Mbps) >=24
CA DL User Throughput (Mbps)
Ave number of CC for CA
Ave throughput per CC
Latency
Time on LTE (%) xx
LTE Reselect Time (Sec) xx
RACH Success Rate >=90%
IRAT Failures xx <=8%
RANKs (%) Rank 1,2,3 and 4
Table 20: 20 MHz TDD Drive Test KPI

Below are tables of DL and UL of TDD throughputs during drive test and can be
used as a reference:

Mobility (DL) Throughput Results (During Drive Test)

Average DL User Throughput DL Avg Throughput Target (Average)

DL User THP High (Near Site) TDD @20MHz: ≥ 18 Mbps                    

DL_SINR ≥ 15.5dB, DL_RSRP>-100dBm

DL User THP Quality Low (Cell edge) TDD @20MHz: ≥ 5 Mbps                    

0dB ≤ DL_SINR < 7dB
-112<=DL_RSRP<=-100dBm
Table 21: TDD DT DL Throughput KPI

Mobility (UL) Throughput Results (During Drive Test)

16
Average UL User Throughput UL Avg Throughput Target (Average)

UL User THP High (Near Site) TDD @20MHz: ≥ 3.2 Mbps                  

DL_RSRP ≥ -80dBm  

UL User THP Quality Low (Cell edge) TDD @20MHz: ≥ 1.5 Mbps                  
-112<=DL_RSRP<=-100dBm  
Table 22: TDD DT UL throughput KPI

1.2.4 TDD Testing for Smart Phones

STC is going to introduce 3 TDD carriers with 20 MHz each dedicated for smart
phones. The testing for these carriers will be very similar to FDD carriers in
addition to the carrier aggregation testing which involved TDD and FDD. Depends
on the number of carriers for a site, each carrier will be tested separately for
functionality first. After functionality testing is completed and PASSED, inter
frequency and inter technology testing will be conducted, where CA will be
included.

1.2.4.1 OSS KPI

Below is the OSS KPI for a TDD carrier:
Name of KPI Value for 20 MHz
Availability (%) >=99%
Accessibility (%) >=98%
Drop Calls Rate (%) <=2%
IRAT to UMTS Success Rate (%) >=96%
Intra Freq HO Success Rate >=97%
Inter Freq HO Success Rate >=96%
DL Cell Throughput xx
UL Cell Throughput xx
DL User Throughput xx
2CC CA DL User Throughput xx
3CC CA DL User Throughput xx

17
 4CC CA DL User Throughput xx
RANKs 1,2,.. % Distribution
VoLTE Acc Succ Rate (IA) >=99%
VoLTE Retainability Succ Rate (IA) >=98%
SRVCC Succ Rate (IA) >=96%
Number of Active Users
DL PRB Utilization (%)
UL PRB Utilization (%)
UL Interference -105 dBm
RACH SR >=98%
Table 23: TDD for Smart Phone OSS KPI

1.2.4.2 Fixed testing

Site Name Value

Sector Name Alpha, Beta or Gamma
PCI xx
Device Name and Model
Date
Test point Lat xx
Test point Long xx
EARFCN (DL) and (UL) xx
RSRP
RSRQ Xx dB
SINR
RANK >=2
MCS DL xx
MCS UL xx
Latency <35 msec
TX1
TX2
DL User Throughput (Mbps) >= 55
UL User Throughput (Mbps) >= 7
2CC CA DL User Throughput (Mbps) (IA) >=110
3CC CA DL User Throughput (Mbps) (IA) >=160
4CC CA DL User Throughput (Mbps) (IA) >=210

18
 VoLTE MO Succ Rate >=98%
VoLTE TM Succ Rate >=98%
SRVCC Succ Rate >=96%
CSFB >=98%
Table 24: TDD for Smart Phone Site Fixed point testing KPI per TDD Carrier

1.2.4.3 Drive Testing

Note, All Max user throughput from drive test KPI has to be selected and averaged from the outer
loop

KPI Name Events Count

PS Accessibility RRC Connection Estab SR >=98%

PS Retainability Drop Rate <=2%
CS Accessibility Successes >=98%
CS Retainability Total Call Established >=98%
Complete >=98%
CSFB Succ Rate >=98%
VoLTE Accessibility (IA) Successes >=99 %
VoLTE Retainability (IA) >=98%
VoLTE MO Succ Rate >=98%
VoLTE MT Succ Rate >=98%
SRVCC Succ Rate >=96%
CS Call Setup Time (Sec) xx
Mean User DL Throughput (Mbps) >= 12
Mean User UL Throughput (Mbps) >= 4
Max User DL Throughput (Mbps) >= 24
CA DL Throughput (Mbps)
Ave Number of CC for CA
Ave throughput per CC
Latency xx
Time on LTE (%) xx
RANK1, 2, 3, 4 Distribution (%)
LTE Reselect Time (Sec) (CSFB) xx
Intra Freq HO Succ Rate >=97%
Inter Freq HO Succ Rate >=96%
IRAT Succ Rate xx >=92%
RACH Succ Rate >=94%
Table 26: TDD for Smart Phone Drive Test KPI per carrier

Note: Drive test latency is end to end and there should be no failing threshold.

19
1.2.5 TESTING OF NEW SITE WITH HIGHER ORDER MIMO SOLUTION

1.2.5.1 OSS KPI

Name of KPI Value
Antenna MIMO Configuration
Availability (%) >=99%
Accessibility (%) >=98%
Drop Calls Rate (%) <=2%
Intra Freq HO Success (X2, S1) (%) >=97%
Inter Freq HO Success (%) >=96%
IRAT Success Rate (%) >=96%
CSFB Success Rate (%) >=98%
Reselection Time xx
DL Traffic Volume (GB) xx
UL Traffic Volume (GB) xx
RANKs 1,2,3,4.. % Distribution
Number of Users
Ave DL User throughput (Mbps)
DL Cell Throughput
UL Cell Throughput
Max Cell DL Throughput
2CC CA Throughput DL (IA)
3CC CA Throughput DL (IA)
4CC CA Throughput DL (IA)
VoLTE Acc Succ Rate (IA) >=99%
VoLTE Retainability Succ Rate (IA) >=98%
SRVCC Succ Rate (IA) >=96%
DL PRB Utilization (%) xx
UL PRB Utilization (%) xx
UL Interference <=-105 dBm/-100 L700
RACH Succ Rate >=98%
Table 27: Higher Order MIMO OSS KPI for High Band

Note: If higher order MIMO is implemented on TDD only, then all CSFB, NB IoT and VoLTE related tests
will not be required.

1.2.5.2 Fixed Testing

20
Site Name 4x4 MIMO FDD/TDD M MIMO FDD/TDD
Sector Name A, B or C A, B or C
PCI xx xx
Device Name and Model Name and Model
Date
Test point Lat xx xx
Test point Long xx xx
EARFCN (DL) and (UL) xx xx
RSRP
RSRQ Xx dB Xx dB
SINR
RANK >=2 >=2
MCS DL xx xx
MCS UL xx xx
Latency <35 msec <35 msec
TX1
TX2
DL User THP (Mbps) >= 140/55 >= 140/55
UL User THP (Mbps) >= 35/7 >= 35/7
2CC CA DL User THP (Mbps) >= 190/110 >= 190/110
3CC CA DL User THP (Mbps) >= 240/130 >= 240/130
4CC CA DL User THP (Mbps) >=290 >=290
VoLTE MO Succ Rate >=98% >=98%
VoLTE TM Succ Rate >=98% >=98%
SRVCC Succ Rate >=96% >=96%
NB IoT Accessibility >=98% >=98%
NB IoT Retainability >=96% >=96%
NB IoT DL THP
NB IoT UL THP
Table 28: Cell expected user THP with higher order MIMO for 20 MHz channel

Carrier Size Expected DL Throughput (Mbps)

21
 2CC CA >190/110
3CC CA >240/130
4CC CA >290
Table 29: Expected throughput for higher order MIMO for FDD and TDD
CA fixed testing with multiple CC should be done on 10% of sites in a cluster
Note, the above throughput KPI is for 20 MHz channel, for 15 MHz, the expected throughput
will be as below:
KPI Name Results for 4x4 MIMO
DL User THP >= 105 Mbps
UL User THP >= 25 Mbps
2CC CA DL User THP >=150 Mbps
3CC CA DL User THP >=200 Mbps
4CC CA DL User THP >=250 Mbps

1.2.5.3 Drive Testing

Note, All Max user throughput from drive test KPI has to be selected and averaged from the outer
loop

KPI Name Events Count

PS Accessibility RRC Connection Estab SR >=98%

PS Retainability ERAB Drop Rate <=2%
CS Accessibility Successes >=98%
CS Retainability Total Call Established >=98%
Complete >=98%
CSFB Succ Rate >=98%
VoLTE Accessibility (IA) Successes >=99 %
VoLTE Retainability (IA) >=98%
VoLTE MO Succ Rate >=98%
VoLTE MT Succ Rate >=98%
SRVCC Succ Rate >=96%
CS Call Setup Time (Sec) xx
Mean DL Throughput (Mbps) >= 35
Mean UL Throughput (Mbps) >= 15
Max DL User Throughput (Mbps) >= 75
CA DL Throughput (Mbps)
Ave Number of CC for CA
Ave throughput per CC
Latency xx
Time on LTE (%) xx

22
 RANK1,2,3, 4 Distribution (%)
LTE Reselect Time (Sec) (CSFB) xx
Intra Freq HO Succ Rate >=97%
Inter Freq HO Succ Rate >=96%
IRAT Succ Rate xx >=92%
RACH Succ Rate >=94%
NB IoT Acc Succ Rate >=98%
NB IoT Retainability >=97%
NB IoT DL THP
NB IoT UL THP
NPRACH Succ Rate >=92%
Table 26: Higher Order MIMO DT expected KPI

PART 2 CO-LOCATION
Note: Site acceptance in this section (Section 2) is separate from the Aspiration testing. Therefore,
when site’s KPI during functionality testing meet the requirements, the site must be PASSED the
functionality testing

2.1 PHYSICAL INSPECTION

If an upgrade is required to have any additional hardware such as antennas and

RRU, physical inspection is a must to make sure design parameters have been
implemented correctly. Please refer to section 1.1 as a reference for this task

2.2 PERFORMANCE

2.2.1 Single LTE Carrier Upgrade

2.2.1.1 OSS KPI

Name of KPI Value
Availability (%) >=99%
Accessibility (%) >=98%

23
 Drop Calls Rate (%) <=2%
Intra Freq HO Success (X2, S1) (%) >=97%
Inter Freq HO Success (%) >=96%
IRAT Success Rate (%) >=96%
CSFB Success Rate (%) >=98%
Reselection Time xx
DL Traffic Volume (GB) xx
UL Traffic Volume (GB) xx
RANKs 1,2,3,4.. % Distribution
Number of Users
Ave DL User throughput (Mbps)
DL Cell Throughput
UL Cell Throughput
Max Cell DL Throughput
2CC CA Throughput DL (IA)
3CC CA Throughput DL (IA)
4CC CA Throughput DL (IA)
VoLTE Acc Succ Rate (IA) >=99%
VoLTE Retainability Succ Rate (IA) >=98%
SRVCC Succ Rate (IA) >=96%
DL PRB Utilization (%) xx
UL PRB Utilization (%) xx
UL Interference (for 700 MHz <=-100) <=-105 dBm
RACH Succ Rate >=98%
NB IoT Acc Succ Rate
NB IoT Retainability Rate
NPRACH Succ Rate >=92%
NB IoT DL Traffic
NB IoT UL Traffic
Table 27: Single carrier upgrade OSS KPI

If the upgrade is not L700, then all NB IoT related tests and OSS KPI will not be
required. Also, if the upgrade is TDD carrier, all NB IoT and VoLTE KPI are not
required.
2.2.1.2 Fixed Testing

Static or Fixed Test KPI for 20 MHz Channel

24
 Site Name Value
Sector Name Alpha, Beta or Gamma
PCI xx
Device Name and Model
Date
Test point Lat xx
Test point Long xx
EARFCN (DL) and (UL) xx
RSRP Xx dBm
RSRQ Xx dB
SINR
RANK >=2
MCS DL xx
MCS UL xx
Latency <35 msec
TX1
TX2
DL User Throughput (Mbps) >= 110
UL User Throughput (Mbps) >= 35
2CC CA DL Throughput (Mbps) >=150
3CC CA DL Throughput (Mbps) >=190
4CC CA DL Throughput (Mbps) >=230
VoLTE MO Succ Rate >=98%
VoLTE TM Succ Rate >=98%
SRVCC Succ Rate >=96%
CSFB >=98%
NB IoT Accessibility >=98%
NB IoT Retainability >=96%
NB IoT DL THP
NB IoT UL THP
Table 28: Single carrier upgrade fixed test KPI for 20 MHz Channel

Carrier Size Expected DL Throughput (Mbps)

2CC CA >150
3CC CA >190

25
 4CC CA >230
Table 29: Expected CA throughput for one carrier co-lo
CA fixed testing with multiple CC should be done on 10% of sites in a cluster
Note: Fixed test throughput will depend on the carrier size added. Please refer to the table
below for the 256 QAM modulation expected throughput:

Carrier Size Expected Throughput DL (Mbps) Expected UL Throughput (Mbps)

20 MHz >110 >35
15 MHz >90 >27
10 MHz >60 >18
Table 30: Throughput per carrier with 256 QAM modulation

2.2.1.3 Drive Testing

Note, All Max user throughput from drive test KPI has to be selected and averaged from the outer
loop

KPI Name Events Count

PS Accessibility RRC Connection Estab SR >=98%

PS Retainability ERAB Drop Rate <=2%
CS Accessibility Successes >=98%
CS Retainability Total Call Established >=98%
Complete >=98%
CSFB Succ Rate >=98%
VoLTE Accessibility (IA) Successes >=99 %
VoLTE Retainability (IA) >=98%
VoLTE MO Succ Rate >=98%
VoLTE MT Succ Rate >=98%
SRVCC Succ Rate >=96%
CS Call Setup Time (Sec) xx
Mean DL Throughput (Mbps) zz
Mean UL Throughput (Mbps) yy
CA DL Throughput (Mbps)
Ave Number of CC for CA DL/UL
Ave throughput per CC DL/UL
Latency xx
Time on LTE (%) xx
RANK1,2,3, 4 Distribution (%)
LTE Reselect Time (Sec) (CSFB) xx
26
 Intra Freq HO Succ Rate >=97%
Inter Freq HO Succ Rate >=96%
IRAT Succ Rate xx >=92%
RACH Succ Rate >=94%
NB IoT Acc Succ Rate >=98%
NB IoT Retainability >=97%
NB IoT DL THP
NB IoT UL THP
NPRACH Succ Rate >=92%
Table 31: Single carrier upgrade drive test KPI

2.2.2 Multi LTE Carrier Upgrade

2.2.2.1 OSS KPI

Name of KPI Value
Availability (%) >=99%
Accessibility (%) >=98%
Drop Calls Rate (%) <=2%
Intra Freq HO Success (X2, S1) (%) >=97%
Inter Freq HO Success (%) >=96%
# of Succ redirection/HO from TDD to FDD (IA)
# of Succ redirection/HO from FDD to TDD (IA)
IRAT Success Rate (%) >=96%
CSFB Success Rate (%) >=98%
Reselection Time xx
DL Traffic Volume (GB) xx
UL Traffic Volume (GB) xx
RANKs 1,2,3,4.. % Distribution
Number of Users
Ave DL User throughput (Mbps)
DL Cell Throughput
UL Cell Throughput
Max Cell DL Throughput
2CC Inter Site CA Throughput DL (IA)
VoLTE Acc Succ Rate (IA) >=99%
VoLTE Retainability Succ Rate (IA) >=98%
SRVCC Succ Rate (IA) >=96%
DL PRB Utilization (%) xx
27
 UL PRB Utilization (%) xx
UL Interference (for 700 <= -100) <=-105 dBm
RACH Succ Rate >=98%
NB IoT Acc Succ Rate
NB IoT Retainability Rate
NPRACH Succ Rate >=92%
NB IoT DL Traffic
NB IoT UL Traffic
Table 32: Multi carriers upgrade OSS KPI

If L700 is not included in the upgrade, no NB IoT testing is required

2.2.2.2 Static Testing

Static or Fixed Test KPI for 20 MHz Channel

Site Name Values for F1 Values for Fx
Sector Name A, B or G A, B or G
PCI xx xx
Device Name and Model Name and Model
Date
Test point Lat xx xx
Test point Long xx xx
EARFCN (DL) and (UL) xx xx
RSRP
RSRQ Xx dB Xx dB
SINR
RANK >=2 >=2
MCS DL xx xx
MCS UL xx xx
Latency <35 msec <35 msec
TX1
TX2
DL User Throughput (Mbps) >= 110 >= 110
UL User Throughput (Mbps) >= 35 >= 35
VoLTE MO Succ Rate >=98% >=98%
2CC CA DL User Throughput (Mbps) >=150 >=150
(IA)
3CC CA DL User Throughput (Mbps) >=190 >=190
28
 (IA)
4CC CA DL User Throughput (Mbps) >=230 >=230
(IA)
VoLTE TM Succ Rate >=98% >=98%
SRVCC Succ Rate >=96% >=96%
CSFB >= 98% >= 98%
NB IoT Accessibility >=98%
NB IoT Retainability >=96%
NB IoT DL THP
NB IoT UL THP
Table 33: Multi carriers upgrade fixed test KPI

Carrier Size Expected DL Throughput (Mbps)

2CC CA >150
3CC CA >190
4CC CA >230
Table 34: expected CA throughput for multi carrier co-lo
CA fixed testing with multiple CC should be done on 10% of sites in a cluster
Note: Fixed test throughput will depend on the carrier size added. Please refer to the table
below for 256 QAM modulation expected throughput:

Carrier Size Expected Throughput DL (Mbps) Expected UL Throughput (Mbps)

20 MHz >110 >35
15 MHz >90 >27
10 MHz >60 >18
Table 35: Expected throughput per LTE carrier with 256 QAM

2.2.2.3 Drive Testing

Note, All Max user throughput from drive test KPI has to be selected and averaged from the outer
loop

Drive Test KPI

KPI Name Events Count

PS Accessibility RRC Connection Estab SR >=98%

PS Retainability ERAB Drop Rate <=2%
CS Accessibility Successes >=98%
CS Retainability Total Call Established >=98%

29
 Complete >=98%
CSFB Succ Rate >=98%
VoLTE Accessibility (IA) Successes >=99 %
VoLTE Retainability (IA) >=98%
VoLTE MO Succ Rate >=98%
VoLTE MT Succ Rate >=98%
SRVCC Succ Rate >=96%
CS Call Setup Time (Sec) xx
Mean DL Throughput (Mbps) zz
Mean UL Throughput (Mbps) yy
CA DL Throughput (Mbps)
Ave Number of CC for CA
Ave throughput per CC
Latency xx
Time on LTE (%) xx
RANK1,2,3, 4 Distribution (%)
LTE Reselect Time (Sec) (CSFB) xx
Intra Freq HO Succ Rate >=97%
Inter Freq HO Succ Rate >=96%
IRAT Succ Rate xx >=92%
RACH Succ Rate >=94%
NB IoT Acc Succ Rate >=98%
NB IoT Retainability >=97%
NB IoT DL THP
NB IoT UL THP
NPRACH Succ Rate >=92%
Table 36: Multi carriers upgrade drive test KPI

In case a new TDD carrier is added and MIMO upgrade is deployed, follow the
testing procedure for TDD and MIMO mentioned in sections 1.2.4 and 1.2.5

2.2.3 TDD for Smart Phone Carrier Upgrade

Please refer to section 1.2.4 in this document for TDD testing for acceptance

2.2.4 Site Upgrade with Higher Order MIMO

Please refer to section 1.2.5 in this document for site with higher order MIMO.

3 REPORT

30
The site/carrier acceptance report will include all essential information that will
facilitate the acceptance checkup. Such information will be the physical
inspection summary sheet, SSD and SSV summary sheet, Drive test Summary
sheet and plots.
Below is the list of drive test plots per site/cluster:
 Drive Route which will include the entire locality
 GSM Signal Strength
 GSM Quality
 UEs Connected RSRP
 UEs Connected SINR
 UEs Connected Serving Cell (PCI)
 DL Throughput
 UL Throughput
 TM plot
 RANK Distribution plot
 Time on LTE
 IRAT Event Plot

PROJECT 2: 5G
The 5G sites in the Aspiration project are not going to be considered as a
contributors to the throughput goals set for the Aspiration project (50 and 65
Mbps).

31
5G sites are none standalone (NSA) type, and will go through the same
acceptance process as LTE sites. It is divided into 2 section:
1. Physical
2. Performance
Beamforming is going to be a very important feature in 5G. Therefore, all
measurements are going to be on per beam basis.
This release includes a conditional acceptance passing KPI due to 2 challenges
just arise:
1. Transport
2. 5G devices
Some sites due are not getting the 10 Gbps capacity transport required for the 5
G sites. Therefore, a conditional passing KPIs are provided.
Also, 5G devices have not matured in market yet. There is evidence of 5G UE
instability in market. For that reason, conditional passing KPI are provided as
well.
However, Vendors are obligated to retest all the sites tested under the
conditional acceptance when 10 Gbps transport link is installed to those sites.
Also, all sites that were tested under conditional due to devices challenges are
required to be tested when new capable devices are available. All sites must
pass or meet all the standard KPI. It is expected that there would be no
conditional testing due to device challenges later than Dec 31 st.

2.1 PHYSICAL INSPECTION

Below is a table that need to be filled for the physical inspection:

Physical Inspection Type Proposed Actual Pass/Fail
Site Name/Number
Site Long
Site Lat
Antenna Height
Antenna Models
Antenna Azimuth
Antenna M DT

32
 NR/BBU Model
Table 37: 5G site physical inspection and verification

2.2 PERFORMANCE

Before any field test takes place, it is recommended that all-important counters
that are related to 5G performance have to be checked and verified.
2.2.1 OSS KPI

This set of OSS KPI will be applied to conditional acceptance as well

OSS KPI is going to be about the performance and data delivery of the 5G site in
the network. It is going to be a good indication about the health of the 5G site
and its performance. Below is a detailed OSS KPI table that includes all
performance and throughput KPIs.

Table 38
4G KPI
Name of KPI Before After 5G KPI (4x4)
Availability (%) xx No degradation >=99%
Accessibility (%) xx No degradation >=98%
Retainability (%) xx No degradation <=2%
Reliability** (If Applicable) xx No degradation >=99.9%
Intra gNB SCell Addition Succ Rate xx No degradation >=99%
Inter gNB SCell Addition Succ Rate xx No degradation >=99%*
Bandwidth xx No degradation Xx MHz
DL Traffic Volume (GB) xx No degradation xx
UL Traffic Volume (GB) xx No degradation xx
DL User Throughput (Mbps) xx No degradation
UL User Throughput (Mbps) xx No degradation
Radio TX Rank Distribution (%) xx No degradation …, %Rnk4, %Rnk3, %Rnk2, %Rnk1
DL PRB Utilization (%) xx No degradation <<Xx
UL PRB Utilization (%) xx No degradation <<Xx
UL Interference xx No degradation -110 dBm
Table 38 5G OSS KPI
Note: Due to the absence or not enough of 5G devices, the OSS KPI will not be considered at this time.

*Note, Inter 5G SCell addition KPI of >=99% will be applicable on 5G adjacent sites.
** Reliability at this point is related to URLLC. It is the probability of transmitting a pocket of 32
Bytes of user plane data in 1 msec latency. The requirement is 1-10 -3. If URLLC service is not offered,
Reliability will not be required.

33
 2.2.2 Fixed Testing

Conditional acceptance KPI for fixed testing will be considered.

For a 100 MHz channel and 30 KHz spacing, the expected throughput with a UE
that is 4x4 MIMO capable device and 256 QAM is over 1Gbps with RANK4.
The table below is required to be filled with all requested information:

Conditional Due to Transport Challenges

Test Date La Long Site EARFCN PCI RSRP BLE SINR RANK Latency^ (ms) MCS Max THP DL Ave THP UL Distance Coupling
Name t Sector R UE DL/UL from Cell EARFCN
ID (%)
5G >=2 <10 >=400 Mbps >=60 Mbps

Table 39.a Fixed testing data report for conditional acceptance with transport issues

Conditional Due to Devices Instability

Test Date La Long Site EARFCN PC RSRP BLER SINR RANK Latency^ (ms) MC Max Usr Ave THP DL Ave THP UL Distance Coupling
Name t Sector I (%) UE DL/UL S THP from Cell EARFCN
ID
5G >2 <10 >= 750 >=550 Mbps >=50 Mbps
Near Mbps
5G >2 <13 >= 320 >=250 Mbps >=40 Mbps
Middl Mbps
e
5G Cell >=2 <15 >= 90 >=50 Mbps >=20 Mbps
Edge Mbps
Table 39.b Fixed testing data report for conditional acceptance wit device issues

None-Conditional
Test Date Lat Long Site EARFCN PCI BI1 BI2 BI3 BI4 BLE SINR/BI RANK Latency MCS Max Ave Ave Distance Coupling
Name Sector RSRP RSRP RSR RSRP R UE DL/UL CP/UP Usr THP THP from Cell EARFCN
ID P (%) (ms) THP* DL UL**
5G >2 <10 >= >=800 >=190
Near 1.2 Mbps Mbps
Gbps
5G >2 <10 >= >=500 >=95
Middle 700 Mbps Mbps
Mbps
5G Cell >=2 <10 >= >=350 >=50
Edge 400 Mbps Mbps
Mbps
Table 39 5G Fixed testing data report

*Note: RANK4 and above may not be obtained due to lack of multipath. Tester should do his best
effort to get the highest RANK by getting a spot in LOS near high buildings, where there is a high
probability of getting high RANK. Below is the expected Max. DL throughput for each RANK:

34
 1. Rank 8  Throughput > 2.4 Gbps
2. Rank ≈ 7  Throughput > 2.1 Gbps
3. Rank ≈ 6  Throughput > 1.8 Gbps
4. Rank ≈ 5  Throughput > 1.5 Gbps
5. Rank ≈ 4  Throughput > 1.2 Gbps
6. Rank ≈ 3  Throughput > 700 Mbps
7. Rank ≈ 2  Throughput > 400 Mbps
8. Rank ≈ 1  Throughput > 100 Mbps

UL Max throughput for each RANK:

1. Rank ≈ 2  Throughput > 190 Mbps

2. Rank ≈ 1  Throughput > 95 Mbps

**The UL throughput above is based on UE with 2TXs capability. In case the UE has only one TX, then
the throughput values above beyond RANK 1 will be divided by 2.

Also, Cell edge is defined as RSRP >= -98 dBm and at a distance up to 1 km.

In order to get a latency of 1ms or less, one of the aspect that contribute to LL is the sub frame
configuration that has to be less than the 1 msec. It is going to be in the range of 0.14 ≤ TTI ≤ 0.5
msec. This is only used in the case of URLLC. Also, other players that contribute to LL is the distance
between the NR and core and the number of hops in the backhaul topology and load conditions in the
core network.

The latency mentioned in this document is strictly RAN latency. E2E latency will add 8 to 10 msec of
delay depends on the number of nodes the packets have to go through.

2.2.3 Mobility Testing

Drive testing will be required for the conditional acceptance to insure
successful mobility (HO) of 5G.
Note, All Max user throughput from drive test KPI has to be selected and averaged from the outer
loop

5G deployment on the 3.6 to 3.8 GHz band is going to be coverage challenged. 5G

dual connectivity for NSA 5G is going to be an essential part of 5G network to
ensure reliability. Also, the UE should maintain a 5G connection if signal permits
when the UE performs Inter EnodeB HO. Therefore, mobility drive testing is the
method to find out how good and reliable the 5G coverage and at what distance.
Massive MU MIMO (64x64) is going to be the antenna Configuration that is going
to be implemented on the 5G sites. As such, mobility testing will be conducted as
a SSV on those sites only. A road map should be generated for each site. The
drive test map will consist of 2 loops interconnected. The inner loop starts from a

35
close proximity of a site, and extends toward the cell edge. A clockwise and
counter clockwise drive should be conducted on the inner loop to assure mobility
is functioning in both directions.
Another feature will be tested in drive testing is the beamforming. While driving
around the site, the UE should be exposed to all beams through beam switching
(Intra sector) and hand over (Intra site). Therefore, while generating plots, there
must be a plot for every beam per sector with its Beam Index ID.
The entire map will be driven in one single drive and the UE should go through all
Ranks of 5G. Below is an example of drive testing map:

Fig. 4.5 Drive Testing Map

Conditional Acceptance due Transport Issues:

KPI 4G 5G
Accessibility No degradation >= 97%
36
 Retainability No degradation <= 2%
Reliability* (If Applicable) N/A >=99.9%
Intra gNB SCell Addition Succ Rate >= 99%
Inter gNB SCell Addition Succ Rate >= 99%*
Mobility Interruption Time
CSFB (IA) No degradation >=92%
Time on 5G
Rank 4 % Rank 4 %
Rank 3 % Rank 3 %
DL /UL Rank distribution
Rank 2 % Rank 2 %
Rank 1 % Rank 1 %
Avg MCS
Avg RSRP
Avg SINR
Ave User DL THP
Ave User UL THP
Max User DL THP
Avg Spectral Efficiency
Table 40.a DT Summary KPI for conditional acceptance with Transport Issues

Conditional Acceptance Due to Device Instability:

KPI 4G 5G
Accessibility No degradation >= 97%
Retainability No degradation <= 2%
Reliability* (If Applicable) N/A >=99.9%
Intra gNB SCell Addition Succ Rate >= 99%
Inter gNB Scell Addition Succ Rate >= 99%*
Mobility Interruption Time
CSFB (IA) No degradation >=92%
Time on 5G
Rank 4 % Rank 4 %
Rank 3 % Rank 3 %
DL /UL Rank distribution Rank 2 % Rank 2 %
Rank 1 % Rank 1 %
Avg MCS
Avg RSRP
Avg SINR

37
 Ave User DL THP >= 230 Mbps
Ave User UL THP >=20 Mbps
Max User DL THP
Avg Spectral Efficiency
Table 40.b DT Summary Report for conditional acceptance with UE Instability
*Note, Inter 5G SCell addition KPI of >=99% will be applicable when 5G sites are adjacent.

None Conditional

Table 40
KPI 4G 5G
Accessibility No degradation >= 97%
Retainability No degradation <= 2%
Availability >= 99%
Number of Beams per Antenna N/A
Reliability* (If Applicable) >=99.9%
Intra gNB SCell Addition Succ Rate >= 99%
Inter gNB SCell Addition Succ Rate
Mobility Interruption Time <= 1 msec
CSFB (IA) No degradation >=92%
Time on 5G
Rank 8 % (IA) Rank 7 %
Rank 7 % (IA) Rank 5 %
DL /UL Rank distribution Rank 6 % (IA) Rank 4 %
Rank 5 % (IA) Rank 5 %
Rank 4 % Rank 4 %
Rank 3 % Rank 3 %
Rank 2 % Rank 2 %
Rank 1 % Rank 1 %
Avg MCS No degradation
Avg RSRP
Avg SINR
Ave User DL THP >= 400 Mbps
Ave User UL THP >=65 Mbps
Max User DL THP >= 1.2 Gbps
Avg Spectral Efficiency No degradation
Table 40 DT Summary Report

38
* Reliability at this point is related to URLLC.
It is the probability of transmitting a pocket of 32 Bytes
of user plane data in 1 msec latency. The requirement is 1-10 -3. If URLLC service is not offered,
Reliability will not be required.

Mobility interruption time is defined as the time span during which a UE cannot exchange UP packets
with any BS during transitions. It can be regarded as intra-system handover interruption time. Note
that in 5G system, handover between adjacent BS may no longer exist due to solutions based on
multi-connectivity and CP / UP decoupling

Mobility Interruption Time is only applicable when the network is in Dual connection or decoupling
mode. If the 5G is in single connection, MIT is not required.

The main reason for collecting both 4 and 5G data is to observe the beamforming
gain of 5G over 4G.

3.0 FINAL REPORT

The conditional acceptance report will have the same format as the normal
acceptance report, with the conditional acceptance KPI.
The final report is consisted of 3 parts, the fixed tests results, the Mobility test
results or summary and the OSS KPI report. In addition to the template that was
provided for the fixed tests and DT data collections, the report must include the
drive test road map, RSRP, SINR Plots, graphs for OSS KPI, tables showing
throughput of the 5G network and the 4G OSS KPI for comparison.
Below is the list of drive test plots that needs to be included in final report:

Site Drive Route

Drive route showing the frequency that was driven
Site UEs Connected gRSRP
Site UEs Connected 4G RSRP
Site UEs Connected gSINR
Site UEs Connected 4G SINR
Site UEs Connected Serving Cell

39
Site DL Throughput
Site UL Throughput
Site 5G RANK plot
Site 4G RANK plot
Also required:
Graphs:
Showing cell throughputs on all testing spots of 5G networks and the below
requirements:
eSINR
eRSRP
Ranks

4 ASPIRATION TESTING

There is no testing is required for Aspiration. The success of the Aspiration

project will be measured by generating the Ookla speed test report per footprint.
The KPI has to meet or pass the milestones set for this project.

Note: Cluster Bandwidth is the total number of MHz in cluster counted on per site base. A site with 2 20 MHz
carriers will be counted as 40 MHz. The purpose of this KPI is to find out how much additional MHz were added
to a cluster and their contribution to the overall throughput KPI improvement.

5 REGIONAL ASPIRATION ACCEPTANCE

This section is strictly to reflect the OOKLA speed test outcome from the network upgrade. It is the most
important and best indicator of the goal that was set. It is simply the reading of OOKLA speed test from
their web page and include them in the table below. The values must be > 50 Mbps for 2019 and > 65
Mbps for 2020.

For 2019 Readings in Mbps:

Region October OOKLA November OOKLA December OOKLA 2019 OOKLA

Huawei
Ericsson

40
 Nokia

For 2020 OOKLA Readings in Mbps:

Region Jan Feb Mar Apr May Jun Jul Sep Oct Nov Dec 2020
Huawei
Ericsson
Nokia

6 CONCLUSION

Aspiration project is put forward as an initiative for STC to improve the user
experience from an average of 35 Mbps to 50 Mbps by the end of 2019 and 65
Mbps by the end of 2020 on the LTE network. The criteria put forward is to
assure that the Aspiration project was executed or implemented accordingly, and
the outcome meets the requirements.

41