Troubleshooting high dropping cells

1 © NOKIA
 SCA Optimization
• TARGET: Reduce DCR to below 1.5%

•
Main drop reasons:

• - Sites out of service or degraded service (Transmission/HW faults)

- Poor coverage (Possible HW problems, should match TDMA)
- Parameter discrepancies (should be audited regularly)
- Interference (Some hotspots especially on hopping layer)

• All these require case by case analysis to identify the cause and correction
=> Effective process required to prioritize the most important dropping cells
and to analyze and correct the problems fast

2 © NOKIA
 Responsibilities
• Market RF leads are responsible for the organization of the optimization
effort in the market
• What is needs to be ensured:
• Proper process to follow
1. Prioritization of work
2. Find problem cause or causes
3. Make corrections
4. Log found problems and corrections made

• Effective troubleshooting methods

(study stats before drive testing, drive test when required to isolate the problems)

• Follow up of progress (daily review of progress of area teams and

network KPIs) and corrective actions if progress is not satisfactory
• Escalation of tough trouble cases with high drop rates to leads
• Cooperation with Cingular engineers (area teams are a joint effort)
• Reporting of progress (weekly readout to report the progress)

3 © NOKIA
 Market Optimization progress reporting
• Area team 1:
• # BTSs analyzed
• breakdown of problems detected
• breakdown of corrections implemented
• KPI evolution graph (DCR8c, RXQUAL 6-7, HO failures…)

• Area team 2:
• # BTSs analyzed
• breakdown of problems detected
• breakdown of corrections implemented
• KPI evolution graph (DCR8c, RXQUAL 6-7, HO failures…)

• Area team 3:
• # BTSs analyzed
• breakdown of problems detected
• breakdown of corrections implemented
• KPI evolution graph (DCR8c, RXQUAL 6-7, HO failures…)

4 © NOKIA
 Troubleshooting process flow (area teams)
Identify the top dropping cells in the area
- For example: DCR % x #raw_drops
- Average at least past 2 days to filter out one-off problems
- Prioritize based on: - #raw_drops
- DCR%_GSM – DCR%_TDMA

1. Check log for previously detected problems

- If problem history exists, check if previous corrective actions were
implemented

2. Alarms Fault identified

- Check alarms (active and Escalate to Record to log
canceled) operatios
No alarms indicating
faults
3. KPI analysis
- Analyze KPIs to find the cause
- Network Doctor report 204
recommended

5 © NOKIA
 KPI analysis
Other than radio
3.1 TCH drop cause distribution reason and HO fails
- Radio fail & Abis fail generally due to radio reason Investigate
- HO drops may be caused by radio reason or bad
adjacency definitions/HO parametrtization Record to log

Mainly radio reason Mainly HO drops

3.2 Signal level/quality investigation 3.3 HO investigation

(see next slides) (see next slides)

In most cases both of these

investigations should be done
anyway

6 © NOKIA
 Signal level/quality investigation
3.2 Signal level (UL/DL quality/level matrix)
- In case of dual band, poor level on 1900 BTS indicates
1. Unavailability of 850 BTS
2. High blocking on 850 BTS
3. incorrect HO/CBCCH parameter settings
- Identify if main problem is UL or DL (RXQUAL 6 &7)
- Compare UL & DL levels (UL level normally about 10 dB lower in
cell without TMA) higher level difference indicates link balance
problem => TMA may be needed).
- Compare to TDMA level distribution if possible => If signal level
much lower than TDMA compare GSM & TDMA tx power levels
(check GSM PMAX settings with MML) => if powers ok then Make
field measurements may be needed to compare TDMA & GSM corrections
levels => if levels different also close to site then site RF paths
need to be checked. Record to log
- Further things to check if signal level is poor
1. check if neighboring cells are/have been off air (alarms)
2. check for HO failures due to blocking in target cells
3. investigate if missing handovers exist (call dragging)
4. also check timing advance distribution for calls too far away
from the site => calls far away may indicate remote
dominance island, can be identified by channel finder
measurement => remote dominance can possibly be
corrected by 1. setting idle BA lists to (ADJ), 2. restricting
max TA in call setup, 3. downtilting antennas.

7 © NOKIA
 Handover investigation

3.3 HO investigation
- Check HO failure reason & HO drop statistics
- Check HO amounts and reasons (lots of HOs can indicate ping
pong) Make
- Check adjacency parameters corrections
- Check HO adjacency statistics (HO failures): Are there some
some adjacencies with significant failure percentage? => if yes Record to log
then identify the reason (incorrect HO parameter, poor level...)
- Possibly field testing to understand the problem

8 © NOKIA
 Some troubleshooting examples

9 © NOKIA
 Transmission / HW failures
• BCT03, 184P20472
• DCR 69.7% ------------------------------------------------------------------------------
- BTS ALARMS
• Raw drops 30 ------------------------------------------------------------------------------

HOLD
ALARM START TIME TIME
ALM_ID ALARM NAME ALARM CANCEL TIME sec
---------- ------------------------- ----------------- --------
7767 BCCH MISSING 03-09-03 33
08:31:21
03-09-03 08:31:54

7767 BCCH MISSING 03-09-03 27

14:26:57
03-09-03 14:27:24

BCCH missing is not a reason, but a consequence of something else

going wrong (Transmission failure, TRX/BTS failure….) Can also occur
due to maintenance or parameter changes…
• HW/Transmission related alarms: TMA failure, TRX failure, PCM failure,
LAPD failure…

10 © NOKIA
 Poor coverage
• BCT03, 319G10653
• DCR 5.5% TCH drop out ratio ......................../dcr_3i 5.53 % 5.29 %

. Drop call ratio components:

• Raw drops 123 .
.
TCH_RADIO_FAIL ......./c1013
TCH_ABIS_FAIL_CALL .../c1084
4.90
0.27
%
%
5.29
0.00
%
%
. TCH_A_IF_FAIL_CALL .../c1087 0.00 % 0.00 %
. TCH_TR_FAIL ........../c1029 0.00 % 0.00 %
. TCH_LAPD_FAIL ......../c1046 0.00 % 0.00 %
. TCH_BTS_FAIL ........./c1047 0.00 % 0.00 %
. TCH_USER_ACT ........./c1048 0.00 % 0.00 %
. TCH_BCSU_RESET ......./c1049 0.00 % 0.00 %
. TCH_NETW_ACT ........./c1050 0.00 % 0.00 %
. TCH_ACT_FAIL_CALL ..../c1081 0.00 % 0.00 %

. TCH_RF_OLD_HO ......../c1014 0.13 % 0.00 %

. TCH_ABIS_FAIL_OLD ..../c1085 0.00 % 0.00 %
. TCH_A_IF_FAIL_OLD ..../c1088 0.22 % 0.00 %
. TCH_TR_FAIL_OLD ....../c1030 0.00 % 0.00 %

Drops per erlang, before re-establishment /dcr_10 1.46 1.09

Drops per erlang, after re-establishment /dcr_10b 1.34 0.98

Drops purely for radio

reason

11 © NOKIA
 Rx lev UL UL UL UL UL UL UL UL
TRX range q0 q1 q2 q3 q4 q5 q6 q7

Poor coverage id
----
9
(dBm) (%@) (%) (%) (%) (%) (%) (%) (%) SAMPLES
------ ------ ------ ------ ------ ------ ------ ------ ------ ----------
-109 18.9 2.9 3.0 2.9 2.6 2.9 2.2 1.6 15508
-105 21.8 0.4 0.3 0.2 0.0 0.0 0.0 0.0 9529
-95 32.5 0.1 0.2 0.1 0.0 0.0 0.0 0.0 13802
-85 6.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2874
BCCH TRX -74
-47
0.6
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
246
0
UL ****
sum
------ ------ ------ ------ ------ ------ ------ ------
80.5 3.3 3.5 3.3 2.7 2.9 2.2 1.6

10 -109 9.2 2.9 3.4 3.8 4.0 3.8 2.7 2.0 178238
-105 18.6 1.1 0.9 0.7 0.4 0.2 0.1 0.0 123130
-95 38.5 0.4 0.4 0.4 0.2 0.1 0.0 0.0 223615
Hopping TRX -85
-74
5.8
0.3
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
33153
1909
UL ****
-47 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
------ ------ ------ ------ ------ ------ ------ ------
9

sum 72.5 4.4 4.7 4.9 4.6 4.0 2.8 2.1

Rx lev DL DL DL DL DL DL DL DL
TRX range q0 q1 q2 q3 q4 q5 q6 q7
id (dBm) (%) (%) (%) (%) (%) (%) (%) (%) SAMPLES
---- ------ ------ ------ ------ ------ ------ ------ ------ ------ ----------
9 -109 0.0 0.1 0.2 0.3 0.5 0.7 1.0 1.5 15508
-105 2.8 1.0 1.1 1.0 1.1 1.3 0.6 0.4 9529
-95 34.2 2.2 2.2 2.0 1.6 2.7 1.7 0.9 13802
BCCH TRX -85
-74
29.0
5.1
0.8
0.0
0.6
0.0
0.5
0.0
0.3
0.0
1.9
0.2
0.1
0.0
0.0
0.0
2874
246
-47 0.2 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0
DL ****
sum
------ ------ ------ ------ ------ ------ ------ ------
71.3 4.0 4.1 3.8 3.6 6.9 3.5 2.9

10 -109 0.1 0.0 0.0 0.1 0.3 0.7 1.4 1.7 178238
-105 0.9 0.5 0.9 1.3 2.0 2.7 2.3 0.8 123130
-95 28.0 3.1 3.5 3.8 3.6 3.0 1.9 0.9 223615
Hopping TRX -85
-74
28.5
4.6
0.6
0.0
0.6
0.0
0.5
0.0
0.4
0.0
0.7
0.1
0.2
0.0
0.0
0.0
33153
1909
DL ****
-47 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0
------ ------ ------ ------ ------ ------ ------ ------
9

Network Doctor report 216 sum 62.2 4.3 5.1 5.8 6.3 7.1 5.9 3.4

has TRX level

RXQUAL/Level stats
12 © NOKIA
 Poor UL coverage (link imbalance)
Rx lev range (dBm) = upper boundary (level) of range
Too much bad quality in
Share
UL, cause poor UL signal
RX lev
range
UL
q0
UL
q1
UL
q2
UL
q3
UL
q4
UL
q5
UL
q6
UL
q7
in
range
level (40% of users at -
(dBm) (%) (%) (%) (%) (%) (%) (%) (%) (%) 109 to
------ ------- ------- ------- ------- ------- ------- ------- ------- -------
-109 13.74 3.35 3.78 3.90 3.86 3.95 3.56 3.36 39.50 -110 range) .
-105 22.00 0.69 0.50 0.34 0.11 0.04 0.01 0.01 23.70
-95 32.61 0.10 0.20 0.16 0.03 0.02 0.01 0.01 33.14
-85 3.36 0.00 0.01 0.02 0.00 0.00 0.00 0.01 3.40 DL levels OK. Why the
-74
-47
0.24
0.01
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.24
0.01
imbalance?
------- ------- ------- ------- ------- ------- ------- ------- -------
sum 71.96 4.15 4.49 4.42 4.00 4.01 3.58 3.38 100.00
=> something may be
Share
RX lev DL DL DL DL DL DL DL DL in significantly wrong in UL.
range q0 q1 q2 q3 q4 q5 q6 q7 range
(dBm) (%) (%) (%) (%) (%) (%) (%) (%) (%) (RF path, TRX HW, MS TX
------ ------- ------- ------- ------- ------- ------- ------- -------
-109 0.14 0.00 0.01 0.03 0.11 0.19 0.32 0.45
-------
1.25
power settings, UL PC
-105
-95
0.36
24.72
0.20
1.52
0.32
1.75
0.47
2.00
0.67
2.04
0.67
2.07
0.64
1.12
0.28
0.58
3.61
35.80
settings, UL/DL goes to
-85 38.61 1.03 0.97 0.76 0.53 1.35 0.22 0.13 43.60 wrong antenna…?)
-74 13.68 0.17 0.13 0.08 0.04 0.42 0.04 0.06 14.63
-47 0.96 0.01 0.00 0.00 0.00 0.06 0.02 0.05 1.11 If no reason found, then
------- ------- ------- ------- ------- ------- ------- ------- -------
sum 78.47 2.94 3.19 3.34 3.39 4.77 2.34 1.55 100.00 TMA is needed.

13 © NOKIA