Month Microwave Link LTK Fiber Passive Failure CARE McBSC CARE MPLS Juniper MX104

Jan 5 2 1
Feb 2 2 2
Mar 1 1 2 2 3
Apr 2 1 4 1
May 2 4
June 5 5 3 4 2
July 4 7 1 1
Aug 10 9 11 1
Sept 5 2
Oct 3 10 2 2
Nov 3 1
Dec 2 5 1 1
Jan'17 3 6
Feb'17 6 4 1
March '17 1 10 6 4
April '17 2 3 4
Grand Total 44 73 47 16 1 6

Monthly Cou
35

30

25
25

20 19 19

15
13
10
10 9
8

0
Jan Feb Mar Apr May June July

Monthly BUCKETIZ
 Monthly BUCKETIZ

10

6
3
5
3 4
2
1
2 6
3 10
4
5
2
6
10 9 1
1
2
4 7 11
5 4
5 1
2 3 2
2 4 1
1 4 1 1 1
1
2 1 4 1 2 1
1 2 2 2 1
1 1
5 2 2 1 1 4
2 1 2 1 3 3 3 2 1
r
n
k er re SC LS 0
4
te BN SR D
M
P
Li Fi
b l u cB M X
1 u P C A
e ai M M Ro M cy S
av K F E RE er 0 RE en TS A
w LT v e
AR
A 0 A g C
ro s si C
C
n
ip X
6 C er
ic Pa C Em
M Ju
ei /
aw E //
u
-H
N
A
R
IP
April '17 March '17 Feb'17 Jan
 E///
IPRAN -Huawei CARE MPBN Emergency TSS ADM CARE RNC WANANCHI
CX 600 Router CSR
3 1 2
1 1
2
1 1
1
4
1 2 1

3
2 1

1
1
1

5 5 7 8 3 2

Monthly Count of Big Outages Jan 1st '16- April 23rd '17

31

25

20
19

12
11
10
9
6

June July Aug Sept Oct Nov Dec Jan'17 Feb'17

Monthly BUCKETIZED Count of Big Outages Jan 1st '16 -April 23rd '17
Monthly BUCKETIZED Count of Big Outages Jan 1st '16 -April 23rd '17

1 1
1
2 1
1
1 4 1 1 1
2 1 2 2 2 1 1
1 1 1 1 1 1 1 1 1 1
C I a e
SR M N H M
ny SC M LU ps SW 2
G er
C A
D R C CO Ke
B D fA La
_ b IO
cy E N A x W B A
N Fi
SS AR
A SE ss Fl
e D LS ty IU R W
en T C AN e LC P
vi
n
g
cc RE P M to
er W
A A K t li
m C ac So
ed
n
l an
P

'17 Feb'17 Jan'17 Dec Nov Oct Sept Aug July June May Apr Mar Feb
 KPLC Planned activity
SEACOM Access Kenya CARE Flex BSC MPLS fALU
DWDM Lapse

1 1

1 1
1
1 1

2
1 1
3 1 3 1 3 2

26

12 12
11

Jan'17 Feb'17 March '17 April '17

 2
1 1 1 1 1 2
G C 0 k
2 er C 2
0
li
n er
b IO b
A
N Fi W X-
4
se Fi
R n E ri ge
o r p n
l it e
te
r ra
ip O
So n En
Ju TL
J

Apr Mar Feb Jan

IUB_SW RAN 2G Soliton Fiber WIOCC Juniper EX-4200

1 1 1

1 1 1 1 1
JTL Enterprise link Orange Fiber Grand Total

19
8
13
10
9
25
19
31
10
20
2 6
9
11
12
2 26
1 12
1 4 240
 Start Time End Time
Start Date End Date
S.No OPCO Months (HH:MM (HH:MM
(DD/MM/YY) (DD/MM/YY)
AM/PM) AM/PM)

40 KE Apr 2-Apr-16 06:43 PM 2-Apr-16 07:08 PM

41 KE Apr 13-Apr-16 12:24 PM 13-Apr-16 01:10 PM

42 KE Apr 13-Apr-16 5:50 PM 13-Apr-16 6:55 PM

43 KE Apr 16-Apr-16 11:27 PM 17-Apr-16 12:21 AM

44 KE Apr 17-Apr-16 03:00 AM 17-Apr-16 03:11 AM

45 KE Apr 17-Apr-16 10:18 PM 18-Apr-16 02:20 AM

46 KE Apr 18-Apr-16 03:26 PM 18-Apr-16 6:26 PM

47 KE Apr 23-Apr-16 09:45 PM 24-Apr-16 12:45 AM

48 KE Apr 26-Apr-16 5:30 AM 26-Apr-16 10:00 AM

49 KE Apr 28-Apr-16 4:13 PM 28-Apr-16 5:45 PM

227 KE April '17 4/3/2017 3:55 PM 4/3/2017 6:20 PM

228 KE April '17 4/4/2017 5:59 AM 4/4/2017 6:20 AM

229 KE April '17 4/10/2017 11:15 AM 4/10/2017 11:17 AM

230 KE April '17 4/10/2017 9:20 AM 4/10/2017 11:41 AM

231 KE April '17 4/10/2017 11:37 AM 4/10/2017 2:06 PM

232 KE April '17 4/10/2017 9:45 PM 4/10/2017 10:50 PM

233 KE April '17 4/15/2017 10:32 PM 4/16/2017 9:02 AM

234 KE April '17 4/18/2017 4:16 AM 4/18/2017 4:25 AM

235 KE April '17 4/18/2017 10:38 AM 4/18/2017 10:42 AM

236 KE April '17 4/18/2017 1:32 PM 4/18/2017 1:34 PM

237 KE April '17 4/22/2017 3:47 AM 4/22/2017 5:30 AM

238 KE April '17 4/22/2017 4:21 PM 4/22/2017 5:11 PM

Outage Duration
(In Problem Details and RFO/RCA Big Outage Bucket
Hours:Mins:Sec)

23_2G SITES OF BGMBSC2 WERE DOWN

RFO: POWER FAILURE AT MALAKISI. The area was experiencing power
fluctuatons which in turn hang the IPMU. Services restored after the
00:00:25 Passive Failure
generator was started manually. Follow up ongoing to close the Ring which
is open due to LOS blockage for one site

GB LINKS WERE DOWN FOR ELDBSC1(3 OUT OF 3) AND KTLBSC1FLX (1 OUT

OF 2) AND KSIBSC1FLX (1 OUT OF 4)
RFO: DWDM failure due to KPLC power failure in Lessos. Traffic restored
00:00:46 Passive Failure
after KPLC restored power. Protecton at the tme was down
(Susian<>Gitthia) due to planned operaton

25_3G SITES OF VPKRNC2 WERE DOWN

RFO: LTK Failure in Kiambaa. LTK restored traffic. Detailed RCA awaited from
30:01:04 LTK Fiber
LTK

21_2G SITE OF KTIBSC1 WERE DOWN

RFO: RFO: This was due to power failure at Kitui Hub site based on alarms
00:00:54 Passive Failure
logs analysis

GB LINKS(3 OUT OF 3) WERE DOWN FOR KTIBSC1

RFO: This was due to power failure at Kitui Hub site based on alarms logs
00:00:11 Passive Failure
analysis

26_2G SITES OF KSMBSC2 WERE DOWN (60% VOICE TRAFFIC DROPED IN

KSMBSC2 FROM NKUMSC5)
30:04:08 RFO: Faulty TSS card port. Traffic was restored after moving traffic from TSS ADM
faulty STM-1 port 1 to port 4. Card to be replaced

50% GI Data Traffic drop for 180mins due to 93 % PDP actvaton failure
SDP1A -(3-2012) is isolated from PKSHLR2 & NKUHR1
SDP2B -(3-2013) is isolated from VPKSTP1, PKSSTP1 & NKUHLR2
PKSA6 -AIR (3-2208) is isolated from NKUHLR2,PKSHLR2 & PKSHLR3
30:03:00 RFO: Ericsson Emergency CSR team recommended restart of m1s3 Board on E/// Emergency CSR
SAPC to resolve the problem. This board was observed to have had high CPU
load.
 KTIBSC1(KITUI) WITH 65-2G SITES LOST 100% TRAFFIC
COMPLETE GB LINKS (3/3)OF KTIBSC1 IS ALSO DOWN.
SUPERVISION LOST FOR KTIBSC1

RFO: Failure attributed to failure of FPMR IDUs at Kitui_Town; One facing

30:03:00 Microwave Link
Tulia<> Kitui_Majengo and the other Kwavonza<>Chuluni, hence breaking
the ring. Also affected was ODU facing Kitui<>Kwavonza by which
supervision traffic is riding on. The Radio Equipment were struck by
lightning. Traffic restored after replacement of faulty IDUs.

57_2G SITES OF PKSBSC6 (PARKSIDE ) WERE DOWN

RFO: Hang BCXUs and ETME. BCXU cards and ETME were restarted to
0:04:30 restore services. Emergency CARE tcket (NA05913454) was opened to CARE McBSC
ascertain why the cards hang.

65-2G Sites and 38-3G Sites in Nairobi Region were intermittently failing.
RFO: Ruaraka<>Runda & Village Market<>Forest road Ceragon link failures
0:01:32 were affected by rain fading. Services restored after heavy rains subsided. Microwave Link
Open Ring to be closed

KSMMG31 was isolated from NKUMSC5 and PKSMSC5

RNGBSC2,KSMBSC2,KSIBSC3 were isolated from NKUMSC5
SFCM15(2-12636) SFCM23(2-12696) was isolated from NKUMSC5
72_3G site of NAKURU_RNC were down
Complete GB links were down of KISBSC3 (4 Out of 4),RNGBSC2 (3 out of 3),
KSMBSC2 (1 out of 1)
RFO: LTK dual fiber cut Molo<>Kericho and Tivoli-kakamega link (Fiber cut
by COVEC road contractors at Mamboleo while cutting the service duct for
KPLC, Splicing done at Tivoli-kakamega to restore the services). The LTK
2:25 LTK Fiber
ADM at Kericho Town had failed and was restored after onsite interventon
by LTK. Kericho equipment was down/unmanageable. This was because of 2
issues: Faulty patch cord for Molo<>Kericho link and Looping on the LAPD
configuratons on the 1642s and 1660s between Kericho and Kisii was
affectng supervision of Kisii, Kericho, Sotk 1660s and the corresponding
1642s. Configuratons were changed and patch cord replaced. Segment was
restored. The DWDM link had being cut at Mamboleo and was also spliced
to restore services. Microwave link routng earlier affected at Sitotwet also
restored to ensure full Nodes redundancy.

22_2G sites of MDIBSC2 and 2_3G sites of MSARNC2 were fluctuatng

RFO:Microwave link ductng issue between MWENDA_POLE to Ngomeni
0:21 due to bad weather. Modulaton Scheme Lowered for the 2 problematc Microwave Link
Hops Haraa-Garsen (32 QAM), Mwendapole-Ngao (64QAM).On capacity
upgrade requirement Airtel shall plan to change the antenna. Link with
Airtel Planning for redesign.

23_3G sites of PKSRNC1 and 2_3G sites of VPKRNC2 were down.

0:02 RFO:IDU restarted at Kitengela due to Zero attery autonomy. Batteries to be Passive Failure
replaced.
 100% traffic dip was observed in GRX_via Easy (8.5 MBPS to 0 MBPS)
RFO: Primay link through Easy flapped while Secondary link through Seacom
2:21 MPLS
had failed on 9th. April. There is a faulty cable that was replaced on MPLS
Router to restore traffic.

51_3G sites of NAKURU_RNC and 2_2G sites of NKUBSC3 were down.

Gb links were down of RNGBSC2 (2 out of 3), KISBSC3 (3 Out of 4)
1 out of 2 HSL(SLC_1) signaling link was down in ELDMG31 towards
2:29 BGMBSC2 LTK Fiber
1 out of 1 HSL Link was down from NKUMG31 towards KSIBSC3.
RFO: Dual fiber link from Molo<>Kericho SDH and Bungoma<>Kakamega
SDH & DWDM Links were Down. Fiber splicing was done to restore services

45_2G sites of MDIBSC2 and 6_3G sites of MSARNC2 were fluctuated.

Gb links of MDIBSC02 were fluctuated (1 out of 1)
1:05 79% call volume drop was observed in MDIBSC2 Passive Failure
RFO:IDU failure caused by faulty fan unit at Malindi Town. The fan unit was
replaced to restore services.

• 100% Data traffic dip was observed in GRX_viaEasy , GRX_viaSeacom,

Tanzania_GRX, Rim Traffic_Paris-BB_Primary and Rim Traffic_Amsterdam-
BB_Primary
• 23% Data Traffic dip is observed in Total-GI
10:30 SEACOM
RFO: Seacom had failed earlier due to a fault in terestrial. WIOCC errors in
djibout . DWDM NBI-MSA Flaps due to errors on cable while protecton on
orange was degraded . Orange degradaton resolved. cable stable and
under observaton.

26_2G Sites of MDIBSC2 were fluctuatng

RFO:Degraded RSL on Mambrui<>Kibokoni FPMR link. Mambrui Tower
0:09 Passive Failure
condemmed hence preventng detailed investgaton on achoring. Pending
with Eaton
37_2G sites of IKUBSC2 were down
0:04 RFO:Failure caused by Passive team working at Meru_Town. Passive Team re Passive Failure
sensitzed on site access to site process
24_3G site of PKSRNC1 were down
0:02 RFO: Human error in Kitengela. FOE accidentally loosened the ethernet Microwave Link
cable.

Multple corporate E1s were down towards JTL down from PKSMSC5. (Total
count 30)
1:43 JTL Enterprise link
RFO: Transmission fault caused by commercial power outage and backup
generator failed to automatcally switch. Manual DG switching was done.

VPKBSC2(3-1266) was fluctuatng from PKSMSC5.

Destnaton towards VPKBSC2 was fluctuatng towards VPKSTP1 (2-12312)
from PKSMSC5
0:50 LTK Fiber
RFO: LTK Fiber Transmission link was flapping PKSD<>KRBG<>VPK. There is
no smooth change over from Microwave to Fiber and vice versa. OSPF area
ID between MX and MPBN needs to be changed
 Repetitive
Owner Status
Failure?

Airtel/Eaton No Closed

Airtel/Eaton Yes Closed

Gachau No Closed

Airtel/Eaton No Closed

Airtel/Eaton Yes Closed

Orutwa No Closed

Gunjan No closed
Orutwa No Closed

Gachau Yes Closed

Orutwa No open

Gachau Yes Closed

Orutwa No Open

Gachau No Open
Gachau No Closed

Gachau Yes Closed

Gachau No open

Gachau No Open

Gachau Yes open

Gachau Yes open

Orutwa No Open

Gachau Yes open

Gachau Yes Open

In summary, assuming we have 1500 sites throughout the week, we will be Red, Amber Green when the count is

·         Red:- Big outage count Greater than (1500/500)*2=6

·         Amber:- Big outage Count greater than (1500/500)*1=3 and Less than or equal to (1500/500)*2=6 i.e ( 4 to
·         Green:- Big Outage count less than or equal to (1500/500)*1=3

In summary, assuming we have 1500 sites throughout the week, we will be Red, Amber Green when the count is

·         Red:- Big outage count Greater than (1500/500)*2=6

·         Amber:- Big outage Count greater than (1500/500)*1=3 and Less than or equal to (1500/500)*2
·         Green:- Big Outage count less than or equal to (1500/500)*1=3
 when the count is

equal to (1500/500)*2=6 i.e ( 4 to 6 Counts)

ber Green when the count is

ss than or equal to (1500/500)*2=6 i.e ( 4 to 6 Counts)