Professional Documents
Culture Documents
AR26
AR26
New AR:
History | States | Assignment | Timetracking
Add Attachment
AR Text Search | More | CARES Home
Assistance Request 1-5778563
Customer Ticket n/a
Description
Current Aug 24: The Fix built in LR14.2 EP5 is GA, awaiting closure
Summary confirmation.
Outage No
Severity 2
Priority 1
Sub-Type Software
Category Software
Internal Yes
Assignment Events
Ask Alcatel-Lucent
Product
Product 9370 RNC
Version LR13.3.W
Product Location
Site ETISALAT ABU DHABI
Contact
Name Karim IBRAHIM ABD EL NABY
Dates
Occurred 16-Jun-2015 16:58 Time now 29-Aug-2015 10:41
(GMT+4)
Targets for Restore & Resolve have been extended by 5 days 19 hours
to account for Pending time
Entitlement
Agreement 242895 (OXIA 627085)
People
Owner TSCd-WLS-WCDMA-IN : mohmadi
Submitter aiovu
Description
Attachments
[empty]
2. 16-Jun-2015 17:09
mohmadi
Update to Current Summary: June 16: Awaiting Receive Log Files / Technical
Info
Cc: INDIA-WCDMA-UTRAN-BE
Hi Karim,
I received one more AR for this RNC, please share some insight on the
issues
2) Was there any activity been performed on this RNC during that time?
1) RNClogcollection.
2) RNC Healthcheck.
3) HFB logs.
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
5. 17-Jun-2015 10:14
mohmadi
Update to Current Summary: June 17: Awaiting Receive Log Files / Technical
Info
Cc: INDIA-WCDMA-UTRAN-BE
Hi Karim,
I received one more AR for this RNC, please share some insight on the
issues
2) Was there any activity been performed on this RNC during that time?
[KARIM] NO activity but the RNC was not stable of stream of TMUs reset.
[KARIM] NO
1) RNClogcollection.
2) RNC Healthcheck.
3) HFB logs.
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Hello Mohamdi,
check the ARs- 1-5774187, 1-5774794 ,for more details for the issue on the
RNC
Thanks,
--
BR,
Karim Ibrahim
Dears,
Please update.
Thanks,
--
BR,
Karim Ibrahim
To: IBRAHIM ABD EL NABY, Karim (Karim)** CTR **; THOTA, Praveen-K (Praveen-
K)**
CTR **
From Previous ARs logs , I did not found any info related to this
Propagation
delay counters.
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Dear Mohamdi
All logs as I mentioned uploaded to the server and attached the required
KPIs
report,
Thanks,
--
BR,
Karim Ibrahim
SW Version: 13.3.7
1. Detailed Description
There was an outage on this RNC on 13th June (AR 1-5774187) and after that
this
1-5774187, 1-5774794).
/home/wcdma/traces/ETISALAT/1-5774187
/home/wcdma/traces/ETISALAT/AR1-5774794
New logs are not available, only Counter details are available on path
/home/wcdma/Etisalat_UAE/1-5778563
As above.
3. All the steps taken before the issue observed by the engineer, who
reported the
AR
N/A
This RNC is bit critical as there was outage and Audit failure and many
activities
were carried out during the earlier AR investigations and hence from
earlier logs
could not find out why only this counter is impacted as new logs are not
5. Current Status
Critical.
7. Attachments
Escalation Note
Cc: INDIA-WCDMA-UTRAN-BE
Hello Paul,
This is for your information that we have escalated this AR to your
workgroup
Old Logs for referral are available on FTP Server 135.240.25.160 wftp/wftp
under
path:
/home/wcdma/traces/ETISALAT/1-5774187
/home/wcdma/traces/ETISALAT/AR1-5774794
New logs are not available, only Counter details are available on path:
/home/wcdma/Etisalat_UAE/1-5778563
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Dear Karim,
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Thanks,
--
BR,
Karim Ibrahim
Hi,
Do you have the name of the propagation delay counters that are zero since
the
outage?
Thanks,
Rameez
17. 24-Jun-2015 10:19 ALCATEL-LUCENT PROPRIETARY
mohmadi
-----Original Message-----
From: owner-india-wcdma-utran-be@LIST.ALCATEL-LUCENT.COM
[mailto:owner-india-wcdma-utran-be@LIST.ALCATEL-LUCENT.COM] On Behalf Of
Cc: INDIA-WCDMA-UTRAN-BE
Hello Rameez,
Please find the attachment for name of the propagation delay counters
received
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Hi,
Can I please have the raw counter files for day when counters were working
and a
/opt/nortel/data/utran/observation/stats
Thanks,
Rameez
Update to Current Summary: June 25: raw counters requested to site team,
awaiting
the same.
VS_DistPropDelayPerRange_DelayGe129Lt257Chips(U1041_17)(Event)
VS_DistPropDelayPerRange_DelayGe15Lt62Chips(U1041_13)(Event)
VS_DistPropDelayPerRange_DelayGe257Lt384Chips(U1041_18)(Event)
VS_DistPropDelayPerRange_DelayGe384Lt579Chips(U1041_19)(Event)
VS_DistPropDelayPerRange_DelayGe579Lt767Chips(U1041_20)(Event)
VS_DistPropDelayPerRange_DelayGe62Lt129Chips(U1041_14)(Event)
VS_DistPropDelayPerRange_DelayGe6Lt15Chips(U1041_12)(Event)
VS_DistPropDelayPerRange_DelayLt6Chips(U1041_4)(Event)
Also please share the raw counter files for day when counters were working
and a
/opt/nortel/data/utran/observation/stats
/home/wcdma/Etisalat_UAE/1-5778563
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Dear Mohamdi
Thank, you check and download the raw counters directly from ramses as it
will
Thanks,
--
BR,
Karim Ibrahim
the same.
Hello Mohamdi
Any progress?!,
Thanks,
--
BR,
Karim Ibrahim
Hi Karim,
The raw counters downloading has been done and uploading is in progress
though the
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Dear Mohamdi,
Thanks,
--
BR,
Karim Ibrahim
Yes Karim,
I understand your concern, hence we use to request the logs to upload from
site
team to save time, as for us remotely it sometime takes time due to slow
network
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
TEC update.
Hi Rameez,
As required the raw counter files for day when counters were working and a
day when not working (ie before and after upgrade date 13 June) have
uploaded on
Thanks,
Mohmadi
Hi,
I was comparing the before and after raw counter files and I noticed that
some of
the "after" raw counters files are of zero size and some of the files are
very
small. Were you able to collect and copy the "after" zip file correctly and
nothing got corrupted? If you copied the files ok then I might be expecting
an
issue on the WNMS when WCE and 9370s are both present. We had a similar
issue
AR1-56696884 where some of the counters were not getting pegged when we had
both
9370 and WCE on the same WNMS. I still need more evidence to prove that.
For now, can you do (ls -l) in the stat directory on the WMS for a few days
before
Thanks,
Rameez
29. 03-Jul-2015 13:56 ALCATEL-LUCENT PROPRIETARY
mohmadi
Hi Rameez,
Please find the (ls -l) output as requested. Kindly have a check at your
side once
wms3g:/opt/nortel/data/utran/observation/stats/20150607/RNCCN-RNC231 <159>
(oamops) % ls -l
total 523008
wms3g:/opt/nortel/data/utran/observation/stats/20150608/RNCCN-RNC231 <153>
(oamops) % ls -l
total 531824
wms3g:/opt/nortel/data/utran/observation/stats/20150609/RNCCN-RNC231 <118>
(oamops) % ls -l
total 538960
wms3g:/opt/nortel/data/utran/observation/stats/20150610/RNCCN-RNC231 <112>
(oamops) % ls -l
total 531392
wms3g:/opt/nortel/data/utran/observation/stats/20150611/RNCCN-RNC231 <106>
(oamops) % ls -l
total 524528
wms3g:/opt/nortel/data/utran/observation/stats/20150612/RNCCN-RNC231 <124>
(oamops) % ls -l
total 478992
wms3g:/opt/nortel/data/utran/observation/stats/20150613/RNCCN-RNC231 <130>
(oamops) % ls -l
total 403232
wms3g:/opt/nortel/data/utran/observation/stats/20150614/RNCCN-RNC231 <136>
(oamops) % ls -l
total 426688
wms3g:/opt/nortel/data/utran/observation/stats/20150615/RNCCN-RNC231 <142>
(oamops) % ls -l
total 432704
wms3g:/opt/nortel/data/utran/observation/stats/20150616/RNCCN-RNC231 <148>
(oamops) % ls -l
total 448528
wms3g:/opt/nortel/data/utran/observation/stats/20150620/RNCCN-RNC231 <164>
(oamops) % ls -l
total 433248
wms3g:/opt/nortel/data/utran/observation/stats/20150701/RNCCN-RNC231 <171>
(oamops) % ls -l
total 472224
Also have attached the session login (session login 3 July.txt) in the AR
for your
reference.
If required you can login through Ramses -> United Arab Emirates - Etisalat
->
Dubai -> Radio Network 3G -> W-NMS -> SSH Main 1 (10.238.200.132)
Thanks,
Mohmadi
30. 03-Jul-2015 16:49 ALCATEL-LUCENT PROPRIETARY
mohmadi
Hi,
You can try this workaround and see if that fixes the issue.
Workaround for this issue is to restart the GPO server and observe that at
least 1
successfully processed XML file for 9370 is generated with the expected
size. If
Solution:
WCE threads. Thus, the relevant mediation context is retrieved for the
If this doesn't work then I will have to dig deeper to find out why its not
Kind Regards,
Rameez
32. 06-Jul-2015 10:06
mohmadi
update.
Hello Karim,
TEC has provided following workaround, can you please try and see if that
fixes
Workaround for this issue is to restart the GPO server and observe that at
least 1
successfully processed XML file for 9370 is generated with the expected
size.
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
why is it required and its RC. Awaiting more info from TEC.
Dear Mohamdi,
Thanks , Please elaborate more based on what we will apply this WA?!, is it
the
same case of below AR?!, If yes, the WA already applied and confirmed by
Ajay that
its OK now,
// AR 1-5755928 /// Critical /// 9353 WMS /// Emirates Telecommunications
Thanks,
--
BR,
Karim Ibrahim
Hi Karim,
So you mean to say that now after performing the below WA the Propagation
delay
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Dear Mohamdi,
same this WA already applied and confimed by Ajay that the CMXML issue
solved,
Please elaborate to us more the RC of the issue and why you need to apply
this WA.
Thanks,
--
BR,
Karim Ibrahim
Hi Rameez,
Thanks,
Mohmadi.
Root Cause:
The issue occurs with the combination of 9370 and 9771 XDR PM file
processing.
For 9370 and WCE because the set of counters are different between them.
Let us
In order to get the correct applicable mediation context for 9370 and WCE,
thread
A common thread specific key is used between 9370 and WCE threads. The
value which
is bound to the key is the applicable mediation context for the thread.
The issue of "out of bound exception" occurs while processing XDR files of
9370
because there is a bug due to which the thread processing the 9370 files
retrieves
the mediation context for WCE instead of mediation context for 9370. This
causes
the particular localization due to which the size of the processed XML
files is
The hotfix built from base LR14.2 EP5 has been verified by ISS.
Thanks,
Rameez
Update to Current Summary: July 8: ROOT cause shared to site team, awaiting
closure confirmation.
Hi Karim,
Root Cause:
The issue occurs with the combination of 9370 and 9771 XDR PM file
processing. The
applicable mediation context for 9370 and WCE, thread specific data is
maintained.
A common thread specific key is used between 9370 and WCE threads. The
value which
is bound to the key is the applicable mediation context for the thread. The
issue
there is a bug due to which the thread processing the 9370 files retrieves
the
mediation context for WCE instead of mediation context for 9370. This
causes the
wrong offset to be retrieved from the context information. When the above
error is
the particular localization due to which the size of the processed XML
files is
The hotfix built from base LR14.2 EP5 has been verified by ISS.
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Update to Current Summary: July 8: Root cause shared to site team, awaiting
closure confirmation.
Update to Current Summary: July 8: Root cause (The hotfix built from base
LR14.2
To: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **; IBRAHIM ABD EL NABY, Karim
(Karim)**
CTR **
Hi
Best Regards
Update to Current Summary: Aug 8: Root cause (The hotfix built from base
LR14.2
Hello Abdessamad,
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Update to Current Summary: Aug 3: Root cause (The hotfix built from base
LR14.2
Hello Mohamdi
BR
Karim
To: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **; IBRAHIM ABD EL NABY, Karim
(Karim)**
CTR **
just by ISS
Best Regards
Cc: INDIA-WCDMA-UTRAN-BE
Hello Rameez,
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Update to Current Summary: Aug 11: Root cause (The hotfix built from base
LR14.2
Cc: INDIA-WCDMA-UTRAN-BE
Hello Rameez,
Please share if any update on the delivery of LR14.2 EP5, and when is it
GA
availability?
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Update to Current Summary: Aug 12: Root cause (The hotfix built from base
LR14.2
To: CUTLERYWALA, Mohmadi (Mohmadi)** CTR **; KHAN, Rameez (Rameez)** CTR **
Cc: INDIA-WCDMA-UTRAN-BE
Hi,
I am from RNC team and asked about this availability.
What is the CR number for this AR and who is the designer worked on this?
Regards,
Girish
Update to Current Summary: Aug 13: Root cause (The hotfix built from base
LR14.2
To: PUJAR, Girish (Girish)** CTR **; CUTLERYWALA, Mohmadi (Mohmadi)** CTR
**
Cc: INDIA-WCDMA-UTRAN-BE
Hi Girish,
The CR 01531738 was opened for this issue. There are other similar ARs 1-
5662780
and 1-5669684 with the same issue. They mentioned that the hotfix will be
provided
in WK29 in LR14.2 EP load. Thats why Mohmadi was asking me when is the
delivery
Thanks,
Rameez
To: KHAN, Rameez (Rameez)** CTR **; CUTLERYWALA, Mohmadi (Mohmadi)** CTR **
Dear Rameez,
Further investigation into CR 01531738 has shown that this is not caused
by the
From a release point of view in absence of Alain, only ERIC MORICELLY can
take a
call.
Regards,
Girish
Update to Current Summary: Aug 17: Root cause (The hotfix built from base
LR14.2
To: PUJAR, Girish (Girish)** CTR **; KHAN, Rameez (Rameez)** CTR **;
CUTLERYWALA,
Ks)** CTR **
Hi Rameez,
I closed CR 01531738 because AR 1-5662780 was not an RNC issue. The fix is
on the
Karen.
To: Lank, Karen Dianne (Karen)** CTR **; PUJAR, Girish (Girish)** CTR **;
KHAN,
Ks)** CTR **
Hello Rameez,
is it GA availability?
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
Update to Current Summary: Aug 20: Root cause (The hotfix built from base
LR14.2
Hi Mohmadi,
The GA load is available. Please check the below link and the attached
slide.
https://wcdma-ll.app.alcatel-
lucent.com/livelink/livelink.exe?func=ll&objid=68192182&objAction=browse&so
rt=name
Thanks,
Rameez
Hello Abdessamad,
The GA load is available. Please check the below link and the attached
slide.
https://wcdma-ll.app.alcatel-
lucent.com/livelink/livelink.exe?func=ll&objid=68192182&objAction=browse&so
rt=name
Let us know if any more query, else allow us to close the AR.
Thanks,
Mohmadi Cutlerywala,
Mobile:+91 9870327804.
closure confirmation.
Add to log
Resolution
The hotfix will be available for customers W29.
The hotfix built from base LR14.2 EP5 has been verified by ISS.