2018/11/6 Security Level:

Introduction to the Fault

Management Assistant (FMA)

www.huawei.com

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential

Content

• Background
• Key Function
• Scenario
• Detailed Function
• Accident Recovery SOP
• Download

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 2

 Background
1. Times of accident (include
non-quality) for 2010
reaches about 238, and
pressure of production line
is very higher
2. Restored time of accident
is too longer, and average
value is 136 minutes by
statistic in last year. Now,
in Canada, the accident
recovery SOP has been
deployed(use FMA tool),
and time decreases as 40
minutes.
3. Quickly restored accident
becomes a key task for
production
OMSTAR，InsightSharp，NIC，UMAT，PRESTAR……
HUAWEI TECHNOLOGIES CO., LTD.
1. Lacked method of location,
and inefficient for
maintenance
2. Various type of log. The
size of log is larger for
Longer Restored Inefficient download by hand and
Time of Accident Location time is longer
3. Generous information. It is
inefficient to gather
information of accident
Various Tools
Huawei Confidential Page 3
Function

• Accident Log Collect Fault

CHR&PCHR
Diagnosis
Quickly
• Effectual relationship
• Quickly Location Real-time
UKPI DashBoard
• Experience integration
Monitor
• Uniform maintenance
FMA
plane MML Performance
Comparison Browsing
• Convenience &Compariso
and Feature
Scan n
Alarm and
Operation
log Analysis

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 4

Scenario
Fault Assistant
Analysis
MML Comparison
Function
Feature Scan

FMA CHR&PCHR Analysis

Tool

Fault Diagnosis
Diagnosis Report

DashBoard 1.Phenomena
1）MML Scripts
2）Alarm Log 2.Result
PC 3）Performance Performance Analysis 导出
3.Workaround
4）Operation
5）CHR&PCHR 4.Information
1. Accident Log 。。。。。。 Alarm Analysis
Collection
2. Performance Operation Analysis
Comparison Online
3. Real-time UKPI
Monitor

Online
Function

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 5

Scenario

FMA提供 1. Accident log collection

Scenario 的解决方案 2. Fault diagnosis and workaround
3. Dashboard, associate performance, alarm
Accident happens in commercial and operation log
network 4. Performance browsing and comparison
online, recognize fault point quickly
5. Alarm and operation log analysis
6. CHR&PCHR analysis

1. CHR&PCHR analysis
Degraded KPI of network 2. Performance browsing, and TOPN Cell
3. MML Comparison

Safeguard for holiday or cell Real-Time UKPI monitor

Which feature is opened? Feature & License Scan in MML scripts

What is network? MML parsing and exported key information

Fault Analysis and Location MML parsing, alarm, performance, operation log,
CHR log Analysis, MML comparison
……

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 6

 Detailed Function Introduction (1)
Functio Detailed Information Remark Level
n
Accident 1. Accident log collection, and 1. The function has been ★★★★★
divide for two batches. The size of deployed with accident
Log the first batch is less than 5M, and recovery SOP for global
Collectio 30M for second batch, operators (about 53 operators
n 2. Provide to collect transmission has been used)
log 2. During the accident, it takes
3. Provide to collect SOP log about 10 minutes to feedback
the accident log

1. Collect expediently, and do not

worry about missing log
2. The size of collection log is
small, and easy to deliver to HQ
by Email

Collection data

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 7

 Detailed Function Introduction (2)
Functio Detailed Information Remark Level
n
Fault 1. Associate the alarm, performance and operation log 1. Cover about 40~50% ★★★★★
by MML script Scenario of accident
Diagnosi 2. Provide the key KPI and alarm statistic of different
2. The function has
s SPU and INT board
3. Provide the visual plane of MML script, and been deployed with
relationship information of cell, link and neighboring accident recovery
cell SOP for global
4. Based on the various original rules, and draws a operators (about 53
workaround operators has been
used)

1. Get conclusion
quickly
2. Classification
3. Impaction
clearly
The FMA has been deployed in
accident SOP of Canada

Run the FMA to check whether result is right or not

when accident occurs

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 8

 Detailed Function Introduction (2)
MML Script Parsing and Visual Display

Zoom Figure of Subrack

Extract MML Scripts

It is not painful to extract MML scripts

of Node and information link now!
Visual display for Plane

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 9

Detailed Function Introduction (3)
Function Detailed Information Remark Level
DashBoar 1. Relationship display for performance, It can be used for ★★★★
alarm and operation analysis of accident,
d 2. The counter can be queried and drawn as and recognize the
curve and recognize the impaction of KPI impaction of alarm
3. Frequency of alarm statistic and chart and operation log for
accident.

1. This function is edge tool to analyze

the accident log. It is convenience to
browse the performance (KPI) ,
alarm and operator log, and
relationship with them.
2. If the SR of RRC or RAB is
deteriorated, this function can check
the alarms and operator log during
the worsen period or KPI.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 10

Detailed Function Introduction (4)
Functio Detailed Information Remark Level
n
Alarm 1. Alarm log parses and display quickly It has been used ★★★★
2. Filter, classification and highlight widely in
Analysis 3. Relationship between alarm and MML scrip, and maintenance, test
provide the SPU subsystem, port and Node and other
information for each fault alarm department
4. Statistic for alarm, provide the proportion of fault
alarm for SPU subsystem or port, and frequency of
alarm to analyze the accident log 1. Relationship between alarm
and MML scrip is a light
spot
2. Recognize the issue quickly,
and check whether the issue
in happened on SPU or
interface board

Frequency of Alarm

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 11

 Detailed Function Introduction (5)

Functio Detailed Information Remark Level

n
Operatio 1. Normal and BAK operation log It has been used widely ★★★★
browsing in maintenance
n Log 2. Filter department
Analysis 3. Priority of command (Critical/Normal)
4. Backup operator log to browse
The traditional accident is caused by
the wrong MML command easily, and
whether this issue is caused by
command or not?
Use the function and browse or filter
the commands quickly.

The backup operation log for several months ago can be analyzed by FMA

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 12

Detailed Function Introduction (6)

Functio Detailed Information Remark Level

n
Real-Time 1. Connect with OMU online, and get UKPI It has been used widely ★★★
UKPI file and User number information in safeguard for the
Monitor 2. Chart to display the UKPI and user South Africa World Cup,
number information, convenience to monitor Asia Sport Game in
3. Cluster Cell to monitor hotspot cell Guangzhou, Hajj of
Saudi Arabia

The FMA can help you to

monitor performance of
system during the holiday

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 13

Detailed Function Introduction (7)
Function Detailed Information Remark Level
MML 1. Comparison for two scripts of different It has been used ★★★
RNC or different version of RNC widely in maintenance,
Compariso 2. Color to denote the results performance and other
n 3. Filter and extract department

1. Comparison for two

scripts of different/same
RNC or different version
of RNC(V2 and V9), and
display the difference
2. The function can be used
for degraded KPI caused
by wrong parameter
The result of comparison with two types

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 14

Detailed Function Introduction (8)
Function Detailed Information Remark Level
NodeB Convert the NodeB XML configuration It has been used ★★★
file to MML commands. The user needs to widely in maintenance,
XML2MML browse the XML file to confirm the performance and other
configuration by CME tool. department

XML configuration MML Commands

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 15

Detailed Function Introduction (9)

Functio Detailed Information Remark Level

n
Performa 1. Support browsing quickly for about 1. It has been used ★★
maximal 200 files, and take about 3 widely in
nce minutes maintenance, test and
Analysis 2. Normal KPI browsing, query, and other department
chart to display 2. The efficiency of
3. TOPN cell analysis, including access, analysis for about one
drop call week is more higher
4. Provide KPI analysis for cluster cell, than other tool, such
and counter query as OMSTAR,
5. Health check, and provide about 300 NASTAR
rules
6. Defined counter, support expression
and logical operation
7. Voice model

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 16

Detailed Function Introduction (9)
• Performance Analysis(to be)

1. FMA can analyze for about 200 performance file(1~2M zip) on normal PC with 2G
memory, and it takes about 1.2s to parse one file averagely.
2. Much experience has been integrated into FMA, and user can analyze TOPN cell,
heath check and voice model expediently.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 17

Detailed Function Introduction (9)
• Performance Analysis(to be)

1. TOPN Analysis

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 18

Detailed Function Introduction (10)

Function Detailed Information Remark Level

Performance 1. Different period of performance to compare The function has ★★
for same RNC and same or different version of been deployed in
Comparison RNC Canada
2. Draw the chart quickly for normal KPI
3. Collection performance files online

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 19

Detailed Function Introduction (11)

Function Detailed Information Remark Level

CHR&PCH 1. CHR&PCHR browsing quickly It has been used ★★
2. Classification of fault for CHR and PCHR widely in
R Analysis 3. Filter, filter by column value or filter by maintenance
condition department
4. Statistic for point code
5. Statistic for parameter

1. About 0~1s to parse one

CHR log file
2. About 2~3s to parse one
PCHR log file
3. About 0~1s to filter one
CHR/PCHR log file

Analyze and browse CHR&PCHR

log expediently and quickly, and
easy to locate KPI issue

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 20

Detailed Function Introduction (11)

• CHR&PCHR Analysis (to be)

The Fault Classification based on the CHR or PCHR log, and

analyze the KPI issue quickly by the function

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 21

Detailed Function Introduction (11)

• CHR&PCHR Analysis (to be)

The chart is shown as the trend of statistic for RRC attempts times. The
FMA can provide the other statistic, such as RAB attempts /Succ times,
or given condition
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 22
Detailed Function Introduction (11)
• CHR&PCHR Analysis (to be)

The chart is shown as the trend of CPU with second period

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 23

Detailed Function Introduction (11)

• CHR&PCHR Analysis (to be)

The statistic of given parameter

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 24

Detailed Function Introduction (11)
Functio Detailed Information Remark Level
n
Feature & 1. Feature scans for MML scripts and License It has been firstly ★
feature scans for License file used in test
License 2. Feature compares for MML scripts or department
Scan between MML script and License file
3. Rule of feature is defined by user in excel file
Result of License Scan

It is quick to known which feature is open for some operator?

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 25

 MML&License Feature Scan

1. MML Feature Scan

2. License File Scan
3. Feature Comparison
between MML and
License File

Result of License

Dialog of feature scan

Feature Result in
Scan
definition Excel

It is quick to known which feature is open for

some operator?

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 26

Detailed Function Introduction (12)
Functio Detailed Information Remark Level
Feature
n & 1. Feature scans for MML scripts and License It has been firstly ★
feature scans for License file used in test
License 1. Includedcompares
of run log,foralarm
MMLlog, call log,
or cell It has been used in ★★★
Node B 2. Feature scripts department
Scan log, operation
between MMLlog .etcand
script parse function.
License file department.
main 2.
3. The
Ruleconfigure
of featurefile figure shown.
is defined by user in excel file Provide fast parse
board log 3. DRD configure compared between RNC function and
parse script and Node B script. RNC/Node B script
4. Transmission configure compared between compare function.
RNC script and Node B script.

It is quick and convenient to known the site configuration.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 27
Detailed Function Introduction (12)

It is quick and convenient to known where is the problem of

DRD configuration.

It is quick and convenient to known where is the problem of

transmission configuration.
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 28
 Accident Recovery SOP
Application :
The accident recovery SOP is guide to prevent
The FMA tools has been deployed for about 53
or recover the accident quickly for the
operators. The following table is sample for
front, and provides technology and support.
Canada, and the recovery time of accident are
1)Precaution and preparative operation for listed.
Op Time Phenomena Recover
accident
time(min)
2) The network and guide for accident
Canada 2011-8-4 It is different to connect 15
collection tool
Sasktel for user
3) The FMA guide book
Canada 2011-7-6 CS RAB SR is decreased as 30
4) The emergency solution for accident, and Dry Run 80%
recover the accident by the guide Canada 2011-6-3 PS RAB SR is worse quickly 50
Dry Run
Benefit
Canada 2011-4-27 RRC SR is decreased as 90% 37
1) The time of collection of accident is Telus
saved by using the tool Canada 2011-4-6 RRC SR is decrease as 50% 90
2) The efficiency of analysis for accident SASKTEL
is improved. FMA can display the key Canada 2011-2-17 The traffic with 72 NodeB 10
information of accident and provide the Bell are interrupted under one
RNC 3811
result of diagnose quickly.
Canada 2011-1-28 The PS traffic for 6 RNC 24
3) The front can recover the accident based Bell are impacted
on the emergency solution
Canada 2011-1-27 The traffic with 789 NodeB 62
4)The average recovery time of accident of Bell are interrupted
UMTS is decreased as 50% with last year Canada 2011-1-6 The SPU Boards in Subrack 50
Telus 1,2,3 are reset
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 29
periodically
 Download
•The download time of FMA tool in Support network, and the comparison
with the other tools

•Up to 2011-8-30，the download time of FMA has reached more than

1000 times. The tool has been widely to use by Maintenance, R&D, Test,
NTS, GTAC and the front engineer.
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 30
 Download

1. UMTS Accident Log Collection Tool

2. UMTS FMA：analysis tool

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential Page 31

Thank you
www.huawei.com