ETOM Key Quality Indicators

Wireless Service
Measurement
Key Quality Indicators
GB923A
Approved Version1.5 March, 2004

TeleManagement Forum 2004
Key Quality Indicators Page ii
Notice
The TeleManagement Forum (“TM Forum”) has made every effort to ensure that the contents of
this document are accurate. However, due to the inherent complexity in the design and
implementation of software and systems, no liability is accepted for any errors or omissions or for
consequences of any use made of this document. Under no circumstances will the TM Forum be
liable for direct or indirect damages or any costs or losses resulting from the use of this
specification. The risk of designing and implementing products in accordance with this
specification is borne solely by the user of this specification. This document may involve a claim of
patent rights by one or more TM Forum members, pursuant to the agreement on Intellectual
Property Rights between TM Forum and its members, and by non-members of TM Forum.
This document is a copyrighted document of TM Forum, and if reproduced in any manner shall
clearly acknowledge the authorship of TM Forum.
Direct inquiries to the TM Forum office:

89 Headquarters Plaza North - Suite 350
Morristown, NJ 07960 USA
Tel No. +1 973 292 1901
Fax No. +1 973 993 3131
TM Forum Web Page: www.tmforum.org
GB923a v1.5 TeleManagement Forum 2004

Key Quality Indicators Page iii
Acknowledgments
This document was prepared by the members of the TeleManagement Forum

Wireless Service Measurements team:
Ian Best Comnitel Technologies
Pertti Pielismaa Nokia
Michael Orzessek Agilent Technologies
Laure Reillier WatchMark
Carolyn Smithson O2
Andy Chalmers Orange
Marta Liminiana Telefonica Moviles España
Loles Noves Telefonica Moviles España
Tim Lock ADC
Håkan Kappelin Ericsson
Sam Sida ADC
Veli Kokkonen Sonera
Steve McElvanney TCSI
Peter Huckett Acterna
Peter Holland Lucent Technologies
Richard Deininger Marconi
Steve Bowker WatchMark
Rosario Drogo De Iacovo TILab
Phil Green Vodafone
David High AirCom
Scott Marshall Agilent Technologies

Key Quality Indicators Page iv
Martina Kurth Lucent Technologies
Christian Dinten Agilent Technologies
Richard Arthur Compaq
Thomas Tenevall KI Consulting
Aline Allul Evidian
Dia Helmy NEC America
Ahcène Latrèche Evidian

Key Quality Indicators Page v
About this document
Document Life Cycle
Document History
Version Date Purpose

Version # Mo/yr Description e.g. first issue of document
V0.1 Feb/ 03 Initial Draft for Brussels Review
V1.0 Sept/’03 Updated for Member Evaluation with

corrected template etc.
V1.1 Dec/03 Updated for Member Approval.
V1.5 Mar/03 Approved version.

Key Quality Indicators Page vi
About TeleManagement Forum
TeleManagement Forum is an international consortium of communications service

providers and their suppliers. Its mission is to help service providers and network
operators automate their business processes in a cost- and time-effective way.
Specifically, the work of the TM Forum includes:
Establishing operational guidance on the shape of business processes.
Agreeing on information that needs to flow from one process activity to

another.
Identifying a realistic systems environment to support the interconnection

of operational support systems.
Enabling the development of a market and real products for integrating

and automating telecom operations processes.
The members of TM Forum include service providers, network operators and

suppliers of equipment and software to the communications industry. With that
combination of buyers and suppliers of operational support systems, TM Forum is
able to achieve results in a pragmatic way that leads to product offerings (from
member companies) as well as paper specifications.

Key Quality Indicators Page vii
Table of Contents
Notice ..................................................................................................................................................................ii
Acknowledgments ...........................................................................................................................................iii
About this document ........................................................................................................................................v
Document Life Cycle ..................................................................................................................................v
Document History .....................................................................................................................................v
About TeleManagement Forum .....................................................................................................................vi
Table of Contents ............................................................................................................................................vii
Table of Figures..............................................................................................................................................viii
Introduction ........................................................................................................................................................9
Background.................................................................................................................................................9
Purpose.......................................................................................................................................................9
Intended Audience and Benefits............................................................................................................. 10
Concepts & Key Definitions.......................................................................................................................... 11
Introduction .............................................................................................................................................. 11
KQI Template Definition.......................................................................................................................... 13
KQI Definition and Usage.............................................................................................................................. 17
Background.............................................................................................................................................. 17
Network KQIs........................................................................................................................................... 20
Service Access KQIs.......................................................................................................................... 20
Service Usage KQIs ........................................................................................................................... 22
Service Disconnection KQIs .............................................................................................................. 23
Non Network KQIs................................................................................................................................... 24
Pre-sales KQIs.................................................................................................................................... 24
Service Ordering KQIs ....................................................................................................................... 25
Billing KQIs.......................................................................................................................................... 26
Customer care KQIs........................................................................................................................... 27
Handset problem management KQIs................................................................................................ 30
SIM Card KQIs.................................................................................................................................... 32
Network provisioning KQIs................................................................................................................. 33
Pre-Paid Specifics KQIs..................................................................................................................... 34
Roaming Specifics KQIs .................................................................................................................... 35
Third Party Providers KQIs ................................................................................................................ 36
Cessation of Service KQIs ................................................................................................................. 38

Key Quality Indicators Page viii
Table of Figures
Figure 1 - Key Indicator Hierarchy 12

Figure 2 - KQI Sliding Window 16
Figure 3 - Product Components 17
Figure 4 - Customer Experience Timeline 18

Key Quality Indicators Page 9
Introduction
Background
As wireless markets mature a common expectation is a guaranteed quality of

service, especially for the high end of the market. There is a need for network
operators and service providers to manage wholesale relationships and guarantee
quality levels at wholesale positions.
The focus shift to service based management has meant that new service
measurements that capture the customer's perception need to be put in place.
Purpose
Based on the results of the work of Wireless Service-based Measurements Team

(GB923), this document lists the KQIs derived by that team and presents them along
with others as a stand alone companion to that document.
This is a living document and the intention is to maintain it as an up to date source of

reference for Key Quality Indicators (KQIs).

Intended Audience and Benefits
This document is intended for use at several levels:
By service managers and designers, responsible for managing the QoS

of any given service or product.
This document will assist Network, Content and Service Providers to

identify the properties of the KQIs specific to their Service Level
Agreements
This document will help vendors of OSS management solutions to

ensure that the correct metrics are available to determine the appropriate
KQI.
The document also delivers a common reference point for KQI

information.

Concepts & Key Definitions
Introduction
Service Providers have historically reported the performance of their networks

against a set of business agreed Key Performance Indicators (KPIs). These KPIs
are inherently network focused and provide little direct indication of the end to end
service delivery that the network supports. Nevertheless KPIs are an important
measurement for network operations and will continue to be so for the foreseeable
future.
The move towards service focused management leads to a requirement for a new
'breed' of indicators that are focused on service quality rather than network
performance. These new indicators or Key Quality Indicators (KQIs) provide a
measurement of a specific aspect of the performance of the product, product
components e.g. services, or service elements and draw their data from a number of
sources including the KPIs.
Two main types of KQI need to be considered. At the highest level, a KQI or group
of KQIs are required to monitor the quality of the product offered to the end-user.
These KQIs will often form part of the contractual SLA between the provider and the
customer. Product KQIs will derive some of their data from the lower level Service
KQIs, with the latter focused on monitoring the performance of individual product
components. In its simplest form a KQI may have one single KPI as its data source.
More commonly a Service KQI will aggregate multiple KPIs to calculate the service
element quality and Product KQIs will aggregate multiple Service KQIs. Figure 1
illustrates the key indicator hierarchy.

Customer SLAs
Product Additional
KQIs Data
Customer Focus
Internal & Supplier/Partner SLAs Product Components
Service Additional
Network Focus
KQIs Data
Service Elements
Additional
KPIs Data
Service Resources
(Grouped by Type)
Performance
Related Data
Service Resource
Instances e.g.
Network Element
Figure 1 - Key Indicator Hierarchy
A primary function of the KQI is in the calculation of Service Level Agreement

compliance. In typical circumstances multiple KQIs will feed the SLA management
processes, with Service KQIs supporting multiple internal or Supplier / Partner SLAs
and Product KQIs supporting multiple Customer (external) SLAs.
As a service element may depend upon a number of network resources, it follows

that a Service KQI may call upon multiple KPIs in the aggregation process.
The nature of services that may be included in products offered by Service Providers
is likely to result in variants of a 'base' service. It is important therefore that a KQI
may also utilise other KQIs as part of its aggregation algorithm.
At all levels additional ‘contextual’ information is required to complete the aggregation

process, for example, the weighting factors applied to each KQI parameter.
To summarise, starting from the network elements, the network performance data is
aggregated to provide KPIs that are an indication of service resource performance.
The KPIs are then used to produce the Service KQIs that are the key indicators of
the service element performance. Service KQIs are then used as the primary input
for management of internal or supplier / partner SLAs that calculate actual service
delivery quality against design targets or in the case of supplier / partner, contractual

agreements. Service KQIs provide the main source of data for the Product KQIs that
are required to manage product quality and support the contractual SLAs with the
customer.
KQI Template Definition
Based on the preceding description of a KQI it is proposed that a KQI template will
be defined by the following parameters:
Parameter Mandatory/ Type Remarks

Optional
Unique Identifier M Alpha-
numeric
Measurement M KQI or KPI
Parameter 1
Measurement O (see note String The Co-ordinated Universal
Parameter 1 3) Time (UTC) string, which enables the identification of
Time Zone the time zone the underlying KPI’s is/are originated from.
Measurement O Alpha- Service resource instance, type or group
Parameter source numeric
1
Weighting Factor O Numeric The weighting factor for all measurements parameters
1 (see note 2) should equal 1.
Measurement O Ditto
Parameter 2
Parameter 2 3) Time (UTC) string, which enables the identification of
2
2 (see note 2) should equal 1.


Optional
Measurement O Ditto
Parameter n
Parameter n 3) Time (UTC) string, which enables the identification of
n
n (see note 2) should equal 1.
Transformation M NA This is the algorithm to be applied to calculate the KQI
Algorithm value from the combination of measurement parameters
and their weighting factors. The simplest form of
algorithm is ‘=’ where a single KPI becomes a KQI.
Objective Value O NA This is the value that is used to determine whether the
calculated KQI is within the target value. The value is
used to determine whether a violation should be
generated
Violation direction O +ve or -ve The determination of whether a breach occurs if the
calculated value is above or below the objective value is
determined by this field. i.e. if this filed is –ve a violation
or warning will be generated if the calculated value drops
below the objective value or warning threshold.
Warning level O % Optionally a % value may be specified to give a warning
that a KQI objective is in danger of violating
Abatement O % This parameter introduces a hysteresis factor to the
threshold warnings and alarms to prevent oscillation between an
alarm and a clear condition.
Measurement M Minutes This is the period over which the KQI performance is
Period calculated (see note 1)
Grace Periods O # How many times a value can remain un-updated without
violation of the SLA in reference to the Measurement
Period and / or KQI reporting period


Optional
Active/Inactive O A UTC A KQI measurement period may be controlled by
period string specifying a start and stop time for the KQI e.g. for a
KQI that is only applicable between 9 a.m. and 5 p.m.
I UTC string this field would be set to A 09:00,17:00
If the field is preceded with the letter A, the time period
has to be included, if preceded with the letter I, the time
period has to be excluded from the measurement.
KQI Time Zone O UTC String The Co-ordinated Universal
Time (UTC) string, which permits the identification of the
time zone that the KQI is originated from. Can be
derived from homogeneous KPI or timestamp if valid.
Active/Inactive O A Day list or List of days that the KQI measurement is active or
days I Day list inactive e.g. for a KQI that is measured from Monday
through Friday the list would be A 2,3,4,5,6 To exclude
Saturday and Sunday’s the list would be I 6,7
Note 1:
As a KQI may be defined by a number of measurement parameters that have

differing sample periods, a KQI instance may be derived in the following ways:
As new data is received
On a fixed time interval
On a fixed time interval with a sliding window, as illustrated in

KQI Sliding Window

Reporting
Parameter 1 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6
Parameter 2 sample 1 sample 2 sample 3
Parameter 3 sample 1 sample 2
KQI report 1
report 2
report 3
report 4
KQI reporting period
Figure 2 - KQI Sliding Window
Note 2:
The Weighting Factor is optional. When applied it must be included for all
parameters. If omitted the effective weighting factor for all parameters are to be
considered as equal.
Note 3:
If the time zone for parameter one is not stated then local time is assumed. If the
time zone for the subsequent parameters is not stated then the same time zone for
parameter 1 is applied.
If the time zone for parameter one is not stated then local time is assumed. If the
time zone for the subsequent parameters is not stated then the same time zone for
parameter 1 is applied.

KQI Definition and Usage
Background
When considering end customer or external SLAs, the nature of the products sold to
the customer needs to be taken into account. While internally the focus may be on
individual services, the offering to the customer tends to be a product that consists of
a number of services which may be either network or non-network based.
• Voice Network
• SMS
• Voicemail
• WAP
• Handset
• Customer Care
• Itemised Billing
• Per Second Billing
• Handset Replacement
• Insurance Non-Network
Figure 3 - Product Components
Figure 3 illustrates the range of product components which could

make up a product offering. As can be seen some of these components are network
derived, however there are a number of the product components that have little or no
dependence on network resources. The SLA with the customer is likely to
encompass all aspects of the product including the non-network aspects. The

external SLA will consist of a wide ranging set of parameters worded in terms that
are understood by the customer. Deriving this set of SLA criteria requires a clear
understanding of what is important to the end user and is therefore best achieved by
discussion and negotiation with the users.
The work of the Wireless Service Measurement Team presented in GB 923 uses a high-level process analysis based on the
Customer Experience Guideline (
Figure 4) to expand beyond the measurement of network derived service quality.
Supplier Interaction considerations
•Marketing contact •Multiple •Transient?

•Quality of info activation routes •May hit usage problem •‘Self-inflicted’ Do you notify customer
•Expectation set •CuCa •Self help vs CuCa call •Service conflicts if not impacted?
•Direct sales •Priority
•Service interruption •Who knows
first? Accuracy
Multiple info Notify of
•Impact
sources on successful
•Content dependent •Response
product/service activation Why?
•Defined ‘sub-service’
Pre-sales Service Ordering In-life Experience
First user
experience Receive
Find out Change
bill
about Service Personalise personalisation
service Activated service
Report
Cease
problem
Subsequent service
Provision Request usage
service for service Experience
customer
problem
Problem
Decide to buy resolved
Customer
advised
Customer Interactions
Figure 4 - Customer Experience Timeline
The timeline is developed to represent both the customer’s and supplier’s actions in
the key processes that support the customer experience from the initial pre-sales
contact through the in-life experience of the customer, to cessation of the service.

In each of these process steps the timeline can be applied as indicated in Figure 4,
resulting in more specific Customer Experience Timelines. The bottom half of the
time line maps the customers interactions with the service at different points in the
life-cycle. The top half of the timeline considers the effect of those actions on the
supplier of service and the options or impacts that need to be considered in the
analysis.
Applying each process to the timeline provides more specific timeline actions and
prompts review of the interactions from a customer perspective which may need to
be measured. By taking a top-down approach the full range of both network and non-
network related interactions can be identified.
The following sections catalogue the network and non-network KQIs which have
been identified. Each KQI has been mapped to the categories defined in ETR003:
• Speed
• Availability
• Reliability
• Security
• Simplicity
• Flexibility

Network KQIs
Service Access KQIs

KQI Algorithm Unit ETR003 Category Variation Description
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
% % Time that each service
SA Service Resource ((total_time - down_time) / mean,
resource in the service delivery
Availability total_time )*100 minimum
chain is available
min(Service Resource % mean, Service availability = worst of
SA Service Availability
Availability) minimum the service element availability
mse Time to connect to the network
SA Network Access network accessible time - mean,
c from time that the terminal is
Time terminal power on time max
powered on
((total pdp attempts-total pdp %
SA PDP Context %Success in defining PDP
failures)/total pdp mean,min
Success Rate context
attempts)*100
(number connections PDP %
SA Downgraded PDP mean, % of connections that are
context downgraded / total
Renegotiation Rate max downgraded
service connections)*100
(number PDP renogotiation %
% of PDP renegotiations that
SA Successful PDP upgrade attempts - failed PDP
have been successfully
Renegotiation renogiation upgrade attempts) mean,min
upgraded
Upgrade Rate / number PDP renogotiation
upgrade attempts ) * 100
((total attach attempts-total %
SA Attach Success %Success in attach to GPRS /
attach failures)/total attachp mean,min
Rate UMTS network
attempts)*100
mean(Service Availability, PDP %
SA Service Setup mean,
Context Success Rate, Attach Combined KQI
Success Rate minimum
Success Rate)

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
attached end time - attach start mse mean, Time to attach to GPRS /
SA Attach Time
time c max UMTS network
((total service establishment %
attempts - total service
SA End to End Service mean, % Success in connecting to
establishment failures)/total
Establishment Rate minimum service
service establishment
attempts)*100
SA End to End Service establishment end time - msc mean, Time to establish connection to
Establishment Time establishment begin time max to service
((number service activations - %
SA Service Activation number service activation mean, For customer self activation -
Success Rate failures) / number service minimum % failures
activations) * 100
coun Number of times that
SA Unauthorised number unauthorised service
t count unauthorised access is
Service Access accesses
attempted to a service

Service Usage KQIs

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
SU Service Data sum(data_transfer_rate_per_tr bps mean, Time to transfer data end to
Transfer Rate ansaction) / sum(transactions) minimum end.
((number %
SU Service _service_transactions -
mean, % of successful service
Transaction Success number_transaction_failures) /
minimum transactions
Rate number_service_transactions)*
100
((number_service_connections %
- % of service connections that
SU Service
number_service_connection_f mean, failure after connection
Connection Failure
ailures) / max establishment i.e. connection
Rate
number_service_connections)* drop out
100
((number_service_transactions %
-
(number_service_transaction_r % service transactions that
SU Service Data eattempts + mean, have to be re-attempted either
Transaction Accuracy number_content_deliveries_co minimum by the user or due to automatic
ntent_not_preserved)) / re-transmissions
number_service_transactions)*
100
((number_content_deliveries_c %
ontent_preserved -
% of content delivered to end
SU Service Content number_content_deliveries_co mean,
user without data corruption /
Integrity ntent_not_preserved) / minimum
modification
number_content_deliveries_co
ntent_preserved) * 100
((number_service_connections %
SU Service Usage - mean, % service uses that complete
Success Rate (number_service_connection_f minimum without errors
ailures+number_service_trans

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
action_reattempts+number_se
rvice_disconnection_failures) /
number_service_connections)
* 100
SU Service end_transaction_time - mse mean,
Time to complete a transaction
Transaction Speed begin_transaction_time c max
end_service_connection_time mse
SU Service Usage - c mean, Total time user is connected to
Time begin_service_connection_tim max a service
e
Service Disconnection KQIs

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
((number service connections

SU Service - number service % service of service
mean,
Disconnect Success disconnection failures) / % connections that disconnect
minimum
Rate number service connections) 'cleanly'
* 100

Non Network KQIs
Pre-sales KQIs
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
% Number of customer enquiries
(number enquires/ target
PS Customer Enquiries during a fixed period vs. target
number enquires) * 100
nor of enquiries
(number sales/ number %
PS Enquiries To Sales % of enquiries resulting in a sale.
enquires) * 100
days mean, Time between initial enquiry and
PS Time To Sale Date of sale – date of enquiry max
sale
(number cancellations/ target % Cancellations during contract
PS % Of Cancellations
number cancellations) * 100 ‘cool off’ period

Service Ordering KQIs
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
%
((Date of activation – date of
Time between order and service
SO Time to Activate order )/ target time to activate)
activation vs. target time
* 100
(number faulty activations/ %
SO % Faulty Activations % faulty service activations
service activations) * 100
(number activation %
SO % Activation % customer complaints re
compalints/ service
Complaints service activation
activations) * 100
(number complaints re % % modifications to service
SO % Modifications activation update/ service activation ( did not meet
activation updates) * 100 customer expectations)
(number CPE access failures/ % Failed to access customer
SO CPE access Failues
CPE access attempts) * 100 premises for CPE deployment

Billing KQIs
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
% Billing accuracy – % of number
(number complaints re billing
of customer complaints resulting
BI Billing Accuracy accuracy in period/ no bills in
in correction to a bill during a fix
period) * 100
period window
% Billing timing – % of bills sent to
(number bills issued within
the customers within a
BI Billing Timing target time/ no bills in period) *
parameter number of days from
100
the target billing date
(number bills issued calls from % Billing content - % of bills
BI Late Billing previous bill period/ no bills in containing calls that should have
period) * 100 been included in the previous bill
% Ease of use – Number of calls
(number calls querying bill due to the customer not being
BI Ease Of Use
content/ target) * 100 able to understand the content of
the bill vs target.
% Customer enquiries – Total
(number calls querying bill /
BI Customer Enquiries number of customer calls
target) * 100
querying their bill (vs Target).

Customer care KQIs
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
(number calls – number calls % % of customer calls that are not
Cc Calls Not Answered
within target / number of calls) answered within the target
Within The Target Time
* 100 answer time
% Actual maximum call queue
(Call queue length / target) * Max,
CC Call Queue Length length vs. target max queue
100 mean
length
% % of customer calls where the
customer terminates the call
(number calls terminated
CC Customer Call before reaching the correct
prematurely/ number of calls) *
Terminations destination i.e. got fed up of
100
waiting or lost patience with the
IVR
% % of customer calls that are not
(number calls – number calls
CC Calls Unresolved dealt with within the target time
unresolved within target time/
within Target Time i.e. call hold time longer than
number of calls) * 100
target
(number calls referred to % % of customer calls that are
CC Call Referrals another operator/ number of referred to another operative e.g.
calls) * 100 passed to another operative.

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
% % of customer calls that are
(number calls escalated / escalated to a more senior
CC Call Escalations
number of calls) * 100 operative e.g. supervisor or
manager
(number calls requiring SP % % of customer calls that require
CC Call Follow-ups follow-up/ number of calls) * a follow-up call from the service
100 provider
(Time to follow-up calls/ % Max, Time between initial contact and
CC Time to Follow-Up
Target) * 100 mean follow-up call vs. target
% Number of customer calls per
CC Service Calls (Calls per service for period/ Total, per
service during a fixed period vs
against Target target) * 100 service
target
% Number of customer trouble
CC Customer Trouble (Trouble tickets for service for Total, per
tickets raised against a service
Tickets by Service period/ Target) * 100 service
during a fixed period vs target.
(Trouble Tickets not fixed in %
CC Fix in Resolution Total, per % of Customer Trouble Tickets
resolution time for period/
Time. service not fixed within the target time.
target) * 100
(Trouble Tickets re-opened for % % of Customer Trouble Tickets
CC Repeat Faults
period/ target) * 100 that are re-opened within a fixed

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
period i.e. not fixed to the
customers’ satisfaction (Repeat
Faults)
(Trouble Tickets closed with no %
% of TTs closed with no problem
CC No Problem Found problem found for period/ No
found
tickets for period) * 100
% Failed to access customer
premises for CPE maintenance.
CC CPE Maintenance (No CPE access denied/ CPE
(Note: This is a switch KQI that
Access Failure maintenance calls) * 100
may switch off or suspend an
SLA) as above
(CPE tasks not completed on % % of CPE maintenance tasks
CC CPE Tasks Not
schedule for period/ target) * not completed according to the
Completed on Schedule
100 agreed schedule.

Handset problem management KQIs
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
(number of handset %
replacements that take longer
HP Time to Replace Total, per % of handset replacements that
than the target time/ total
Handset type take longer than the target time
number of handset
replacements)* 100
(number of handset % % of handset replacements
HP Handset Replaced
replacements within the within the guarantee period vs
within Guarantee
guarantee period/ target) * 100 target
(Number of handset %
% of handset replacements that
HP Faulty Handset replacements that are faulty
are faulty i.e. replacement
Replacements /total number of handset
handset is also faulty
replacements) *100
(Number of handset %
HP Handset
replacements where no fault % handset replacements where
Replacements –No
was found /total number of there is no fault found
Fault Found
handset replacements) *100
(Number of customer calls Number of customer calls
HP Handset Usage
seeking assistance on handset seeking assistance on handset
Calls
usage / target) *100 usage (due to poor handset user

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
manual) vs target
% % of customers reporting
(Number of handset complaints/ dissatisfaction with a handset
HP Handset Complaints
target) *100 (requesting new handset type)
vs target
% % of customers seeking a
handset upgrade within a set
(Number of customers seeking a
interval e.g. one year vs target,
HP Customers Handset handset upgrade within a set
by reason; access to new
Upgrade Requests interval e.g. one year by reason/
service(s), usability (ergonomics,
target) *100
battery life etc.), poor
performance
Total number of trouble tickets Number of trouble tickets raised
HP Handset Trouble
raised due to problems with new due to problems with new
Tickets
software installation software installation

SIM Card KQIs
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
(number of SIM card failures %
SI % SIM Failures % of SIM cards that fail
/number of SIM cards) * 100
(number of SIM replacements %
SI % SIM Timely % of SIM replacements that take
that take less time than the
Replacements less time than the target time
target time/ target) * 100
(number of SIM replacements %
SI % SIM % of SIM replacements per SIM
per SIM type/ number of SIM
Replacements by Type type
cards per SIM type) *100
(number of SIM replacements % % of SIM replacements that are
SI % Faulty SIM
that are faulty/ number of SIM faulty i.e. replacement SIM is
Replacements
replacements) *100 also faulty

Network provisioning KQIs
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
(Provisioning tasks completed % % of network provisioning tasks
NP % Tasks Completed
on time for period/ Tasks in that are completed by the target
On Time.
period) * 100 time.
% % of network provisioning tasks
NP % Failed (Failed Provisioning tasks for
that fail i.e. corrective action is
Provisioning Tasks period/ Tasks in period) * 100
necessary
% % of network provisioning tasks
NP % Network (Faulty Provisioning tasks for that are incorrect, i.e. complete
Provisioning Errors period/ Tasks in period) * 100 as per plan but need corrective
action.
(Provisioning tasks not %
NP % Problems Not % of network problems not
completed on time for period/
Corrected in Time corrected within the target time
Tasks in period) * 100
% Average number of network
NP Failed Maintenance (Failed tasks due to inventory Mean,ma maintenance activities that fail
Activities data/ Tasks in period) * 100 x due to incorrect network
inventory data
NP % Problems (Provisioning tasks waiting % % of network problems
Awaiting Spares spares / Tasks in period) * 100 outstanding due to insufficient

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
equipment spares
% % of network problems
(Provisioning tasks waiting outstanding awaiting
NP % Problems
engineer allocation / Tasks in engineering resource allocation
Awaiting Engineer
period) * 100 i.e. awaiting the availability of a
suitably trained engineer.
Pre-Paid Specifics KQIs
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
% Actual maximum/mean time to

(Time taken to top up/ Target) Mean,ma
PP Time to Top-up access the top-up service vs.
* 100 x
target
PP % Customers with (Customers with zero credit/ % % customers that have zero
Zero credit Target) * 100 credit vs. target
(Successful top-ups for period/ % % top-up service access

PP % Top-up Access
Top-ups period) * 100 success

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
% % top-up service failures
(Failed top-ups for period/ Top-
PP % Top-up Failures (including call dropped during
ups period) * 100
top-up)
Roaming Specifics KQIs
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
% Number of customers reporting
RO No Roaming (Customers reporting FN Mean,ma
roaming problems on a foreign
Problems FN problems/ Target) * 100 x
network (per network) vs. target.
% Number of roaming customers
(Customers reporting host
RO No Roaming reporting problems on the host
network problems/ Target) *
Problems host network (per remote SP) vs.
100
target.
% Time for roaming billing records
RO Billing Records (Time to transfer billing records Mean,ma
to be transferred to the home SP
Transfer Time -home to home SP/ Target) * 100 x
vs. target (per network)

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
% Time for roaming billing records
RO Billing Records (Time to transfer billing records Mean,ma
to be transferred to the remote
Transfer Time -remote to remote SP/ Target) * 100 x
SP vs. target (per network)
Third Party Providers KQIs
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
(number calls – number calls % % of problem reporting calls that
TP % Calls Answered
answered within target / are not answered within the
within Target
number of calls) * 100 target answer time
% % of calls that are escalated to a
(number calls escalated /
TP % Call Escalations more senior operative e.g.
number of calls) * 100
supervisor or manager
% Average time between initial
TP Time to Follow-up (Time to follow-up calls/ Mean,
contact and follow-up call vs.
call Target) * 100 max
target
TP Time to problem (Time to resolve calls/ Target) % Mean, Average time between initial

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
resolution * 100 max contact and problem resolution
call vs. target
% Number of calls to 3rd party (per
rd
(No calls to 3 party per service) during a fixed period vs
TP Number of calls to Total, Per
service per period/ Target) * target (per reason: Service
3rd party reason
100 outage, Service operation,
Latency, Data accuracy)
rd
(No 3 trouble tickets per % Number of 3rd party trouble
TP Number Trouble
service per period/ Target) * tickets raised against a service
Tickets
100 during a fixed period vs target.
(No 3rd trouble tickets not fixed %
TP Trouble Tickets not % of 3rd Party Trouble Tickets
in target per period/ No tickets)
fixed not fixed within the target time.
* 100
(No 3rd trouble tickets re- % % of 3rd Party Trouble Tickets
TP Re-opened Trouble
opened per period/ No tickets) that are re-opened i.e. not fixed
Tickets
* 100 to the SP’s satisfaction
% % of 3rd Party trouble tickets
TP Trouble Tickets with (No 3rd trouble tickets followed- that result in a follow-up call from
Follow-up up per period/ No tickets) * 100 the SP i.e. not fixed within the
SP’s expected fix time.

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
(3rd party CPE tasks not % % of 3rd party CPE maintenance
TP Incomplete CPE
completed on schedule for tasks not completed according to
maintenance tasks
period/ total tasks) * 100 the agreed schedule.
Cessation of Service KQIs
Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
CS % of subscriptions (No subscriptions terminated %

% of subscriptions terminated
terminated outside of outside target/ No
outside of the target time
the target time subscriptions) * 100
(No subscriptions incorrectly %
CS % of subscriptions % of subscriptions incorrectly
terminated/ No subscriptions) *
incorrectly terminated terminated
100
(No subscriptions failed % % of subscriptions terminates
CS % of subscriptions
terminations/ No subscriptions) that fail (and have to be
terminates that fail
* 100 corrected)

Availability
Simplicity
Reliability
Flexibility
Accuracy
Security
Speed
% % of subscriptions terminated
i.e. service churn per reason:
(No subscriptions terminated
CS % of subscriptions Mis--match with customer
per reason/ No subscriptions) *
terminated expectations / value, Coverage,
100
reliability, service offering, too
complicated

ETOM Key Quality Indicators

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ETOM Key Quality Indicators

Uploaded by

Copyright:

Available Formats

Wireless Service

Approved Version1.5 March, 2004

Direct inquiries to the TM Forum office:

GB923a v1.5 TeleManagement Forum 2004

This document was prepared by the members of the TeleManagement Forum

Ian Best Comnitel Technologies

Pertti Pielismaa Nokia

Michael Orzessek Agilent Technologies

Laure Reillier WatchMark

Andy Chalmers Orange

Marta Liminiana Telefonica Moviles España

Loles Noves Telefonica Moviles España

Tim Lock ADC

Håkan Kappelin Ericsson

Sam Sida ADC

Veli Kokkonen Sonera

Steve McElvanney TCSI

Peter Huckett Acterna

Peter Holland Lucent Technologies

Richard Deininger Marconi

Steve Bowker WatchMark

Rosario Drogo De Iacovo TILab

Phil Green Vodafone

David High AirCom

Scott Marshall Agilent Technologies

GB923a v1.5 TeleManagement Forum 2004

Martina Kurth Lucent Technologies

Christian Dinten Agilent Technologies

Richard Arthur Compaq

Thomas Tenevall KI Consulting

Aline Allul Evidian

Dia Helmy NEC America

Ahcène Latrèche Evidian

GB923a v1.5 TeleManagement Forum 2004

About this document

Document Life Cycle

Version Date Purpose

V0.1 Feb/ 03 Initial Draft for Brussels Review

V1.0 Sept/’03 Updated for Member Evaluation with

V1.1 Dec/03 Updated for Member Approval.

V1.5 Mar/03 Approved version.

GB923a v1.5 TeleManagement Forum 2004

About TeleManagement Forum

TeleManagement Forum is an international consortium of communications service

Establishing operational guidance on the shape of business processes.

Agreeing on information that needs to flow from one process activity to

Identifying a realistic systems environment to support the interconnection

Enabling the development of a market and real products for integrating

The members of TM Forum include service providers, network operators and

GB923a v1.5 TeleManagement Forum 2004

GB923a v1.5 TeleManagement Forum 2004

Figure 1 - Key Indicator Hierarchy 12

GB923a v1.5 TeleManagement Forum 2004

As wireless markets mature a common expectation is a guaranteed quality of

Based on the results of the work of Wireless Service-based Measurements Team

This is a living document and the intention is to maintain it as an up to date source of

GB923a v1.5 TeleManagement Forum 2004

Intended Audience and Benefits

This document is intended for use at several levels:

By service managers and designers, responsible for managing the QoS

This document will assist Network, Content and Service Providers to

This document will help vendors of OSS management solutions to

The document also delivers a common reference point for KQI

GB923a v1.5 TeleManagement Forum 2004