TEMS Investigation KPI Definitions Streaming

TEMS Investigation KPI Definitions Streaming

2009-05-14

Ascom 2009. All rights reserved. TEMS is a trademark of Ascom. All other trademarks are the property of their respective holders. No part of this document may be reproduced in any form without the written permission of the copyright holder. The contents of this document are subject to revision without notice due to continued progress in methodology, design and manufacturing. Ascom shall have no liability for any error or damage of any kind resulting from the use of this document.

Public

2009-05-14

TEMS Investigation KPI Definitions Streaming

Contents
1. 2. Introduction ................................................................1 Service Independent KPIs .........................................1
2.1. 2.2. 2.3. 2.4. 2.5. 2.6. Network Unavailability [%] ......................................................1 Attach Failure Ratio [%] ..........................................................1 Attach Setup Time [s]..............................................................2 PDP Context Activation Failure Ratio [%] ...............................2 PDP Context Activation Time [s] .............................................2 PDP Context Cut-off Ratio [%] ................................................3 Service Non-Accessibility [%] .................................................3 Service Access Time [s] ..........................................................3 Reproduction Cut-off Ratio [%] ...............................................4 Streaming Quality [MOS] ........................................................4 Reproduction Start Failure Ratio [%].......................................5 Reproduction Start Delay [s] ...................................................5 Signaling Diagram...................................................................6

3.

Streaming Specific KPIs ............................................3

3.1. 3.2. 3.3. 3.4. 3.5. 3.6. 3.7.

TMS-06:000171 Uen Rev C

2009-05-14

TEMS Investigation KPI Definitions Streaming

1.

Introduction

This document describes in detail how KPIs (Key Performance Indicators) for the streaming service are calculated in TEMS Investigation 8.0. The KPI definitions are based on ETSI TS 102 250-2 v1.4.1 How input data to KPI calculations is collected is not dealt with in this document.

2.
2.1.

Service Independent KPIs

Network Unavailability [%]

Denotes the probability that no packet-switched network is available in the cell currently used by the customer. In GSM, the phone has access to a PS network if it has received System Information 13. This message is read once per KPI measurement cycle, at the beginning of the cycle. In WCDMA, matters are simpler: the phone is always known to have access to a PS network. The information element "Mode - System" in TEMS Investigation indicates whether the phone is connected to a WCDMA or a GSM network.
# of unsuccessful attempts to access PS network Network Unavailability [%] = -------------------------------------------------------------------- x 100 total # of attempts to access PS network

Because of the nature of this KPI, no start or stop triggers can be identified for it in the signaling diagrams.

2.2.

Attach Failure Ratio [%]

Denotes the probability that a subscriber cannot attach to the GPRS/UMTS PS network: # of unsuccessful attach attempts Attach Failure Ratio [%] = ----------------------------------------------- x 100 total # of attach attempts Start trigger: Phone sending Attach Request message. Trigger point 1 in section 3.7 Stop trigger: Phone receiving Attach Accept/Reject message. Trigger point 4 in section 3.7

TMS-06:000171 Uen Rev C

1(6)

TEMS Investigation KPI Definitions Streaming

2009-05-14

2.3.

Attach Setup Time [s]

Denotes the length of the time period taken to attach to the GPRS/UMTS PS network:

Attach Setup Time [s] = tAttach Accept tAttach Request

Start trigger: Phone sending Attach Request message. Trigger point 1 in section 3.7 Stop trigger: Phone receiving Attach Accept message. Trigger point 4 in section 3.7

2.4.

PDP Context Activation Failure Ratio [%]

Denotes the probability that the PDP context cannot be activated. It is the ratio of unsuccessful PDP context activation attempts to the total number of PDP context activation attempts:
# of unsuccessful PDP context activation attempts PDP Context Activation Failure Ratio [%] = ---------------------------------------------------------------------- x 100 total # of PDP context activation attempts

Start trigger: Phone sending PDP Context Activation Request message. Trigger point 5 in section 3.7 Stop trigger: Phone receiving PDP Context Activation Accept message. Trigger point 8 in section 3.7

2.5.

PDP Context Activation Time [s]

Denotes the length of the time period taken to activate a PDP context:

PDP Context Activation Time [s] = tPDP Context Activation Accept tPDP Context Activation Request

Start trigger: Phone sending PDP Context Activation Request message. Trigger point 5 in section 3.7 Stop trigger: Phone receiving PDP Context Activation Accept message. Trigger point 8 in section 3.7

2(6)

Public

2009-05-14

TEMS Investigation KPI Definitions Streaming

2.6.

PDP Context Cut-off Ratio [%]

Denotes the probability that a PDP context is deactivated without this being initiated intentionally by the user:
# of PDP context deactivations (not initiated by user) PDP Context Cut-off Ratio [%] = ------------------------------------------------------------------------ x 100 total # of successfully activated PDP contexts

Start trigger: Phone receiving PDP Context Activation Accept message. Trigger point 8 in section 3.7 Stop trigger: Phone receiving the last data packet containing content. Trigger point 28 in section 3.7 PDP context deactivation not initiated intentionally by the user can be caused by either SGSN failure or GGSN failure, so the PDP context may be deactivated either by the SGSN or by the GGSN. Note: The precondition for measuring this parameter is that a PDP context has been successfully established.

3.
3.1.

Streaming Specific KPIs

Service Non-Accessibility [%]

Denotes the probability that the first RTP data packet of the stream cannot be received by the phone when requested by the user. The reception of a packet is completed by the appearance of a "buffering" message in the users streaming client:
# of unsuccessful streaming attempts Service Non-Accessibility [%] = ---------------------------------------------------- x 100 # of streaming attempts

Start trigger: RTSP Setup sent to the streaming server. Trigger point 15 in section 3.7 Stop trigger: Reception of first RTP data packet. Trigger point 22 in section 3.7

3.2.

Service Access Time [s]

Denotes the duration of a service access from requesting the stream at the portal until the reception of the first stream data packet by the phone:

TMS-06:000171 Uen Rev C

TEMS Investigation KPI Definitions Streaming

2009-05-14

Service Access Time [s] = tfirst streaming packet tstreaming request

Start trigger: RTSP Setup sent to the streaming server. Trigger point 15 in section 3.7 Stop trigger: Reception of first RTP data packet. Trigger point 22 in section 3.7

3.3.

Reproduction Cut-off Ratio [%]

Denotes the probability that a successfully started stream reproduction is ended by a cause other than the intentional termination by the user:
# of lost streaming reproductions Reproduction Cut-off Ratio [%] = ----------------------------------------------- x 100 total # of streaming reproductions

Possible causes for stream reproduction cut-off include: Radio bearer loss Synchronization errors Streaming server/system failure/errors Protocol errors Streaming player failure/errors Start trigger: Reception of first data packet. Trigger point 22 in section 3.7 Stop trigger: RTSP TEARDOWN sent to the streaming server. Trigger point 28 in section 3.7

3.4.

Streaming Quality [MOS]

Denotes the quality of the stream reproduction as assessed by the VSQI algorithm. VSQI takes both audio and video into account:

Streaming Quality [MOS] = VSQI score for entire clip

Start trigger: Reception of first data packet. Trigger point 22 in section 3.7

4(6)

Public

2009-05-14

TEMS Investigation KPI Definitions Streaming

Stop trigger: Sending RTSP TEARDOWN to the streaming server. Trigger point 28 in section 3.7

3.5.

Reproduction Start Failure Ratio [%]

Denotes the probability of unsuccessful stream reproduction:

# of reproduction failures Reproduction Start Failure Ratio [%] = -------------------------------------------------- x 100 # of successful streaming accesses

Start trigger: Reception of first data packet. Trigger point 22 in section 3.7 Stop trigger: Streaming player signaling that the reproduction of the stream has started. Trigger point 23 in section 3.7

3.6.

Reproduction Start Delay [s]

Denotes the time elapsing from reception of the first stream data packet by the phone until the phone starts reproducing the stream:

Reproduction Start Delay [s] = tstart of reproduction tfirst data packet

Start trigger: Reception of first data packet. Trigger point 22 in section 3.7 Stop trigger: Streaming player signaling that the reproduction of the stream has started. Trigger point 23 in section 3.7

TMS-06:000171 Uen Rev C

TEMS Investigation KPI Definitions Streaming

2009-05-14

3.7.
UE

Signaling Diagram
Network

1 Attach 4 5 PDP 8 12 14 15 18 Service 19 22 Reproduction 23

Attach Request Attach Response PDP Context Activation PDP Context Activation Accept HTTP Get (Session Description) Session Description RTSP Setup RTSP SETUP ACK RTSP PLAY RTP: First media data packet Reproduction started Data Transfer

2 3 6 7 10 13 16 17 20 21

Media Transfer 25 27 28 RTP: Last media data packet RTCP BYE RTSP TEARDOWN 24 26

29

6(6)

Public