TEMS Investigation KPI Definitions - FTP

TEMS Investigation KPI Definitions FTP
2009-05-14
Ascom 2009. All rights reserved. TEMS is a trademark of Ascom. All other trademarks are the property of their respective holders. No part of this document may be reproduced in any form without the written permission of the copyright holder. The contents of this document are subject to revision without notice due to continued progress in methodology, design and manufacturing. Ascom shall have no liability for any error or damage of any kind resulting from the use of this document.
Public
2009-05-14
Contents
1. 2. Introduction ................................................................1 Service Independent KPIs .........................................1
2.1. 2.2. 2.3. 2.4. 2.5. 2.6. Network Unavailability [%] ......................................................1 Attach Failure Ratio [%] ..........................................................1 Attach Setup Time [s]..............................................................2 PDP Context Activation Failure Ratio [%] ...............................2 PDP Context Activation Time [s] .............................................2 PDP Context Cut-off Ratio [%] ................................................3
3.
FTP Specific KPIs.......................................................3

3.1. FTP Download Method A .....................................................4
3.1.1. 3.1.2. 3.1.3. 3.1.4. 3.1.5. 3.1.6. 3.1.7. 3.2.1. 3.2.2. 3.2.3. 3.2.4. 3.2.5. 3.2.6. 3.2.7. 3.3.1. 3.3.2. 3.3.3. 3.3.4. 3.3.5. 3.3.6. 3.3.7. 3.4.1. 3.4.2. 3.4.3. 3.4.4. 3.4.5. 3.4.6. 3.4.7. Service Non-Accessibility [%] .................................................................4 Setup Time [s].........................................................................................4 IP Service Access Failure Ratio [%] .......................................................4 IP Service Setup Time [s] .......................................................................5 Mean Data Rate [kbit/s] ..........................................................................5 Data Transfer Cut-off Ratio [%] ..............................................................5 Signaling Diagram...................................................................................6 Service Non-Accessibility [%] .................................................................7 Setup Time [s].........................................................................................7 IP Service Access Failure Ratio [%] .......................................................7 IP Service Setup Time [s] .......................................................................8 Mean Data Rate [kbit/s] ..........................................................................8 Data Transfer Cut-off Ratio [%] ..............................................................8 Signaling Diagram...................................................................................9 Service Non-Accessibility [%] ...............................................................10 Setup Time [s].......................................................................................10 IP Service Access Failure Ratio [%] .....................................................10 IP Service Setup Time [s] .....................................................................11 Mean Data Rate [kbit/s] ........................................................................11 Data Transfer Cut-off Ratio [%] ............................................................11 Signaling Diagram.................................................................................12 Service Non-Accessibility [%] ...............................................................13 Setup Time [s].......................................................................................13 IP Service Access Failure Ratio [%] .....................................................13 IP Service Setup Time [s] .....................................................................14 Mean Data Rate [kbit/s] ........................................................................14 Data Transfer Cut-off Ratio [%] ............................................................14 Signaling Diagram.................................................................................15
3.2. FTP Download Method B.....................................................7
3.3. FTP Upload Method A........................................................10
3.4. FTP Upload Method B .......................................................13
TMS-06:000076 Uen Rev C
2009-05-14
4(15)
Public
2009-05-14
1.
Introduction
This document describes in detail how KPIs (Key Performance Indicators) for the FTP service are calculated in TEMS Investigation 8.0. The KPI definitions are based on ETSI TS 102 250-2 V1.4.1. How input data to KPI calculations is collected is not dealt with in this document.
2.
2.1.
Service Independent KPIs

Network Unavailability [%]
Denotes the probability that no packet-switched network is available in the cell currently used by the customer. In GSM, the phone has access to a PS network if it has received System Information 13. This message is read once per KPI measurement cycle, at the beginning of the cycle. In WCDMA, matters are simpler: the phone is always known to have access to a PS network. The information element "Mode - System" in TEMS Investigation indicates whether the phone is connected to a WCDMA or a GSM network.
# of unsuccessful attempts to access PS network Network Unavailability [%] = -------------------------------------------------------------------- x 100 total # of attempts to access PS network
Because of the nature of this KPI, no start or stop triggers can be identified for it in the signaling diagrams.
2.2.
Attach Failure Ratio [%]
Denotes the probability that a subscriber cannot attach to the GPRS/UMTS PS network: # of unsuccessful attach attempts Attach Failure Ratio [%] = ----------------------------------------------- x 100 total # of attach attempts Start trigger: Phone sending Attach Request message. Trigger point 1 in sections 3.1.7, 3.2.7, 3.3.7, and 3.4.7 Stop trigger: Phone receiving Attach Accept/Reject message. Trigger point 4 in sections 3.1.7, 3.2.7, 3.3.7, and 3.4.7
1(15)
2009-05-14
2.3.
Attach Setup Time [s]
Denotes the length of the time period taken to attach to the GPRS/UMTS PS network:
Attach Setup Time [s] = tAttach Accept tAttach Request
Start trigger: Phone sending Attach Request message. Trigger point 1 in sections 3.1.7, 3.2.7, 3.3.7, and 3.4.7 Stop trigger: Phone receiving Attach Accept message. Trigger point 4 in sections 3.1.7, 3.2.7, 3.3.7, and 3.4.7
2.4.
PDP Context Activation Failure Ratio [%]
Denotes the probability that the PDP context cannot be activated. It is the ratio of unsuccessful PDP context activation attempts to the total number of PDP context activation attempts:
# of unsuccessful PDP context activation attempts PDP Context Activation Failure Ratio [%] = ---------------------------------------------------------------------- x 100 total # of PDP context activation attempts
Start trigger: Phone sending PDP Context Activation Request message. Trigger point 5 in sections 3.1.7, 3.2.7, 3.3.7, and 3.4.7 Stop trigger: Phone receiving PDP Context Activation Accept message. Trigger point 8 in sections 3.1.7, 3.2.7, 3.3.7, and 3.4.7
2.5.
PDP Context Activation Time [s]
Denotes the length of the time period taken to activate a PDP context:
PDP Context Activation Time [s] = tPDP Context Activation Accept tPDP Context Activation Request
Start trigger: Phone sending PDP Context Activation Request message. Trigger point 5 in sections 3.1.7, 3.2.7, 3.3.7, and 3.4.7 Stop trigger: Phone receiving PDP Context Activation Accept message. Trigger point 8 in sections 3.1.7, 3.2.7, 3.3.7, and 3.4.7
2(15)
Public
2009-05-14
2.6.
PDP Context Cut-off Ratio [%]
Denotes the probability that a PDP context is deactivated without this being initiated intentionally by the user:
# of PDP context deactivations (not initiated by user) PDP Context Cut-off Ratio [%] = ------------------------------------------------------------------------ x 100 total # of successfully activated PDP contexts
Start trigger: Phone receiving PDP Context Activation Accept message. Trigger point 8 in sections 3.1.7, 3.2.7, 3.3.7, and 3.4.7 Stop trigger: Phone receiving the last data packet containing content. Trigger point 41 in sections 3.1.7 and 3.2.7 Trigger point 42 in sections 3.3.7 and 3.4.7 PDP context deactivation not initiated intentionally by the user can be caused by either SGSN failure or GGSN failure, so the PDP context may be deactivated either by the SGSN or by the GGSN. Note: The precondition for measuring this parameter is that a PDP context has been successfully established.
3.
FTP Specific KPIs
Two slightly different methods exist for calculating the KPIs. The difference is explained in the following extract from the ETSI specifications: (ETSI TS 102 250-2 V1.4.1 (2006-03), section 4.2) Currently two main views about the best way to reflect the user's experience are in place: One preferring the payload throughput philosophy and the other preferring the transaction throughput philosophy: Method A [...] defines trigger points which are as independent as possible from the service used, therefore representing a more generic view (payload throughput) Method B [...] defines trigger points on application layer, therefore representing a more service oriented view (transaction throughput). Both methods are described in the subsections that follow.
3(15)
2009-05-14
3.1.
3.1.1.
FTP Download Method A

Service Non-Accessibility [%]
Denotes the probability that a subscriber cannot access the service successfully due to a failure that has occurred either during PDP context activation or during service access. This means that the data transfer cannot be started: # of unsuccessful attempts Service Non-Accessibility [%] = -------------------------------------- x 100 total # of attempts Start trigger: Connect on command socket. Trigger point 5 in section 3.1.7 Stop trigger: Reception of the first data packet containing content. Trigger point 36 in section 3.1.7
3.1.2.
Setup Time [s]
Denotes the period of time it takes to access a service successfully, from the moment the dial-up connection is established until the first data packet is received:
Setup Time [s] = treception of first data packet tconnect to socket
Start trigger: Connect on command socket. Trigger point 5 in section 3.1.7 Stop trigger: Reception of the first data packet containing content. Trigger point 36 in section 3.1.7
3.1.3.
IP Service Access Failure Ratio [%]
Denotes the probability that, after successfully activating a PDP context, a subscriber cannot access the service, so that the data transfer cannot be started: # of unsuccessful service accesses IP Service Access Failure Ratio [%] = -------------------------------------------------- x 100 total # of service access attempts
Start trigger: First [SYN] sent. Trigger point 9 in section 3.1.7 Stop trigger: Reception of the first data packet containing content. Trigger point 36 in section 3.1.7
4(15)
Public
2009-05-14
3.1.4.
IP Service Setup Time [s]
Denotes the time period needed to establish a TCP/IP connection to the FTP server, from sending the initial query to a server until the first data packet is received:
IP Service Setup Time [s] = treception of first data packet tfirst [SYN] sent
Start trigger: First [SYN] sent. Trigger point 9 in section 3.1.7 Stop trigger: Reception of the first data packet containing content. Trigger point 36 in section 3.1.7
3.1.5.
Mean Data Rate [kbit/s]
Denotes the average data rate measured throughout the entire connect time (application throughput): file size [kB] x 8 Mean Data Rate [kbit/s] = ---------------------------------------tlast data packet tfirst data packet Start trigger: Reception of the first data packet containing content. Trigger point 36 in section 3.1.7 Stop trigger: Reception of the last data packet containing content. Trigger point 41 in section 3.1.7
3.1.6.
Data Transfer Cut-off Ratio [%]
Denotes the probability that a data transfer cannot be completed when it has been started successfully: # of incomplete data transfers Data Transfer Cut-off Ratio [%] = ------------------------------------------------------ x 100 # of successfully started data transfers
Start trigger: Reception of the first data packet containing content. Trigger point 36 in section 3.1.7 Stop trigger: Reception of the last data packet containing content. Trigger point 41 in section 3.1.7
5(15)
2009-05-14
3.1.7.
UE
Signaling Diagram
Network
1 Attach 4 5 PDP 8 9 12 13 16 17
Attach Request Attach Response PDP Context Activation PDP Context Activation Accept 2077 > 21 [SYN] 21 > 2077 [SYN, ACK] 2077 > 21 [ACK] 21 > 2077 FTP Response: 220 Login
2 3 6 7 10 11 14 15 18
Service
19 IP Service 21 24 25 28 30 31 34 36 37
Change Directory 2077 > 21 FTP Request: PORT X 21 > 2077 FTP Response: 200 2077 > 21 FTP Request: RETR Y 21 > 2077 FTP Response: 150 20 > 2113 [SYN] 2113 > 20 [SYN, ACK] 20 > 2113 [ACK] 20 > 2113 FTP DATA [ACK] 2113 > 20 [ACK]
20 22 23 26 27 29 32 33 35 First Data Packet 38
Data 40 41 43 46 48 50
Data Transfer 20 > 2113 FTP DATA [FIN, ACK] 2113 > 20 [ACK] 2113 > 20 [FIN, ACK] 20 > 2113 [ACK] 21 > 2077 FTP Response: 226 2077 > 21 [ACK] 39 Last Data Packet 42 44 45 47 49
6(15)
Public
2009-05-14
3.2.
3.2.1.
FTP Download Method B

Denotes the probability that a subscriber cannot access the service successfully due to a failure that has occurred either during PDP context activation or during service access. This means that the data transfer cannot be started: # of unsuccessful attempts Service Non-Accessibility [%] = -------------------------------------- x 100 total # of attempts Start trigger: Connect on command socket. Trigger point 5 in section 3.2.7 Stop trigger: Reception of the ACK from the first SYN, ACK. Trigger point 34 in section 3.2.7
3.2.2.
Setup Time [s]
Denotes the period of time it takes to access a service successfully, from the moment the dial-up connection is established until the first data packet is received:
Setup Time [s] = treception of first data packet tconnect to socket
Start trigger: Connect on command socket. Trigger point 5 in section 3.2.7 Stop trigger: Reception of the ACK from the first SYN, ACK. Trigger point 34 in section 3.2.7
3.2.3.
Denotes the probability that, after successfully activating a PDP context, a subscriber cannot access the service, so that the data transfer cannot be started: # of unsuccessful service accesses IP Service Access Failure Ratio [%] = -------------------------------------------------- x 100 total # of service access attempts Start trigger: First [SYN] sent. Trigger point 9 in section 3.2.7 Stop trigger: Reception ofhe ACK from the first SYN, ACK. Trigger point 34 in section 3.2.7
7(15)
2009-05-14
3.2.4.
Denotes the time period needed to establish a TCP/IP connection to the FTP server, from sending the initial query to a server until the first data packet is received:
IP Service Setup Time [s] = treception of first data packet tfirst [SYN] sent
Start trigger: First [SYN] sent. Trigger point 9 in section 3.2.7 Stop trigger: Reception of the ACK from the first SYN, ACK. Trigger point 34 in section 3.2.7
3.2.5.
Denotes the average data rate measured throughout the entire connect time (application throughput): File size (kB) x 8 Mean Data Rate [kbit/s] = ---------------------------------------tlast data packet tfirst data packet Start trigger: Reception of the first data packet containing content. Trigger point 34 in section 3.2.7 Stop trigger: Reception of the last data packet containing content. Trigger point 41 in section 3.2.7
3.2.6.
Denotes the probability that a data transfer cannot be completed when it has been started successfully: # of incomplete data transfers Data Transfer Cut-off Ratio [%] = -------------------------------------------------------- x 100 # of successfully started data transfers Start trigger: Reception of the first data packet containing content. Trigger point 34 in section 3.2.7 Stop trigger: Reception of the last data packet containing content. Trigger point 41 in section 3.2.7
8(15)
Public
2009-05-14
3.2.7.
UE
Signaling Diagram
Network
1 Attach 4 5 PDP 8 9 12 13 16 17 Service IP Service 21 24 25 28 30 31 34 36 37 Data
2 3 6 7 10 11 14 15 18
19
Change Directory 2077 > 21 FTP Request: PORT X 21 > 2077 FTP Response: 200 2077 > 21 FTP Request: RETR Y 21 > 2077 FTP Response: 150 20 > 2113 [SYN] 2113 > 20 [SYN, ACK] 20 > 2113 [ACK] 20 > 2113 FTP DATA [ACK] 2113 > 20 [ACK]
20 22 23 26 27 29 32 33 35 38
First Data Packet
Data Transfer 40 41 43 46 50 51 20 > 2113 FTP DATA [FIN, ACK] 2113 > 20 [ACK] 2113 > 20 [FIN, ACK] 20 > 2113 [ACK] 21 > 2077 FTP Response: 226 2077 > 21 [ACK] 39 42 44 45 49 52
Last Data Packet
9(15)
2009-05-14
3.3.
3.3.1.
FTP Upload Method A

Denotes the probability that a subscriber cannot access the service successfully due to a failure that has occurred either during PDP context activation or during service access. This means that the data transfer cannot be started: # of unsuccessful attempts Service Non-Accessibility [%] = -------------------------------------- x 100 total # of attempts Start trigger: Connect on command socket. Trigger point 5 in section 3.3.7 Stop trigger: Sending the first data packet containing content. Trigger point 35 in section 3.3.7
3.3.2.
Setup Time [s]
Denotes the period of time it takes to access a service successfully, from the moment the dial-up connection is established until the first data packet is sent:
Setup Time [s] = tsending first data packet tconnect to socket
Start trigger: Connect on command socket. Trigger point 5 in section 3.3.7 Stop trigger: Sending the first data packet containing content. Trigger point 35 in section 3.3.7
3.3.3.
Denotes the probability that, after successfully activating a PDP context, a subscriber cannot access the service, so that the data transfer cannot be started: # of unsuccessful service accesses IP Service Access Failure Ratio [%] = -------------------------------------------------- x 100 total # of service access attempts Start trigger: First [SYN] sent. Trigger point 5 in section 3.3.7 Stop trigger: Sending the first data packet containing content. Trigger point 35 in section 3.3.7
10(15)
Public
2009-05-14
3.3.4.
Denotes the time period needed to establish a TCP/IP connection to the FTP server, from sending the initial query to a server until the first data packet is sent:
IP Service Setup Time [s] = tsending first data packet tfirst [SYN] sent
Start trigger: First [SYN] sent. Trigger point 9 in section 3.3.7 Stop trigger: Sending the first data packet containing content. Trigger point 35 in section 3.3.7
3.3.5.
Denotes the average data rate measured throughout the entire connect time (application throughput): file size (kB) x 8 Mean Data Rate [kbit/s] = ---------------------------------tlast data packet tfirst data packet Start trigger: Sending the first data packet containing content. Trigger point 35 in section 3.3.7 Stop trigger: Sending the last data packet containing content. Trigger point 42 in section 3.3.7
3.3.6.
Denotes the probability that a data transfer cannot be completed when it has been started successfully: # of incomplete data transfers Data Transfer Cut-off Ratio [%] = ---------------------------------------------------- x 100 # of successfully started data transfers Start trigger: Sending the first data packet containing content. Trigger point 35 in section 3.3.7 Stop trigger: Sending the last data packet containing content. Trigger point 42 in section 3.3.7
11(15)
2009-05-14
3.3.7.
UE
Signaling Diagram
Network
1 Attach 4 5 PDP 8 9 12 13 16 17
2 3 6 7 10 11 14 15 18
Service
19 IP Service 21 24 25 28 30 31 34 35 38
Change Directory 2077 > 21 FTP Request: PORT X 21 > 2077 FTP Response: 200 2077 > 21 FTP Request: STOR Y 21 > 2077 FTP Response: 150 20 > 2113 [SYN] 2113 > 20 [SYN, ACK] 20 > 2113 [ACK] 2113 > 20 FTP DATA [ACK] 20 > 2113 [ACK]
Thrughput Cut off 39 42 43 48 51 53 55 58
Data Transfer
2113 > 20 FTP DATA [FIN, ACK] 20 > 2113 [FIN, ACK] 2113 > 20 [ACK] 2113 > 20 [FIN, ACK] 2113 > 20 [ACK] 2077 > 21 [ACK] 21 > 2077 FTP Response: 226 2077 > 21 [ACK]
40 Last Data Packet 41 44 50 52 54 56 57
12(15)
Public
2009-05-14
3.4.
3.4.1.
FTP Upload Method B

Denotes the probability that a subscriber cannot access the service successfully due to a failure that has occurred either during PDP context activation or during service access. This means that the data transfer cannot be started: # of unsuccessful attempts Service Non-Accessibility [%] = -------------------------------------- x 100 total # of attempts Start trigger: Connect on command socket. Trigger point 5 in section 3.4.7 Stop trigger: Reception of the ACK from the first SYN, ACK. Trigger point 34 in section 3.4.7
3.4.2.
Setup Time [s]
Denotes the period of time it takes to access a service successfully, from the moment the dial-up connection is established until the first data packet is sent:
Setup Time [s] = tsending first data packet tconnect to socket
Start trigger: Connect on command socket. Trigger point 5 in section 3.4.7 Stop trigger: Reception of the ACK from the first SYN, ACK. Trigger point 34 in section 3.4.7
3.4.3.
Denotes the probability that, after successfully activating a PDP context, a subscriber cannot access the service, so that the data transfer cannot be started: # of unsuccessful service accesses IP Service Access Failure Ratio [%] = -------------------------------------------------- x 100 total # of service access attempts Start trigger: First [SYN] sent. Trigger point 5 in section 3.4.7 Stop trigger: Reception of the ACK from the first SYN, ACK. Trigger point 34 in section 3.4.7
13(15)
2009-05-14
3.4.4.
Denotes the time period needed to establish a TCP/IP connection to the FTP server, from sending the initial query to a server until the first data packet is sent:
IP Service Setup Time [s] = tsending first data packet tfirst [SYN] sent
Start trigger: First [SYN] sent. Trigger point 9 in section 3.4.7 Stop trigger: Reception of the ACK from the first SYN, ACK. Trigger point 34 in section 3.4.7
3.4.5.
Denotes the average data rate measured throughout the entire connect time (application throughput): file size (kB) x 8 Mean Data Rate [kbit/s] = --------------------------------tfirst data packet tlast data packet Start trigger: Reception of the ACK from the first SYN, ACK. Trigger point 34 in section 3.4.7 Stop trigger: Reception of the [FIN, ACK] for the last data packet containing content. Trigger point 42 in section 3.4.7
3.4.6.
Denotes the probability that a data transfer cannot be completed when it has been started successfully: # of incomplete data transfers Data Transfer Cut-off Ratio [%] = ------------------------------------------------------- x 100 # of successfully started data transfers Start trigger: Reception of the ACK from the first SYN, ACK. Trigger point 34 in section 3.4.7 Stop trigger: Reception of the [FIN, ACK] for the last data packet containing content. Trigger point 42 in section 3.4.7
14(15)
Public
2009-05-14
3.4.7.
UE
Signaling Diagram
Network
1 Attach 4 5 PDP 8 9 12 13 16 17 Service IP Service 21 24 25 28 30 31 34 35 38 Data
2 3 6 7 10 11 14 15 18
19
Change Directory 2077 > 21 FTP Request: PORT X 21 > 2077 FTP Response: 200 2077 > 21 FTP Request: STOR Y 21 > 2077 FTP Response: 150 20 > 2113 [SYN] 2113 > 20 [SYN, ACK] 20 > 2113 [ACK] 2113 > 20 FTP DATA [ACK] 20 > 2113 [ACK]
Data Transfer
39 42 43 48 51 53 55 58
2113 > 20 FTP DATA [FIN, ACK] 20 > 2113 [FIN, ACK] 2113 > 20 [ACK] 2113 > 20 [FIN, ACK] 2113 > 20 [ACK] 2077 > 21 [ACK] 21 > 2077 FTP Response: 226 2077 > 21 [ACK]
40 Last Data Packet 41 44 50 52 54 56 57
15(15)

TEMS Investigation KPI Definitions - FTP

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

TEMS Investigation KPI Definitions - FTP

Uploaded by

Copyright:

Available Formats

TEMS Investigation KPI Definitions FTP