Professional Documents
Culture Documents
VoIP Testing With TEMS Investigation PC-based Clients
VoIP Testing With TEMS Investigation PC-based Clients
PC-based Clients
Technical Paper
No part of this document may be reproduced in any form without the written permission of the copyright holder.
TEMS is a trademark owned by Infovista. All other trademarks belong to their respective owner.
The contents of this document are subject to revision without notice due to continued progress in methodology,
design and manufacturing. Infovista shall have no liability for any error or damage of any kind resulting from the
use of this document.
www.Infovista.com | 1
Contents
1 Introduction....................................................................................................................................................... 3
2 How VoIP Works: A Brief Technology Overview .............................................................................................. 3
3 Testing VoIP with TEMS Investigation ............................................................................................................. 4
3.1 Physical Configuration............................................................................................................................... 4
3.2 Scripting .................................................................................................................................................... 5
3.2.1 Tips on Scripting ................................................................................................................................ 7
3.3 Voice Quality Measurement ...................................................................................................................... 9
3.4 Output ....................................................................................................................................................... 9
3.4.1 VoIP-specific Information Elements ................................................................................................... 9
3.4.2 Other Information Elements of Interest ............................................................................................ 11
3.4.3 VoIP Events ..................................................................................................................................... 12
3.4.4 VoIP KPIs (Key Performance Indicators) ......................................................................................... 12
3.5 Presentation in TEMS Investigation Windows ......................................................................................... 12
3.6 TEMS Test Setup .................................................................................................................................... 13
4 Troubleshooting.............................................................................................................................................. 13
4.1 Problem: Script Activity Fails ................................................................................................................... 13
4.2 Problem: Poor Audio Quality (PESQ/POLQA Score Low) ....................................................................... 13
5 Limitations ...................................................................................................................................................... 14
6 Appendices..................................................................................................................................................... 15
6.1 SIP Response Codes .............................................................................................................................. 15
6.1.1 Informational Responses ................................................................................................................. 15
6.1.2 Successful Responses .................................................................................................................... 15
6.1.3 Redirection Responses .................................................................................................................... 15
6.1.4 Client Failure Responses ................................................................................................................. 15
6.1.5 Server Failure Responses ............................................................................................................... 16
6.1.6 Global Failure Responses ............................................................................................................... 17
6.1.7 Extended Codes .............................................................................................................................. 17
6.2 Abbreviations .......................................................................................................................................... 17
www.Infovista.com | 2
1 Introduction
VoIP, Voice over IP, is a technology for delivering voice communications over IP networks such as the Internet.
Being a voice-over-data service, VoIP has characteristics from both realms. On one hand it is very delay-sensitive
(like circuit-switched voice); on the other it is subject to all of the various challenges associated with packet-
switched services, such as keeping down packet loss and jitter. This makes VoIP a very complicated service to
optimize.
The present paper describes VoIP in general terms and tells how to test and measure VoIP performance using
TEMS Investigation PC-based clients. The document does not cover VoIP testing with on-device clients, which is
another option available in TEMS Investigation.
The voice signal is digitized and encoded using audio codecs, just as in circuit-switched cellular telephony, and
then divided into IP packets for transmission over the packet-switched network. On the receiving side similar
steps are applied in the reverse order to reproduce the original voice stream: reception and decoding of IP
packets followed by digital-to-analog conversion.
The range of audio codecs used differs between VoIP implementations; some implementations rely on
narrowband and compressed speech, while others support high fidelity stereo codecs.
VoIP systems employ session control protocols to control the setup and teardown of calls. Examples of such
protocols are:
• H.323
Of these, SIP and RTP have gained particularly widespread use, and these protocols also figure in the present
document.
www.Infovista.com | 3
3 Testing VoIP with TEMS Investigation
The calling device (caller) is connected to one PC and the called device (callee) to the other. Audio is sent in
semi-duplex fashion between the parties, that is, in both directions but only in one direction at a time.
TEMS Investigation has a built-in PJSIP VoIP client; the VoIP clients thus reside in the PCs and not in the mobile
devices.
It should be noted from the outset that no further devices running data services can be connected to the PCs
during VoIP testing. See also chapter 5.
VoIP
server
VoIP
client Mobile phones VoIP
client
1
Two TEMS Investigation licenses are thus also required, as well as a special license option for VoIP.
www.Infovista.com | 4
3.2 Scripting
Two scripts are needed in TEMS Investigation, one for the caller and one for the callee. Predefined snippets,
VoIP PC Dial and VoIP PC Answer, are supplied with TEMS Investigation for this purpose.
It is worth underlining that the timing between caller and callee is essential. The callee must be registered with
the SIP server and finish its preparations for answering before the caller dials the call. See step below.
Screenshots of VoIP PC Dial snippet (left) and VoIP PC Answer snippet (right) as displayed in the TEMS
Investigation Service Control Designer. The numbering refers to the step-by-step description that follows
below.
www.Infovista.com | 5
Network Connect
First, both parties need to have an active data session. This is done in TEMS Investigation through the Network
Connect activity.
SIP Registration
Before a VoIP call can begin, both caller and callee must register with the SIP server to be used for VoIP. The SIP
Register activity is used for this purpose. Here you indicate the IP address or host name of the server to use. If
no special domain needs to be chosen, enter the server address under Domain, and leave Proxy empty. If on the
other hand you need to specify a domain within the server, enter the server address in the Proxy field and the
domain in the Domain field. You also specify the user and password the client should use when registering.
Answer
The callee must be ready to answer before the caller can initiate a call. To this end the callee executes the script
activity Answer with Call Type set to “VoIP PC”. In this activity you select the audio codec and encoding rate the
callee should use. The callee will communicate these settings to the caller, so that the parties agree on the same
codec and rate.
To ensure that the callee has reached Answer before the caller dials the call, you should insert a wait period in
the caller’s script. See section 3.2.1. This detail has been left out of the above diagram to keep things
straightforward.
Dial
The caller initiates the call by running the activity Dial, again with Call Type set to “VoIP PC”. In this activity you
indicate the codec the caller should use, which must be the same as the callee’s designated codec (Answer
activity, see step ). You also specify the codec rate and the phone number to call.
Once the call has been connected, voice quality can be measured on both sides using one out of several
algorithms supported (see section 3.3 for details on this matter). This is done with the Voice Quality activity,
where Audio Source is set to “VoIP (PC)”. The call durations should preferably differ between caller and callee,
so that it can be controlled which side hangs up the call. Compare step .
It is possible to store audio files containing the received audio. All audio sentences having a MOS score lower
than or equal to the MOS limit will be stored on the PC.2 If three consecutive voice quality measurement reports
with indication of silence are received, the VoIP call is terminated.
If for some reason you do not wish to measure voice quality, simply use a Wait activity instead of Voice Quality.
However, be aware that in this case you will not obtain any other VoIP quality measurements either, such as FER
and jitter buffer metrics (see the list in section 3.4.1), nor will MTSI Session Completion Failure events (“dropped
calls”) be generated.
2
Storage location: C:\Users\<user>\Documents\TEMS Product Files\TEMS Investigation <version no.>\PESQ.
www.Infovista.com | 6
Hangup
One of the parties (the one with the duration of the Voice Quality activity set lower) hangs up the call. This is to
ensure that the party hanging up has the time to do so before the other party unregisters; otherwise the hangup
will fail. This is done through the Hang Up activity (Call Type = “VoIP PC”). In the above diagram, the caller
performs the hangup.
SIP Unregister
Both sides unregister from the SIP server. This is done using the SIP Unregister activity.
Network Disconnect
Finally the data session is terminated as each party performs a Network Disconnect. If the snippet is executed in
a loop, this activity is necessary to force a disconnect from the network after each VoIP call (desirable for the
purpose of KPI calculation).
Suggested setup:
• On both sides, run all activities within a while loop. Add an extra Wait activity as the last item in each loop,
with the caller’s wait period longer than the callee’s, to make sure the callee is ready and waiting for the call
when the caller dials. Suggested wait durations are 30 s for the caller and 10 s for the callee. Compare step
in section 3.2 above.
• To ensure that the timing becomes right for the first VoIP call, start the script on the callee side first, then the
caller script.
If the parties are not in sync, that is, if the callee is not registered when the caller places the call, then the caller
will generate an MTSI Session Setup Failure event (see section 3.4.3).
www.Infovista.com | 7
Caller Callee
Use of while loops with VoIP scripting: caller (left) and callee (right).
www.Infovista.com | 8
3.3 Voice Quality Measurement
The Voice Quality script activity has an Algorithm parameter where the following choices can be made:
• PESQ P862.1
• Non-intrusive
PESQ and POLQA are industry standard algorithms for assessing voice quality as perceived by a human listener,
POLQA being a refinement of the older PESQ algorithm. Their output is a value on the MOS scale ranging from 1
(worst) to 5 (best). The score obtained is a function of the radio environment, of the speech codec and codec rate
used, and of other factors. Both algorithms are dealt with at length in the document “AQM in TEMS Products
(Including PESQ and POLQA)”.
Each of the two algorithms requires the purchase of a special license option to be enabled in TEMS Investigation.
Even without PESQ and POLQA, however, a more basic estimate of voice quality is obtained in the form of the
quantity VoIP FER Combined Packet Loss (see section 0), which is always computed. This is the meaning of the
term Non-intrusive.
Note that both uplink and downlink must use the same voice quality algorithm. It is not possible to use PESQ on
one link and POLQA on the other, even if you possess license options for both.
3.4 Output
3.4.1.1 Jitter
Packet jitter or delay variation as defined in IETF RFC 1889, section 6.3.1:
An estimate of the statistical variance of the RTP data packet interarrival time [...] The interarrival jitter 𝐽 is
defined to be the mean deviation (smoothed absolute value) of the difference 𝐷 in packet spacing at the
receiver compared to the sender for a pair of packets. As shown in the equation below, this is equivalent to
the difference in the “relative transit time” for the two packets [...].
If 𝑆𝑖 is the RTP timestamp from packet 𝑖, and 𝑅𝑖 is the time of arrival in RTP timestamp units for packet 𝑖,
then for two packets 𝑖 and 𝑗, 𝐷 may be expressed as
www.Infovista.com | 9
The interarrival jitter is calculated continuously as each data packet 𝑖 is received [...] using this difference 𝐷
for that packet and the previous packet 𝑖 − 1 in order of arrival (not necessarily in sequence), according to
the formula
|𝐷(𝑖 − 1, 𝑖)| − 𝐽
𝐽←𝐽+
16
The quantity 𝐽 is what is output in the “VoIP RFC 1889 Jitter” information element. The latter is updated once
every second.
www.Infovista.com | 10
3.4.1.3 Audio Quality Related
“Data” category
In the real time presentation, the PESQ and POLQA scores appear the moment they have been computed. When
loading a logfile for analysis, on the other hand, the PESQ and POLQA scores are moved backward in time to the
point when the corresponding speech sentence was received by the VoIP client. That is, sentences are aligned in
time with their quality scores. This is not much of an issue for PESQ, which takes only a fraction of a second to
compute, but it can be for POLQA, whose computation may require several seconds (the worse the degradation
of the signal, the more complex POLQA is to evaluate).
• RAN throughput at various protocol levels (details being dependent on the cellular technology used; the IEs
are found in the relevant category, such as “LTE”, “WCDMA”).
www.Infovista.com | 11
3.4.3 VoIP Events
These events underlie the KPIs in section 3.4.4:
MTSI Registration Failure One of the parties failed to register with the SIP server.
Time required for the terminal to register with the SIP server.
MTSI Registration Time
Also functions as a “success” event.
A VoIP session that was successfully set up failed to complete.
MTSI Session Completion Failure
Similar to “dropped call” for CS voice.
Duration of the VoIP session. Also functions as a “success”
event.
MTSI Session Completion Time
Note: This event does not have an associated KPI, since the
VoIP session duration is not a relevant performance measure.
The terminal failed in setting up a VoIP session. Similar to
MTSI Session Setup Failure
“blocked call” for CS voice.
Time required to set up the VoIP session. Also functions as a
MTSI Session Setup Time
“success” event.
TEMS Investigation also generates the following VoIP events, which are unrelated to KPI computation:
MTSI Registration Failure Ratio Denotes the probability that the terminal cannot register
(%) towards IMS when requested.
Denotes the time elapsing from the IMS registration request
MTSI Registration Time (s)
until the terminal is registered to IMS.
Denotes the probability that the terminal cannot set up an
MTSI session. An MTSI session setup is initiated when the
MTSI Session Setup Failure Ratio
user presses the “call” button and concludes when the user
(%)
receives, within a predetermined time, a notification that the
callee has answered.
Denotes the time elapsing from initiation of an MTSI session
MTSI Session Setup Time (s)
until a notification is received that the session has been set up.
Denotes the probability that a successfully set up MTSI call is
MTSI Session Completion Failure
ended by a cause other than intentional termination by either
Ratio (%)
party.
• VoIP Quality status window containing the information elements described in section 3.4.1
www.Infovista.com | 12
• VoIP Quality Line Chart tracking VoIP PESQ/POLQA scores and VoIP FER Combined Packet Loss, also
indicating MTSI events
4 Troubleshooting
• In the Data Reports message window, look into the VoIP Error Message category.
• If any other ports than SIP port 5060 and RTP port 4000 are used on the SIP server, the corresponding
settings have to be changed in the file <TEMS Investigation install dir>\Application\Configuration\
Investigation.Voip.config.
• If SIP response code 422 (“Session interval too small”) is received, set DisableTimers="false" in the same file.
• Check channel quality indicators and serving/neighbor signal strength. Example: In an LTE network, study
CQI, Serving Cell RSRP, and Neighbor Cell RSRP.
• In the IP Protocol Reports message window, look into the RTP messages.
www.Infovista.com | 13
5 Limitations
• You cannot have any other internet connections in parallel while running VoIP measurements. That is, the
PCs cannot be connected to any further IP addresses, whether through other external devices, through an
Ethernet cable, or by other means. All network interfaces except the testing devices, both fixed and
wireless, must be disabled. It is however possible to make CS voice calls with devices connected to the
PCs.
www.Infovista.com | 14
6 Appendices
www.Infovista.com | 15
409 Conflict
Gone
410
The user existed once, but is no longer available here
412 Conditional request failed
413 Request entity too large
414 Request URI too long
415 Unsupported media type
416 Unsupported URI scheme
417 Unknown resource priority
Bad extension
420
Bad SIP protocol extension used, not understood by the server
421 Extension required
422 Session interval too small
423 Interval too brief
424 Bad location information
428 Use identity header
429 Provide referrer identity
433 Anonymity disallowed
436 Bad identity info
437 Unsupported certificate
438 Invalid identity header
480 Temporarily unavailable
481 Call/transaction does not exist
482 Loop detected
483 Too many hops
484 Address incomplete
485 Ambiguous
486 Busy here
487 Request terminated
488 Not acceptable here
489 Bad event
491 Request pending
Undecipherable
493
Could not decrypt S/MIME body part
494 Security agreement required
www.Infovista.com | 16
502 Bad gateway
503 Service unavailable
504 Server timeout
Version not supported
505
The server does not support this version of the SIP protocol
513 Message too large
580 Precondition failure
6.2 Abbreviations
AMR-NB Adaptive Multi Rate Narrowband
AMR-WB Adaptive Multi Rate Wideband
BLER Block Error Rate
CQI Channel Quality Indicator
www.Infovista.com | 17
FER Frame Erasure Rate
www.Infovista.com | 18