Best Practices for

Handset Testing
with Spirent
Communications
Nomad

Spirent Communications
7340 Executive Way
Suite A
Frederick, MD 21704

Document Version: 2.2

 Best Practices for Handset Testing with Spirent Communications Nomad

Contents
1 Introduction ..................................................................................................................................................................... 3
2 Time Synchronization .................................................................................................................................................. 3
3 Optimal Downlink Volume (Input Level) ............................................................................................................. 5
4 Optimal Uplink Volume (Output Level) ................................................................................................................ 6
5 High Speed Internet Connectivity ........................................................................................................................... 8
6 Handset Placement and Settings ............................................................................................................................. 8
7 Phone Number Entry .................................................................................................................................................... 9
8 Caller ID Block ............................................................................................................................................................... 10
9 Redial After a Drop ...................................................................................................................................................... 11
10 Best Practices for Mobile Terminated Call Performance Testing ....................................................... 12
Appendix A: Nomad Troubleshooting Techniques .................................................................................................. 13
Downlink Data Missing................................................................................................................................................... 13
Uplink Data Missing ......................................................................................................................................................... 14
Unsuccessful Online Merge ........................................................................................................................................... 15

Bluetooth
The Bluetooth word mark and logos are owned by the Bluetooth® SIG, Inc. and any use of such
marks by Spirent Communications, Inc. is under license. Other trademarks and trade names are
those of their respective owners.

Copyright © Spirent Communications, Inc. 2013 2

 Best Practices for Handset Testing with Spirent Communications Nomad

1 Introduction
In order to obtain the most accurate results from the Nomad system, Spirent Communications has
compiled a set of Best Practices to follow during testing. The Best Practice categories are listed
below along with the applicable test types:

Best Practice Voice Quality Call Performance

Time Synchronization •
Optimal Downlink Volume (Input Level) •
Optimal Uplink Volume (Output Level) •
High Speed Internet Connectivity • •
Handset Placement and Settings • •
Phone Number Entry •
Caller ID Block • •
Redial After a Drop •
Best Practices for Mobile Terminated Call
•
Performance Testing

Each of the Best Practices described are followed by Spirent Communications engineers during field
testing, and are recommended for all users of the system.

For additional guidance in resolving issues with the Nomad system, please see Appendix A of this
document which addresses troubleshooting techniques.

2 Time Synchronization
Nomad calculates MOS in Voice Quality tests by comparing audio samples degraded by the
communication channel to the original source sample. Therefore, the audio waveform must be
centered within the playback window in order for Nomad to accurately calculate MOS.

The two waveforms below are both clipped. The first waveform is clipped on the left. The second
waveform is clipped on the right. Nomad will not be able to score either sample accurately and will
display a NULL value.

Figure 2-1 - Nomad Waveform Clipped on the Left

Copyright © Spirent Communications, Inc. 2013 3

 Best Practices for Handset Testing with Spirent Communications Nomad

Figure 2-2 - Nomad Waveform Clipped on the Right

The illumination of the MOS statistics area in red also indicates this problem.

This waveform is centered within the window and MOS will be calculated correctly:

Figure 2-3 - Centered Waveform

Synchronize the test laptop’s clock with the Audio Server to ensure consistent timing between the
two:

 During testing, select the Time Sync button.

 Set the Time Synchronization Source as either Internet Time Server or GPS.
o Note: Spirent does not recommend using Internet Time Server synchronization
for Voice Quality testing performed while drive testing. Substantial latency may be
introduced when the internet time protocol operates in a mobile environment. If
possible, use GPS time synchronization in this case.
 Click the Sync Now button.

Enable automatic time synchronization to prevent time drift in the synchronization between the
test laptop and the Audio Server:

 In the Time Sync dialog, check Enable automatic time synchronization.

 Set the period between time synchronization updates in minutes.
 Spirent recommends setting this to 5 minutes for frequent updates.

Copyright © Spirent Communications, Inc. 2013 4

 Best Practices for Handset Testing with Spirent Communications Nomad

3 Optimal Downlink Volume (Input Level)

Nomad calculates MOS in Voice Quality tests by comparing audio samples degraded by the
communication channel to the original source sample. Therefore, the downlink waveform must be
completely visible within the playback window and free of noise distortion for Nomad to accurately
calculate downlink MOS.

When using analog device connections, Spirent recommends running the Auto-Level Assist
Wizard prior to testing a new device to determine the optimal volume settings for that device.
Minor adjustments to the Wizard recommendations can be made during testing as described below.

The Downlink Volume (Input Level) for this audio sample is too high. The top and bottom of the
waveform are clipped and distortion is present. Nomad will not be able to score this sample
accurately:

Figure 3-1 - High Downlink Volume

An additional indicator of clipping, the Overdrive light for the saturated channel blinks red on the
Spirent ME hardware unit and on the Levels tab of the Settings dialog:

Figure 3-2 - Overdrive Indicator Illuminated

This sample represents a channel set at the appropriate Downlink Volume. The waveform is
intact and downlink MOS will be calculated correctly:

Copyright © Spirent Communications, Inc. 2013 5

 Best Practices for Handset Testing with Spirent Communications Nomad

Figure 3-3 - Appropriate Downlink Volume

To find the optimal Downlink Volume for any channel:

 Set the handset volume one level below the maximum. Further volume adjustments will be
performed using the Nomad software.
 While a test session is in progress, click the Settings button and navigate to the Levels tab.
 For the channel of interest, move the Input slider to the far right until the Overdrive light
illuminates in the GUI and on the hardware unit.
 Move the Input slider to the left one notch at a time and observe the Overdrive light during
the Recording state. The first position at which the Overdrive light stays off for the
current and next recording cycle should be used as the Input setting.
 The optimal Input slider position should fall within the Normal Operating Range of 170-
225 shown on the slider.
 The downlink insertion gain (PESQ) or attenuation (POLQA) shown in the test window will
likely read around 0.

Note that high downlink volume will not be an issue when using a Bluetooth connection. In cases
where the downlink insertion gain (PESQ) or attenuation (POLQA) for a Bluetooth channel is far
from 0 (roughly less than -8 dB for insertion gain or greater than 8 dB for attenuation), the Input
Level may be adjusted by moving the slider one or two notches to the right. The slider should be
kept within the Normal Operating Range. If MOS deteriorates after the adjustment, return the
Input Level to the original value.

4 Optimal Uplink Volume (Output Level)

Nomad calculates MOS in Voice Quality tests by comparing audio samples degraded by the
communication channel to the original source sample. Therefore, the uplink waveform must be
free of noise distortion in order for Nomad to accurately calculate uplink MOS.

When using analog device connections, Spirent recommends running the Auto-Level Assist
Wizard prior to testing a new device to determine the optimal volume settings for that device.
Minor adjustments to the Wizard recommendations can be made during testing as described below.

Copyright © Spirent Communications, Inc. 2013 6

 Best Practices for Handset Testing with Spirent Communications Nomad

The uplink speech waveform is not displayed during testing. In most cases, uplink volume
adjustment is only necessary if uplink MOS values are unexpectedly poor. The MOS Trend Window
in this sample presents symptoms of an uplink volume setting that is too high:

Figure 4-1 - High Uplink Volume

This uplink MOS trend in this sample represents a channel set at an appropriate uplink volume:

Figure 4-2 - Appropriate Uplink Volume

To find the optimal Uplink Volume for any channel in both the Bluetooth and analog connection
cases:

 While a test session is in progress, click the Settings button and navigate to the Levels tab.
 Move the Output slider for the channel of interest within the Normal Operating Range of
120-175.
 If uplink MOS deteriorates after the adjustment, return the Output Level to the original
value.

Copyright © Spirent Communications, Inc. 2013 7

 Best Practices for Handset Testing with Spirent Communications Nomad

5 High Speed Internet Connectivity

Nomad requires a sustained internet connection of at least 50 kbps in the following situations:

 Determining the optimal volume levels for a Voice Quality test using Automatic Level Assist.
 Viewing live uplink data during a Voice Quality test.
 Starting a Mobile Terminated call campaign.
 Obtaining the report for a Mobile Terminated or Voice Quality call campaign.
 Automatic internet-based time sync during testing (GPS time sync is recommended).

Follow these guidelines for the best results when using internet-dependent features:

 A sustained connection of 50 kbps is required to run these features.

 If the internet connection is lost during testing, these features will become unavailable.
Functionality will return when the connection comes back up. The internet connection may
need to be restarted manually.
 If possible, Spirent recommends using a different provider for the high speed internet
connection than for the voice network.
 Spirent recommends using the Auto Level Assist Wizard to optimize volume prior to testing
in a location with good RF coverage and a strong internet connection.

6 Handset Placement and Settings

Noise coupling can induce buzzing or an equivalent noise on the audio path and reduce MOS scores.
Follow these guidelines regarding handset placement and settings to minimize noise coupling:

 During testing, position the handsets as far away from the Spirent ME unit as possible.
 If an AC inverter is used, keep it away from both the hardware unit and the handsets.
 Ensure that the test PC is not running on batteries.
 Do not charge the handsets during testing.
 During testing, the handsets should be set to standard audio – not speakerphone.
 If a 2.5 mm adaptor is required, ensure that the correct adaptor has been selected and that
all adaptor connections are secure.
 Prior to testing, disable all Bluetooth connections with the exception of those used to couple
the handset to the hardware unit.

Copyright © Spirent Communications, Inc. 2013 8

 Best Practices for Handset Testing with Spirent Communications Nomad

7 Phone Number Entry

When performing Voice Quality testing using the Audio Server, it is imperative that the Number to
call and Number calling from phone numbers match the actual numbers used. This ensures that
uplink data may be obtained from the server and merged with the corresponding downlink data.

Nomad provides the Connection Assistant utility to assist in entering the phone numbers with the
highest accuracy. While Spirent strongly recommends the use of this utility for all customers,
international users outside of North America are urged to take advantage of the Connection
Assistant to ensure that phone numbers are entered in the correct format.

To use the Connection Assistant:

 On the Test Setup tab, select the Session button for any configured Mobile task.
 Confirm that the Uplink Device is set to Audio Server.
 Click the Connection Assistant button to start the utility.
 Click the Edit Audio Server Settings button. In the Edit Audio Server Settings dialog,
select the Audio Server Name and enter the Username and Password. Click OK to accept
the changes.

Figure 7-1 - Edit Audio Server Settings Dialog

 Dial the Audio Server using the handset to be connected to this channel. The handset does
not need to be connected at this time.
 All active calls to the Audio Server will be represented in the Connection Assistant drop-
down window using the {4 Digit DNIS}-{10 Digit ANI} convention:
o {4 Digit DNIS} – The last four digits of the Audio Server phone number called by the
mobile.
o {10 Digit ANI} – The phone number of the mobile being tested.
 If the test handset does not appear in the list, use the Refresh button to update. It may take
up to one minute for the handset to appear.
 Use the drop-down to select the correct number combination from the list.
 Click OK.

Copyright © Spirent Communications, Inc. 2013 9

 Best Practices for Handset Testing with Spirent Communications Nomad

Figure 7-2 - Connection Assistant

 The Number to call and Number calling from fields will be populated based on your
selection, removing the risk of manually entering this information incorrectly.

Figure 7-3 - Phone Number Settings Populated by the Connection Assistant

8 Caller ID Block
Nomad relies on the phone numbers of both the calling and called devices to identify and merge
session data. Nomad will not be able to correlate session uplink data if Caller ID Block is turned on.
To this end, it is critical that Caller ID Block is turned OFF for all test devices.

Copyright © Spirent Communications, Inc. 2013 10

 Best Practices for Handset Testing with Spirent Communications Nomad

9 Redial After a Drop

Unlike most RF collection tools, Nomad channels must be configured by the user in order to
automatically redial after a dropped call during Voice Quality testing. Auto redial is available for all
devices using a Bluetooth connection and for some devices using an analog connection. Bluetooth
auto redial may be enabled via the Channel Settings button for any configured Voice Quality task.

If auto redial is not enabled, the handset must be manually redialed before proceeding with testing
after a drop. This is important to keep in mind during long call testing when calls are allowed to
stay connected until they drop.

To enable auto redial:

 In the case of a Bluetooth connection:

o Auto redial is available for all devices.
o In the Channel Settings dialog, set the Audio Interface to Bluetooth.
o Check the Enable Bluetooth auto redial box and specify the number of seconds to
pause before redialing.
o With this configuration, a call that is dropped during testing will be automatically
redialed.
 In the case of an analog connection:
o Auto redial is available for phones which support the following options:
 Ability to maintain an analog and Bluetooth connection simultaneously.
 Ability to transfer from a Bluetooth connection to an analog connection.
 Ability for the Bluetooth connection to be re-established while the phone is
on a call.
o To utilize auto redial with a qualified device:
 Connect the test handset to the Spirent ME hardware using Bluetooth.
 Next, attach the test handset to the same channel using an analog
connection.
 Verify that the Bluetooth connection is still active.
 Set the Audio interface for this channel as Analog.
 Check the Enable Bluetooth auto redial option.
 Specify the number of seconds to Pause before redialing.
 With this configuration, a call that is dropped during testing will be redialed
using the Bluetooth connection. After three cycles of silence, Nomad will
then issue a command to transfer the audio from the Bluetooth to the analog
connection.

Figure 9-1 - Auto Redial Option in Channel Settings Dialog

Copyright © Spirent Communications, Inc. 2013 11

 Best Practices for Handset Testing with Spirent Communications Nomad

10 Best Practices for Mobile Terminated Call Performance Testing

In a Mobile Terminated Call Performance Test, the Call Server makes calls to the mobile test device
for call control testing. Statistics gathered during this type of test are based on the point of view of
the test handset. Spirent recommends following these best practices for the best results with
Mobile Terminated testing:

 Confirm that the test handset is properly configured with voicemail. This will ensure that
voicemail is properly detected during testing.
 The Access Timeout parameter for a Mobile Terminated call campaign must be set to a
larger duration than it takes the phone to go to voicemail. If Access Timeout is too short,
Nomad will fail to recognize a call sent to voicemail. Instead, such a call will be classified as
an Access Timeout before voicemail picks up. A 45-second Access Timeout is generally
sufficient for North American carriers.
 Nomad can take up to seven seconds to detect voicemail. In a Synchronous campaign it is
best practice to set Duration at least seven seconds longer than Access Timeout to ensure
that no Voicemail events are missed.

Copyright © Spirent Communications, Inc. 2013 12

 Best Practices for Handset Testing with Spirent Communications Nomad

Appendix A: Nomad Troubleshooting Techniques

This Appendix provides techniques for troubleshooting and resolving some of the most common
issues encountered when testing with the Nomad System:

 Downlink data missing (uplink data may be available)

 Uplink data missing (downlink data may be available)
 Unsuccessful online merge (data appears on Incomplete tab)

If the troubleshooting tips presented here are insufficient, or if you encounter other issues, please
contact Spirent Product Support for assistance.

Downlink Data Missing

In the event that downlink data is missing during Voice Quality testing (even though uplink data
may be available), check these possible causes:

Phone Volume  Is the phone volume level acceptable?

 Confirm that the handset volume is set one level below the
maximum.

Time Synchronization
 Is the test laptop’s clock synchronized with the Audio Server?
 If performing the test while drive testing, is the GPS time
synchronization method used?
 See Chapter 2 – Time Synchronization for instructions on
checking and setting synchronization.

Input Level  Is the Input Level properly set?

 Input Level refers to the volume of the sourced downlink audio
sample.
 See Chapter 3 – Optimal Downlink Volume to make the proper
adjustments.

Adapter Issues  Is there a problem with the 2.5 mm adapter?

 Confirm that you are using an appropriate adapter for the
device.
 Ensure that all adapter connections are secure.
 If necessary, try a different adapter to help isolate the source of
the problem.

Task Type /  Has the appropriate Task Type (i.e. Mobile, Base, Source Only,
Corresponding Device Record Only) been applied to this channel?
 In the Session dialog, is the Uplink, Downlink, Sourcing or
Recording Device set as appropriate?

Copyright © Spirent Communications, Inc. 2013 13

 Best Practices for Handset Testing with Spirent Communications Nomad

Uplink Data Missing

In the event that uplink data is missing during Voice Quality testing (even though downlink data
may be available), check these possible causes:

Internet Access  Does the test laptop have a sustained high-speed internet
connection of at least 50 kbps?
 If the internet connection was lost during testing, live uplink data
will become unavailable.
 Functionality will return when the connection comes back up.
 In some cases, the internet connection may need to be restarted
manually.
 See Chapter 5 – High Speed Internet Connectivity for more
information on this topic.

Phone Numbers  On the Test Setup tab, open the Session dialog for the affected
channel and confirm that the Phone Number Settings are correct.
 Spirent recommends using the Connection Assistant to confirm
phone numbers as described in Chapter 7 – Phone Number Entry.

Phone Volume  Is the phone volume level acceptable?

 Confirm that the handset volume is set one level below the
maximum.

Time Synchronization  Is the test laptop’s clock synchronized with the Audio Server?
 If performing the test while drive testing, is the GPS time
synchronization method used?
 See Chapter 2 – Time Synchronization for instructions on checking
and setting synchronization.

Output Level  Is the Output Level properly set?

 Output Level refers to the volume of the sourced uplink audio
sample.
 See Chapter 4 – Optimal Uplink Volume to make the proper
adjustments.

Adapter Issues  Is there a problem with the 2.5 mm adapter?

 Confirm that you are using an appropriate adapter for the device.
 Ensure that all adapter connections are secure.
 If necessary, try a different adapter to help isolate the source of
the problem.

Task Type /  Has the appropriate Task Type (i.e. Mobile, Base, Source Only,
Corresponding Device Record Only) been applied to this channel?
 In the Session dialog, is the Uplink, Downlink, Sourcing or
Recording Device set as appropriate?

Caller ID Block  Confirm that Caller ID Block is turned OFF for the test device.

Copyright © Spirent Communications, Inc. 2013 14

 Best Practices for Handset Testing with Spirent Communications Nomad

Unsuccessful Online Merge

On occasion, Nomad may be unsuccessful in merging the uplink and downlink data from a test
session into a unified log file for report generation. Such files will appear on the Data →
Incomplete tab at the conclusion of a test (successfully merged files appear on the Data →
Complete tab).

Files on the Incomplete tab contain an icon signifying each problematic channel:

 The half-circle icon indicates that Nomad does not have access to all of the data
required to merge this file. This might appear if no calls were started for a channel during a
test. This might also appear in the case of a Remote Unit test where Nomad can access the
locally collected data but not the remotely collected data.
 The yellow triangle icon indicates that an error has occurred during data collection
preventing Nomad from merging the uplink and downlink data associated with this test
case. One likely error cause is the incorrect entry of the Phone Number Settings. Another
common cause of this error is lack of internet connectivity at merge time.

Click on the or indicator icon to invoke the Modify Incomplete Logging Session dialog to
troubleshoot:

Retry Auto-Complete  If lack of internet connectivity is believed to be the reason for

the error, re-establish the internet connection and then use
the Retry Auto-Complete button to merge the data and move
to the Complete tab.

Phone Number Settings

 If the icon indicates Unable to download uplink scores,
the Phone Number Settings for the session are likely
incorrect.
 Select the Edit button to Edit Channel Configuration.
 Enter the correct values for Number to call and Number
calling from and then Close the dialog.
 Click Retry Auto-Complete to merge the data.

Uplink Scoring Utility  Some files that are unable to be merged automatically can still
be merged using the Uplink Scoring Utility.
 Contact Spirent if you require access to your uplink data for
merging.

Mark As Complete  If you believe that a channel has been flagged as incomplete in
error, use the Mark As Complete option to disregard the
incomplete data warning.
 Any report generated with this data may contain missing or
incomplete information.

Copyright © Spirent Communications, Inc. 2013 15