You are on page 1of 7

White Paper

Best Practices for


Monitoring Nortel
CS1000 VoIP Metrics
With eHealth for Voice
Dan Lewis
Voice Engineering
September 2005
Nortel CS 1000 systems enable QoS mechanisms by
Executive Summary classifying IP packets at point of origin:
This document summarizes issues in maintaining quality
in Voice over IP network transmissions and discusses how • CS 1000 Signaling Server classifies packets as signaling
eHealth for Voice can be used to monitor these issues in packets.
the Nortel Communication Server 1000 environment.
• Voice Gateway Media Cards classify packets as voice or
signaling packets.
VoIP Call Quality and the Network • Internet Telephones classify their packets as voice or
Environment signaling packets and mark them for priority.

For details on implementing QoS mechanisms on signaling


Need for Continuous Monitoring. servers, voice gateway media cards, and internet telephones
Voice over IP is intended to replace existing analog and at Layer 2 (802.1p) or Layer 3 (DiffServ), consult your
digital telephone applications in such a way as to be virtually Nortel documentation.
transparent to the end user. While the economic and strategic
benefits of VoIP are certainly notable, users of traditional
telephone technology, who have come to expect a high degree Network Obstacles to Successful Voice Transfer
of quality and reliability in voice transmissions, are not The Internet Protocol has become the de-facto standard
likely to be willing to make significant sacrifices in those for data communication in both local area and wide area
areas to embrace the new technology. networks. While it is remarkably successful in transmitting
quantities of data both large and small, it presents some
Much of the economic benefit of deploying VoIP comes inherent difficulties to voice communications which must
from making use of existing network infrastructure as a travel point to point in a more-or-less continuous stream.
medium for voice communications. Yet existing network Latency, packet loss, and jitter (variable delay) are all
structures have come into being to deliver data, primarily potential obstacles to successful voice transmission which
in the form of discreet files, not in the continuous streams must be overcome. They are also network issues which
which characterize voice and media transmission. must be monitored.

The vendors of today’s enterprise VoIP applications have had Latency. Latency, or delay, is the time between the sending
remarkable success in developing and utilizing mechanisms to of a packet and its receipt at a destination. Since no
overcome these limitations. Nortel CS 1000 servers, equipped communication is instantaneous, some latency is inherent
with voice gateway media cards, are capable of delivering in all transmissions. For traditional data traffic delays on
voice communication to IP telephone terminal equipment the order of a few hundred milliseconds are not generally of
with a quality and reliability that approaches that of traditional great importance. For voice communication, delays above
telephony. But for that quality and reliability to be maintained 150 ms have been shown to be noticeable to users, and
in an enterprise network, both the application and the network delays above 250 ms are generally observed as
must be monitored continuously so that potential issues may degradation in voice quality, especially with higher
be identified and corrected before they result in degradation compression codecs.
of service. In addition, it is important to manage and monitor
bandwidth utlization on the network, to insure that necessary Packet Loss. The loss of individual IP packets during
bandwidth can be provided for the continuous streams of transmission can be attributed to a number of causes.
packets with voice transmissions require. Some packet loss is characteristic of all IP communication
and is compensated for in data applications by protocols
which allow for the retransmission of lost packets.
Quality of Service Mechanisms to Ensure Because of the streaming nature of VoIP transmissions,
Bandwidth Availability retransmission of packets is not generally feasible. Voice
Bandwidth availability on shared networks tends to fluctuate applications are, however, somewhat more tolerant of
significantly since data transmission is bursty by nature. packet loss, since no one packet will contain more than a
Because VoIP packets require real-time transmission, it is fraction of a phoneme. In addition packet loss
crucial that some means of ensuring bandwidth availability concealment (PLC) is built into VoIP codecs to smooth out
for VoIP packets is instituted. This is generally accomplished the effects of packet loss by duplicating or synthesizing voice
through QoS mechanisms which allow VoIP packets to be from existing packets. PLC can typically repair a gap of 20 to
given priority at routers, switches and endpoints. For effective 40 ms with no discernable effect on the vocalization. For
transmission of voice over IP, networks must be QoS enabled longer gaps (the loss of a number of successive packets)
and configurable to support packet priorities1. the PLC algorithms are less effective and noticeable
clipping or choppy speech are the result.

2
Jitter. It is a characteristic of IP networks that packets do not The R value scale is from 0 to 100 and R values may be
remain evenly spaced in time as they traverse the network. approximately correlated to Mean Opinion Scores:
Jitter, or variable delay, is the variation in latency between
adjacent packets in the voice stream. Voice applications R MOS Opinion
require that packets are fed to the decoder at a generally
90 4.5 Very satisfied
constant rate; packets which do not arrive at the decoder
in time are considered lost. If jitter is very high, whole groups 80 4.0 Satisfied
of packets can be lost so that PLC algorithms cannot prevent
significant degradation of voice quality. Jitter generally 70 3.5 Some users dissatisfied
becomes more severe as available bandwidth becomes
compromised by network congestion, or as a result of 60 3.0 Many users dissatisfied
absent, or poorly configured QoS mechanisms.
50 2.5 Nearly all users dissatisfied
Most VoIP applications make use of a configurable buffer
Nortel recommends maintaining R values above 70.
on the receiving end to smooth out the stream of packets
fed to the decoder. However, buffering adds to the overall
latency of the transmission end to end. Effective configuration
of the jitter buffer requires that it be just large enough to
Using eHealth for Voice to Manage
smooth out the stream and minimize packet loss, without Nortel Communication Servers
being so large as to add significant delay.
Continuous monitoring of network performance is crucial
to maintaining voice quality in VoIP applications on shared
Voice Quality Metrics networks. Managers need to be able to monitor bandwidth
To a large degree, voice quality describes a subjective availability and VoIP packet latency, jitter and packet loss on
experience. While the bottom-line difference between a continuous basis to identify problems before they seriously
understanding and not understanding spoken language impair voice quality. By collecting and organizing data from
over a telephone connection is crucial and fundamental, the Nortel Server, eHealth for Voice is able to produce a
finer grain measurements of voice quality are not easily number of reports and data views which assist in monitoring
quantifiable. VoIP performance. In addition, the eHealth Policy Manager
can be configured to automatically alert operators and
Mean Opinion Scores (MOS). The industry has for some managers when user defined thresholds are met.
time used Mean Opinion Score (MOS) as a rough means
of quantifying voice quality at the destination end of a
communication link. Mean opinion scores, are based on Setting Baselines for Policy-Based Management
averaged ratings by a sample of listeners. Formally, MOS eHealth for Voice provides a number of reports to track
is determined by asking a sample of listeners to evaluate network issues and usage statistics. Running reports for
test sentences read over a communications link by male bandwidth utilization, call traffic statistics, QoS IP statistics, and
and female readers. Listeners use a 1 to 5 scoring system: Operational Measurements on a regular basis can provide a
1 (bad), 2 (poor) 3 (fair) 4 (good) 5(excellent). The baseline picture of VoIP performance in the network environment.
arithmetic mean of all scores is the mean opinion score for Key reports should be set as scheduled reports to be
the transmission. In practice, MOS serves as means of automatically delivered to appropriate personnel for daily
expressing estimates of voice quality. monitoring of conditions. Once the system has been monitored
in this way for several days, it should be possible to establish
The E-Model and R Values. The E-Model Transmission reasonable thresholds for key values such as bandwidth
Planning Tool, developed by the ITU, is a complex utilization, latency, jitter, packet loss, and overall voice
calculation which attempts to provide a quantifiable quality (R-value). These thresholds then can form the basis
estimate of overall voice transmission quality expressed as of policies monitored via eHealth Policy Manager so that
a single value, R. Nortel uses a simplified E-Model operators may be alerted automatically (via page, console
calculation which takes into account the characteristics of alert, email, or third-party application) when thresholds
Nortel's IP phones. The standard E-Model is described in are exceeded.
ITU-T standards E.107 and E.108. Nortel's R value calculation
is described in detail in Data Networking for Voice over IP As a continuing process reports should be analyzed and
(Nortel document 553-3001-160). thresholds adjusted as appropriate to meet changing
conditions. In addition to providing monitoring, the
collection and retention of data in the eHealth for Voice
database, allows analysis of issues and provides a basis
for further system and network management.

3
Direct Monitoring of Network Issues would like to know if peak bandwidth use increases going
A number of eHealth for Voice reports are designed to forward, so we consider a threshold of 30% and we set a
assist managers in monitoring network traffic conditions policy to alert if either interzone or intrazone peak bandwidth
affecting VoIP packets. In addition, data collected by exceeds 30 percent at any given point. An alert at 30%
eHealth for Voice from the Nortel Signaling Server, and will tell us that our bandwidth use for calls in a particular
organized into discrete tables within the eHealth for Voice zone is increasing, and it will warn us in time to investigate
database, may be viewed directly through the use of the the cause and to consider making changes to available
program’s query designer. Automatic alerting can be set bandwidth or the network topology if necessary.
up through the eHealth for Voice Policy Manager, so that
operators are alerted when threshold values are exceeded. We create a new node-based policy for the condition from
the eHealth for Voice console.
Bandwidth Utilization. eHealth for Voice stores bandwidth
utilization statistics in the Internet Telephone Zone Traffic
Data table. Data from this table can be viewed directly via
the Query Analyzer and filtered as appropriate. The Zone
Bandwidth Utilization report should be run on a regular
basis. Ideally it should be scheduled using the program’s
scheduled reporting feature, and delivered to appropriate
personnel for regular analysis. The report details interzone
and intrazone: total bandwidth, bandwidth usage and
bandwidth percentage used at peak. The statistics are
separately reported for each established bandwidth zone.

Once the report has been run for a period of time it should
be possible to set a reasonable baseline for peak performance
Figire 2. Adding a New Policy
so that appropriate personnel are notified when the
threshold is exceeded. We create a condition consisting of two statements linked
by OR. The statements are selected via the interface and
Bandwidth Example. To illustrate the process of reference the specific fields for interzone peak and intrazone
establishing and monitoring thresholds, consider the peak in the from the zone utilization table of the eHealth
following bandwidth example: for Voice database.

Figure 1. Selecting the Zone Bandwidth Report


Figure 3. Creating the Condition
Run the Zone Bandwidth Utilization report for several
days. Although here we are running the report manually Finally, on the conditions tab of the policy form we assign
from the report selector in the eHealth for Voice console, operation interval (all hours, so that the policy is in effect
it may be preferable to schedule the report to run daily and bandwidth is monitored at all hours), we set the severity
and be distributed automatically to the voice network level of the policy to “Warning” and assign an action plan.
manager (see your eHealth for Voice documentation). Action plans specify what happens when the threshold
conditions are met. In this example we use an action plan
Assume that the report indicates that peak bandwidth for
which will send an email message to an operator at any
both inter and intra zone calls is generally in the 20% —
hour, and an alert message to the operator’s console during
35% range. This appears to be a good level and it therefore
business hours when the console is monitored. eHealth for
appears that there is plenty of bandwidth available. But we

4
Voice Policy Manager allows great flexibility in specification Jitter. Managing jitter in VoIP networks consists largely in
of actions and action plans and can send messages to pagers finding the optimum size for the jitter buffer. Nortel allows
and to eHealth Live Health or other network monitoring jitter buffer size to be configured via the voice gateway
software as well as email and console alerts. For details on media card configuration as part of the DSP profile. A
setting up and maintaining eHealth for Voice policies via buffer too small will result in discarded packets, while
Policy Manager see the eHealth for Voice documentation. increasing the jitter buffer increases overall latency. Nortel
generally recommends that internet telephone firmware be
Once this policy is activated a notice will be sent anytime configured with a jitter buffer of 3.
peak bandwidth usage exceeds 30%. In this way eHealth
for Voice becomes an active watchdog monitoring the eHealth for Voice stores maximum and average values for
bandwidth utilization of your Nortel VoIP traffic. jitter broken down by telephone type. To monitor the amount
of jitter in your VoIP traffic, consult Operational Measurements
To assist the user in setting up policies for monitoring data, either through the eHealth for Voice query designer
VoIP performance, eHealth for Voice Policy Manager or on the Operational Measurements report. Baseline and
includes a number of pre-configured policy templates which alerting procedures would be similar to those outlined
may be modified as appropriate. There are templates for above. In addition, you should run the Operational
bandwidth usage, inter, and intrazone, as well as templates Measurements report or consult the data any time the
for monitoring latency, jitter, and voice quality. A detailed jitter buffer is changed in the voice gateway media card
list of policy templates is included in the eHealth for Voice (VGMC) or telephone firmware.
documentation.
Packet Loss. Packet Loss is perhaps the one area where
VoIP applications are somewhat more tolerant than most
data applications. Nortel’s packet loss concealment (PLC)
can typically handle losses of 40 — 60 ms without
noticeable loss of voice quality. Loss of a number of
consecutive packets, resulting in longer “holes” in the vocal
stream will, however become noticeable as clipping or
choppy speech.

Packet loss is also stored in the Operational Measurements


table and reported via the Operational Measurements report.
Packet loss is reported as a percentage of total at the gateway
and at the internet telephone. Monitor the Operational
Measurements report for percentage of packet loss. As with
Figure 4. The Completed Policy other network issues, an eHealth for Voice policy may be
set up to monitor packet loss send notifications as appropriate.
Latency. Maximum and average latency statistics, reported
by internet telephone type, is collected by eHealth for Voice Packet loss should be examined in conjunction with jitter to
and stored in the Operational Measurements table. The determine if it is caused by excessive jitter or insufficient
data may be viewed directly via the eHealth for Voice jitter buffering. It should also be considered in conjunction
Query Designer, or reported in the Operational with bandwidth utilization. Examination of the Operational
Measurements report. In general, values above 150 ms are Measurements report, where these items are reported
seen as signs of deteriorating voice quality. In the same together is helpful in spotting trends and isolating issues.
manner described above, a baseline for average and
maximum latency should be established by consulting a
Monitoring Voice Quality via R Value
run of operational measurements reports. Once baseline
The E-model attempts to quantify voice quality into a single
threshold values are established, a policy can be
value, R, by analyzing a number of parameters that affect
established based on either average or maximum values
voice quality (see discussion above). Another approach to
(or both), to notify an operator when latency values
monitoring voice quality is to monitor the R value itself. In
exceed the established thresholds. This would provide a
most instances this should not replace monitoring latency,
timely warning that latency issues might be beginning to
jitter, and packet loss, but it does give another approach
impair voice quality. Solutions to latency issues might be
which can be helpful in conjunction with monitoring the
changing QoS configuration, reducing the size of the jitter
“demons.” In fact, setting policies based on R-value may
buffer, increasing link capacity, or changing network
prove to be the most direct way of developing alerts when
topology to reduce the number of hops between nodes.
voice quality begins to be impaired. However, a reduced
R-value by itself tells little of its cause. For that the other
data is essential.

5
eHealth for Voice stores listening R-value data in the References
Operational Measurements table as Listening R-Factor. MCK Communications, Inc.
Nortel’s supplied R-value is used for Nortel IP telephones. Voice Over IP: a primer for telecom professionals.
For other hardware, eHealth for Voice calculates the R- MCK Communications, 2002
value using the standard E-model calculation.
Morrissey, Peter
By monitoring the R-value on the Operational Prepare Your Network for VoIP
Measurements Report it should be possible to establish a Network Computing, July 8, 2004 p.69-74.
reasonable baseline and threshold. Nortel recommends
that R-value be maintained above 70. Nortel Networks, Inc.
Data Networking for Voice over IP
Once thresholds have been determined, policies may be Document Number: 553-3001-160
created in the same way as with other elements to be Nortel Networks, Inc., October 2003
monitored. Policy Manager includes a number of pre-
defined policy templates for monitoring R-value and MOS Seligman, Dan
metrics. Introduction to IP Quality of Service Technology
(Concord Communications Technical White Paper)
Concord Communications, Inc., 2005
Succession QoS Thresholds
Through its QoS interface, Nortel Succession software
also allows storage thresholds for warning and
unacceptable levels of packet loss, jitter, latency, echo,
and R-value on the signaling server. eHealth for Voice
collects counts of threshold violations in the Internet
Telephone Zone Traffic table. If the QoS configuration is
set on the Succession, eHealth for Voice policies may be
set up based on number of violations.

Effective thresholds are arrived at by an ongoing process


of monitoring and refining. By storing data indefinitely and
making it available in trending reports, eHealth for Voice
can greatly assist in arriving at effective QoS thresholds.
Using eHealth for Voice Policy Manager in conjunction
with QoS output to alert appropriate personnel of
threshold violations, provides a 24/7 watchdog to insure
voice quality.

6
1
Quality of Service mechanisms can be monitored network-wide with other eHealth products, also available from CA. For a
discussion of the technology, see the white paper, “Introduction to IP Quality of Service Technology,” listed in the reference
section at the end of this paper.

Copyright © 2006 CA. All rights reserved. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies. This document is for your informational
purposes only. To the extent permitted by applicable law, CA provides this document “AS IS” without warranty of any kind, including, without limitation, any implied warranties of merchantability,
fitness for a particular purpose, or non-infringement. In no event will CA be liable for any loss or damage, direct or indirect, from the use of this document, including, without limitation, lost profits,
business interruption, goodwill or lost data, even if CA is expressly advised of such damages. MP295620206