Cisco Prime Workshop Prime Collaboration Manager Lab guide

1. Abstract
The objective of this lab is to discover Cisco Prime Collaboration Manager capabilities. The associated document can also be used to assist a system engineer in demonstrating Cisco Prime Collaboration Manager during a POC. It does not cover all CPCM capabilities, but commonly asked customer requests during a POC plus some extras. This document will cover the following topics : Device preparation for monitoring CPCM platform Installation Network Discovery Video session troubleshooting Proactive troubleshooting

2. CPCM Overview
CPCM is a video session management and troubleshooting tool that provides the following features :

A traditional CPCM deployment within a customer network would look somehow like this:

CPCM is the result of a joint inititiative of Cisco IT and NMTG and has appeared on the market in April 2011. Release 1.0 was a that time focusing on Cisco Telepresence equipments (CTS Manager, CTMS and CTS endpoints). Release 1.1 (which will be used throughout this lab) has been announced in October 2011 and adds supports for the (former) Tandberg equipments (TMS, VCS, TPS and Codian MCUs, EX- and C- series endpoints). Release 1.1 supports the following list of equipments:

CPCM is available for free download from the Cisco web site at http://www.cisco.com/cisco/software/release.html?mdfid=283929900&flowid=29941&softwareid=283781546&rele ase=1.1&relind=AVAILABLE&rellifecycle=&reltype=latest For those with a CCO login. It has a 90 day evaluation license built-in with 5000 units. Units are consumed in the system on a CODEC type basis. CODECs supported by CPCM Rel 1.1 are:

The license unit calculator available in the system in the Administration menu allows the user to compute the number of credits needed for a specific installation based on the numbers and types of CODECs deployed throughout the network.

3. Preparing Devices for Monitoring

3.1. Device access
CPCM uses several protocols to collect configuration and statistics data from the various video service devices, and also to initiate actions on those devices (path discovery during video session troubleshooting for example). The following diagram provides a list of those protocols:

For the Cisco Telepresence equipments:

For the (former) Tandberg equipments:

This lab will be primarily focusing on the latter. As we can see on the above picture, we need mainly SNMP and http/https (depending on device configuration) credentials to be able to monitor the various equipments that compose the video service network. In addition to the protocols that are used by CPCM to exchange data with the video service devices, protocols also need to be set up for CPCM to exchange data with the network infrastructure devices (routers and switches). There are three medianet features for which CPCM need access to those equipments and below is a summary on how data are exchanged depending on the medianet feature in use:

Mediatrace: operations are initiated on the first L3 device on the RTP path that supports the mediatrace initiator role (refer to the medianet lab for details) using the WSMA and statistics are also collected from the devices that support the mediatrace responder role using the WSMA NOTE: Cisco IOS Web Services Management Agent (WSMA) defines a set of web services, through which a network device can be fully managed from configuration to on-going monitoring to troubleshooting. WSMA operates in both listener mode (Connections are initiated by external applications.) and initiator mode (WSMA initiates the connections.). WSMA supports HTTP, HTTPS, Secure Shell Version 2 (SSHv2) and TLS transports; provides XML encoded model for configuration and operational data; publishes schemas for web services; avoids screen scraping; allows faster NMS application development; and provides faster response times compared to traditional telnet-based access mechanisms.

Performance Monitor: CPCM does not configure anything for Perf Mon and the Perf Mon feature must be preconfigured so that CPCM can retrieve the statistics from the devices where it is supported and configured. CPCM collect the Perf Mon statistics using SNMP.
MIB Description

CISCO-FLOW-MONITOR-TCMIB

This MIB module defines textual conventions used to describe flow monitoring data.

CISCO-FLOW-MONITOR-MIB

This MIB module defines objects that describe monitored flows carrying media streams, statistics relating to these flows, and a history of computed metrics for these flows.

CISCO-RTP-METRICS-MIB

This MIB module defines objects describing RTP metrics.

CISCO-IP-CBR-METRICSMIB

This MIB module defines objects describing IP CBR metrics.

The table above lists the MIBs that have been introduced for Performance Monitor. Please refer to the White Paper on Application Performance Assurance using Perf Mon available at http://www.cisco.com/en/US/solutions/collateral/ns340/ns856/ns156/ns1094/whitepaper_c11-653060.html IP SLA VO: CPCM initiates the IP SLA VO tests on the device that acts as initiator (refer to the medianet lab) using CLI and it collects the statistics using SNMP and the CISCO-IPSLA-VIDEO-MIB.

The table below summarizes the access methods and associated rights that must be set on the various equipments:

3.2. SNMP Communities

This has been set during the LMS lab for the infrastructure equipments (routers and switches):

It must also be set in: TMS

You can check the settings on http://192.168.40.89/tms - user = administrator and password = C1sc0123 VCS

You can check the settings on http://192.168.40.80 - user = admin and password = C1sc0123 TPS/MCU and endpoints

You can check the settings on: o The EX60 endpoint on http://10.14.59.2 under Configuration Advanced Configuration Network Services SNMP (user = admin and password = TANDBERG)

The P52 endpoint on http://10.14.69.1 under Configuration Advanced Configuration Network Services SNMP (user = admin and password = TANDBERG)

CUCM

You can check the settings on http://192.168.40.4 under Cisco Unified Serviceability SNMP V1/V2 Community String (user = Administrator and password = C1sc0Cxd92)

3.3. http or https access

Below are snippets to allow http access for the infrastructure equipments (routers and switches) : username admin privilege 15 password 0 cisco aaa new-model ! aaa authentication login default local ! ip http server ip http authentication local no ip http secure-server ip http timeout-policy idle 60 life 86400 requests 10000 ! wsma agent exec profile wsma_listener_http wsma agent config profile wsma_listener_http ! wsma profile listener wsma_listener_http transport http ! wsma profile listener wsma_listener_https transport https

But again, credentials for http or https access to the video services equipments (TMS, VCS, TPS/MCU and endpoints) must be provided to CPCM according to what is configured: in TMS

In order to retrieve the information about scheduled meetings, the user set for http access must be enabled for read access to List Conferences All under Booking as shown below.

in VCS

in the TPS/MCU and endpoints (below the example of an EX60):

3.4. Medianet features

CPCM was the first Cisco application to make use of medianet features. Release 1.0 already had support for mediatrace and IPSLA VO. Release 1.1 has added support for Performance Monitor. Below is the list of the medianet features supported by the various Cisco equipments in the 2.2 release of medianet.

3.5. Medianet mediatrace feature

To be able to use the mediatrace feature (medianet path determination and RTP statistics collection), the following need to be configured in the routers (DIST-W, CORE-W, CORE-E, DIST-E). mediatrace responder mediatrace initiator source-interface Loopback0 Since the routers are shared between the PODs, this has already been configured for you. Note: CPCM does not currently make use of the mediatrace feature on Cisco switches. This is planned for Release 1.2.

3.6. Medianet IPSLA Video Operations (IPSLA VO) feature

To be able to use the IPSLA VO feature (CPCM Proactive Troubleshooting), the following is needed in the switches configuration:

3.7. Medianet Performance Monitor feature

To be able to use the Performance Monitor feature of medianet, a specific service-policy needs to be applied on the switch/router interfaces that we want to monitor from a traffic flow standpoint. Below is a very simple example (refer to the medianet lab for details): service-policy type performance-monitor inline input flow monitor inline record default-rtp service-policy type performance-monitor inline output flow monitor inline record default-rtp For detailed configuration of the Performance Monitor feature, please refer to the Medianet Deployment guide available on the Cisco web site at http://www.cisco.com/web/solutions/medianet/docs/guide_c07-684466_v2.pdf

3.8. NTP Configuration

NOTE : NTP configuration is mandatory for IPSLA VO successful operations. In our configuration, two NTP servers can be used, 171.68.10.80 and 192.168.40.254 (which is a client of the first one). Check the configuration on the various equipments and use the show ntp status to verify the connection. clock timezone CET 1 0 clock summer-time CET recurring ntp server 171.68.10.80

4. Installing CPCM FOR REFERENCE SECTION

Installation has already been done and is described in this section for Reference

4.1. CPCM Installation.

CPCM is provided as an OVA file only and must be installed on a ESXi 4.1 (minimum level) environment. Connect to a VCenter and select : > file > deploy OVF Template and point to the OVA that you have downloaded from CCO. Below are the characteristics of the CPCM virtual machine as it will be deployed:

When finished, power on the VM and open a console to configure it

The following information must be provided Host Name Ip address Network Mask Default Gateway Domain Name DNS server IP address NTP server IP or Name

Timezone Password for admin (used for CLI console or ssh to CPCM)

Go through the Software License Agreement pages by hitting enter .

Accept the Software License Agreement.

After reboot, the CPCM application can be accessed via http://192.168.40.1x where x is the POD number. Patch cpcm-patchbundle-1.1.0-11714.tar which is available on the Cisco web site must then be installed using the patch install shell command available in the admin console. Below is the output of a show version issued at the admin console after successful patch installation.

Reboot the server or restart the CPCM application (named emsam). After reboot, you can use the admin shell to check the application status as shown below:

Important: Oracle must be shown in the list of active processes. Upon first login, you must use admin / admin which is the default web admin login set during CPCM installation.

The system will then ask you to change the default password. NOTE: default security policy in CPCM does not allow for simple passwords like Cisco. Passwords must at least contain upper case and lower case characters, numerics and one special character. Example used in this setup is C1sc$123.

Once the new password is successfully set, the system will automatically log you out and ask that you log back in using the newly set password. Default home page contains a certain number of real time statistics dashlets as shown below (they are empty here since we have not yet started to monitor anything).

4.2. Time Zone setup

Go to Administration Configuration User Preference Configuration

Set the correct time zone and click save.

4.3. LMS and NAM access setup

Go to Administration System Configuration

Enter the LMS IP address for your POD (10.3.198.21x, where x = POD number) and associated credentials:

Enter the NAM Management IP addresses and their credentials with the corresponding hosting devices:

4.4. Credentials configuration

Go to Inventory Device Inventory:

Go to Manage Credentials :

You will use the Credential Profiles entry menu to set the credentials for the various equipments that need to be discovered.

Depending on the equipment you are setting credentials for, you will have to enter one or more of : SNMP communities http / https access CLI access JTAPI user

Equipment types that are currently supported are listed on the pull-down menu shown below:

Enter the Credentials as follows: TMS: o o o

SNMP Read Community = public http username = cpcm http password = C1sc0123

You can use the verify tab to check your credentials against the related equipment.

VCS: o o o

SNMP Read Community = public http username = admin http password = C1sc0123

CUCM: o o o o o

SNMP Read Community = public http username = Aministrator http password = C1sc0Cxd92 JTAPI username = CPCM JTAPI password = C1sc0Cxd92

Video endpoints: they are attached on POD99, one on the West side and one on the East side o SNMP Read Community = public o CLI login username = admin o CLI password = TANDBERG o http username = admin o http password = TANDBERG

Ethernet switches: o SNMP Read Community = public o CLI login username = admin o CLI login password = C1sc0123 o CLI enable password = C1sc0123 Enter the IP addresses for your PODs Ethernet switches (10.14.20x.1 and .2) and for POD99s Ethernet switches (10.14.209.1 and .2)

Routers: SNMP Read Community = public CLI login username = admin CLI login password = C1sc0123 CLI enable password = C1sc0123 Enter the IP addresses for the six WAN routers (10.14.200.1 to .6) and the WAN links subnets (10.14.1.x to 10.14.5.x)

4.5. Device discovery

To run the discovery, go to Discover Devices:

Discovery of the video service equipments (including VCS and video endpoints) is done via TMS. Enter TMS IP address and CUCM IP address:

You can follow what happens then by looking at the discovery job status under List Discovery Jobs:

As you can see, the system automatically goes through a discovery of the VCS (as it is referred to in TMS configuration) that it will use to in turn discover the TPS / MCU and video endpoints that are registered to it. When all the jobs show a completed status, go to Device Inventory and the display should look like this:

Successful discovery puts the various equipments in the managed state. Other states available are:

Unreachable: the device cannot be found in the network Inaccessible: the device cannot be accessed using the provided credentials

The bottom of the screen displays details about the equipments configuration. Example below is for the Profile 52, which is shown as registered to the VCS:

Example below is for the EX90, which is shown as registered to the CUCM:

After the video service equipments discovery, we need to discover the infrastructure equipments, that is the switches and routers. We use the same discovery process, but need to enter the IP addresses of those devices, including your PODs switches, POD99s switches and the six routers.

After successful discovery, the switches and routers also go to the managed state.

For these equipments, the system displays their medianet-enabled features under: Mediatrace Role: Initiator / Responder IPSLA Role: Responder Performance Monitor: Configured

This needs to be successfully discovered for CPCM to be able to use the corresponding features during a video session troubleshooting or during proactive troubleshooting. Go to the home page. The lower part of the welcome page now includes information about the video service deployment: the video endpoints, the management servers (TMS), the multipoint control equipments (not present in our scenario), the signaling servers (VCS and CUCM).

5. Video session troubleshooting

This is the main feature of CPCM. The session monitoring feature allows you to visualize in real-time the video sessions that exist in the network, and , based on the alarms that are raised on them, to determine those that deserve specific attention. The troubleshooting feature will allow you to drill down if needed into the various components that participate in the video session to locate the source of the problem, and to collect statistics that are relevant to that session. Video sessions exist in the nework. To visualize them go to Monitoring Session Monitoring:

You should have one video session displayed as shown below:

CPCM displays only the sessions that are active and for the current day (those that started today). To display the others, both those that already happened and those that are scheduled to happen, use the all sessions icon a nd also the calendar pull-down menu (shown below).

As you can see, we can have a 7-day look back, and a 3-day look ahead, in addition to the current day (March 12 in the example above). Sessions that are scheduled can be retrieved from TMS using the Booking API (and/or from CTS Manager). The retrieval can be manual, using the Import Sessions shown below:

After import, the scheduled conferences appear in the sessions list:

The retrieval happens also automatically and the retrieval recurrence is configured under Administration Device Monitoring Configuration:

On the left, the sessions are displayed in a table format with their attributes. Hover your mouse on the various icons to find out what those attributes are.

The topology diagram on the right, shows the session that is selected in the list on the left. Click on both endpoints to show the statistics that are relevant to the selected endpoint. Use the see all button to display more statistics. Hover your mouse over the endpoints on the topology diagram. A quick view icon appears close the endpoint icon.

Click on it to get information about the endpoint.

Action buttons are available. What is their purpose? At the bottom left of the screen, use the peripherals, system information and session information links to display information about the endpoint and the session it is in.

The peripherals information pane displays the current state of the audio and video equipments that are connected to the CODEC. It allows to differentiate between problems with the endpoint itself and problems with the session when an alarm is raised on the video session both on the video session list and on the topology diagram.

In the example above we see that the alarm raised on the session line is due to a peripheral problem on one of the endpoints (the Profile 52). There is no alarm shown in the session statistics that are reported by the endpoints. Packet loss and jitter are below the thresholds that have been set for the installation. Use the see all link to display detailed statistics:

Click on the link between the endpoints. The display below shows the characteristics of a normal session behavior.

The display below shows the characteristics of an abnormal session behavior.

Another way to access the troubleshoot network link link is to hover your mouse on the session line until you reach the quick view icon. Complementary information about the session is displayed.

Click on the troubleshoot session link.

Troubleshooting can be started on one flow direction or the other or both. CPCM Rel 1.1 can support a maximum of 25 concurrent troubleshooting sessions at a given time. Troubleshooting can be manual (as done here) or it can be started automatically: If one of the endpoints is in the watch list, troubleshooting starts automatically as soon as the endpoint enters a video session If one of the thresholds set in the system for packet loss, jitter or latency, is exceeded during a video session, troubleshooting starts automatically for that session if automatic troubleshooting is sel ected in the configuration pane as shown below (Administration Event Configuration).

Start the troubleshooting in any of the directions shown on the troubleshooting page shown before.

The troubleshooting process starts a sequence of steps (step 1 above, step 3 below).

Until the whole path between the endpoints is discovered and displayed in the troubleshooting topology. This might take some time, depending on the network complexity, so please be patient . You can use this wait time to explore other CPCM features like the summary statistics dashlets on the home page or the reports and inventory features.

Please refer to the lab topology below to map the path above based on how the lab network is built. The EX90 video endpoint is connected to the Catalyst switch on the East side of POD 99, then the flow goes thru the aggregation, distribution and core routers, to the Profile 52 which is attached to the Catalyst switch on the West side of POD99.

Cat 3750 ou 3560 SW-PODx-W 10.14.20x.2

SRE-NAM 10.15.4.1 SRE-NAM 10.15.3.1

Cat 3750 ou 3560 SW-PODx-E 10.14.20x.1

POD 1

POD 1

POD 2
ISR 2811 AGG-W 10.14.200.1 ISR 2811 DIST-W 10.14.200.2 ISR 2911 CORE-W 10.14.200.3 ISR 2911 CORE-E 10.14.200.4 ISR 2811 DIST-E 10.14.200.5 ISR 2811 AGG-E 10.14.200.6

POD 2

POD 3

POD 3

POD 4
NME-NAM 10.15.6.1 NME-NAM 10.15.5.1 NME-NAM 10.15.2.1 NME-NAM 10.15.1.1

POD 4

POD 5

POD 5

LMS-PODx (10.3.198.21x)
POD 6

PAM-PODx (192.168.40.2x)

POD 6 POD 99
entnmsvpn-eu.cisco.com VPN Server

CPCM-PODx (192.168.40.1x)
POD 99

TMS (192.168.40.89) VCS (192.168.40.80)

CUCM (192.168.40.4)
AD (192.168.40.1)
Internet

DNS (192.168.40.1/10.3.192.209) Default GW and NTP (192.168.40.254)

Presentation_ID 2006 Cisco Systems, Inc. All rights reserved. Cisco Public

Polling the equipments on the path starts after the path discovery is complete. For mediatrace enabled infrastructure equipments, a blue I icon appears as soon as mediatrace statistics are available for those equipments. What is the other blue icon that is displayed on the routers?

The logs tab displays the sequence of operations that have been processed:

The Catalyst used as access switches on the path between the endpoints do not have mediatrace statistics ready for use. Why? Hover your mouse on one of the equipments icon, to make the quick view icon appear.

Then click on the quick view icon.

This menu can be used to display statistics pertaining to the equipment. System Status and Interface Details are shown for every equipment for which the correct SNMP community is set. What is needed for the Mediatrace Flow Information and View all Flows tabs to appear? Which specific features do they correspond to? How does that relate to the information displayed on the upper left part of that menu? According to the flow direction that we are troubleshooting, which router has initiated the mediatrace operations upon CPCM request? Login to that router and display the mediatrace CLI commands that have been sent to the router by CPCM. Below is an example of what you should get.

mediatrace profile perf-monitor EMSAM_PROFILE_FLOW metric-list rtp clock-rate dvi4 90000 admin-params sampling-interval 10 mediatrace path-specifier EMSAM_PATH_685864824 destination ip 10.14.69.1 port 2358 source ip 10.14.59.2 port 16426 mediatrace flow-specifier EMSAM_FLOW_685864824 source-ip 10.14.59.2 source-port 16426 dest-ip 10.14.69.1 dest-port 2358 mediatrace session-params EMSAM_PARAMS_685864824 response-timeout 10 frequency 30 inactivity-timeout 180 route-change reaction-time 15 mediatrace 685864824 path-specifier EMSAM_PATH_685864824 session-params EMSAM_PARAMS_685864824 profile perf-monitor EMSAM_PROFILE_FLOW flow-specifier EMSAM_FLOW_685864824 mediatrace schedule 685864824 life forever start-time now Based on what you have learned during the medianet lab, what kind of polling does this mediatrace operation start? Click on Mediatrace Flow Information.

What can you make out the information that is displayed? Click on the View all Flows tab, and select one of the routers interfaces.

Then click start. The system starts collecting statistics pertaining to your selection: all will display information about all the flows that go thru the selected interface RTP will display information about the RTP traffic only (both video and audio)

This information can be used to validate the boundaries that have been set in the routers configuration to limit the bandwidth of the various types of traffic. It also helps find out which flow is impacting the others, together with their operational characteristics (packet loss and jitter). Click on Network Diagnostics.

This allows to cross-launch LMS (above) or NAM (below) that you configured earlier.

Click on device view on the Network Diagnostics menu under the LMS tab.

Click on faults (24 hours).

Click on view/edit configuration. This allows to find out about the mediatrace CLI that we talked about earlier.

From the topology displayed in the session troubleshooting, open the Medianet Path View pane.

The various subsequent tabs show the statistics collected thru SNMP polling (CPU, Memory) and those collected using mediatrace (Bit Rate, Packet Loss, Jitter, DSCP). The equipments on the RTP path associated with the video session we are troubleshooting are displayed on the X axis. The attribute on the left of each couple (CPU in the CPU,Memory couple) is displayed on the Y axis, and the attribute on the right of each couple (Memory in the

CPU,Memory couple) is displayed as the radius of the bubble. When one of the attributes violates the threshold set in CPCM configuration, the bubbles color changes to reflect a potential issue. Click on CPU, Packet Loss.

Find an explanation for the blue square icons vs. the green bubbles that are associated with the various equipments. Click on Bit Rate,PacketLoss.

Data are not displayed for the access switches or the aggregation routers. Why? The following screen capture from the CPCM US demo site that can be scheduled on NMTG IWE, is FOR YOUR REFERENCE ONLY.

As the session we have been troubleshooting does not reflect any issue, this is to show a practical case with a session that does have an issue.

First of all, this session topology shows infrastructure equipments with specific attributes: SF-2911-RBR has a NAM module in it that can be accessed directly thru the troubleshooting panes that we have already seen 7206-Core routers are managed but they do not support medianet 10.0.4.2 and 10.0.2.1 routers are not managed (simulate an SP network over which we have no control)

This is to be compared with the same screen displayed for our EX90-P52video session. Here we clearly see that we have packet loss detected first by 3945-East-1. That does not mean its responsible for the issue, but that helps locate the area where the problem is. Further digging into the router itself can help state whether the problem is in the router or before it on the path. Clearly it is not at the ingress to the SP network.

6. Proactive troubleshooting
To assess the capability of your network to sustain the load of video traffic and ensure optimal user experience, you need to proactively observe the behavior of the network infrastructure equipments under load and for this you need to have video traffic flow through the network. IPSLA VO enables you to generate video traffic between switches or routers. This allows you to actually make sure that video quality will be satisfactory for the users, before you even deploy the video equipments.

To start the network assessment, go to Monitoring Proactive Troubleshooting:

Set the from device to one of your PODs access switches and the to device to the other switch. Choose the loopback as the target address and choose the video profile you want to try out. Then select the duration of the test.

Then click start.

The sequence of operations thru which the system is going is exactly the same as for a troubleshooting session.

We see that in the path trace, the access switches have replaced the video endpoints as they are handling the video flow. SW-PODx-W is sending the Telepresence flow to SW-PODx-E, which is responding to the initiator with statistics data. In addition to the jitter calculation made by IP SLA VO, statistical data are collected along the path using mediatrace. The jitter is calculated according to RFC 5481 IPDV model (Inter Packet Delay Variation).