You are on page 1of 212

Junos Troubleshooting in the NOC

12.b

Student Guide
Volume2

Worldwide Education Services

1133 Innovation Way


Sunnyvale, CA 94089
USA
408-745-2000
www.juniper.net

Course Number: EDU-JUN-JTNOC


This document is produced by Juniper Networks, Inc.
This document or any part thereof may not be reproduced or transmitted in any form under penalty of 1aw, without the prior written permission of Juniper Networks
Education Services.
Juniper Networks. the Juniper Networks logo. Junos. NetScreen, and ScreenOS are registered trademarks of Juniper Networks. Inc. in the United States and other
countries. AU other trademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners.

Junos Troubleshooting in the NOC Student Gulde, Revision 12.b


Copyright© 2014Juniper Networks. Inc. All rights reserved.
Printed in USA.
Revision History:
Revision 10.a-December 2010
Revision 11.a-June 2011
Revision 12.a-March 2013
Revision 12.b-January 2014
The information in this document is current as of the date listed above.
The information in this document has been carefully verified and is believed to be accurate for software Release 12.2R2.5. Juniper Networks assumes no
responsibilities for any inaccuracies that may appear in this document. In no event will Juniper Networks be liable for direct, indirect. special, exemplary,
incidental, or consequential damages resulting from any defect or omission in this document. even if advised of the possibility of such damages.

Juniper Networks reserves the right to change. modify, transfer. or otherwise revise this publication without notice.
YEAR 2000 NOTICE
Juniper Networks hardware and software products do not suffer from Year 2000 problems and hence are Year 2000 compliant. TheJunos operating system has
no known time-related limitations through the year 2038. However. the NTP application is known to have some difficulty in the year 2036.
SOFTWARE LICENSE
The terms and conditions for using Juniper Networks software are described in the software license provided with the software, or to the extent applicable, in an
agreement executed between you and Juniper Networks, or Juniper Networks agent. By usingJuniper Networks software, you indicate that you understand and
agree to be bound by its license terms and conditions. Generally speaking, the software license restricts the manner in which you are permitted to use the Juniper
Networks software. may contain prohibitions against certain uses, and may state conditions under which the license is automatically terminated. You should
consult the software license for further details.
Contents

Chapter 9: Staging and Acceptance Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-1


Initial Inspection and Power-on .................................................. 9-3
General System Checks .......................................•.........•...... 9-7
InterfaceTesting ............................................................. 9-15

Chapter 10: Troubleshooting Routing Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-1


Troubleshooting OSPF ......................................................... 10-3
Troubleshooting BGP ........................................................ 10-19
Troubleshooting Routing Loops and Route Oscillation............................. 10-45
Troubleshooting Routing Protocols Lab ......................................... 10-63

Chapter 11: High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-1


High Availability Overview ...................................................... 11-3
Graceful Routing Engine Switchover ............................................. 11-5
Graceful Restart ............................................................ 11-13
Nonstop Active Routing and Bridging........................................... 11-17
Unified In-Service Software Upgrade ........................................... 11-29

Chapter 12: Network Monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-1


SNMP ...................................................................... 12-3
RMON .................................................................... 12-30
Flow Monitoring ...................... _ ..................................... 12-35
Monitoring the Network Lab .................................................. 12-53

Chapter 13: JTAC Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-1


Opening a Support Case ....................................................... 13-3
Customer Support Tools ....................................................... 13-8
Transferring Files to JTAC .................................................... 13-18

Appendix A: Interface Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1


Interface Troubleshooting Chart .................................................. A-3
Troubleshooting Various Interface Types........................................... A-5

www.juniper.net Contents • iii


iv • Contents www.juniper.net
Course Overview

This three-day course is designed to provide introductory troubleshooting skills for engineers in a
network operations center (NOC) environment. Key topics within this course include
troubleshooting methodology, troubleshooting tools, hardware monitoring and troubleshooting,
interface monitoring and troubleshooting, troubleshooting the data plane and control plane on
devices running the Junos operating system, staging and acceptance methodology,
troubleshooting routing protocols, monitoring the network, and working with JTAC. This course is
based on Junos operating system Release 12.2R2.5.
Objectives
After successfully completing this course, you should be able to:
Reduce the time it takes to identify and isolate the root cause of an issue impacting
your network.
Gain familiarity with Junos products as they pertain to troubleshooting.
Become familiar with online resources valuable to Junos troubleshooting.
Gain familiarity with Junos tools used in troubleshooting.
Identify and isolate hardware issues.
Troubleshoot problems with the control plane.
Troubleshoot problems with interfaces and other data plane components.
Describe the staging and acceptance methodology.
Troubleshoot routing protocols.
Describe how to monitor your network with SNMP, RMON, JFlow, and port mirroring.
Become familiar with JTAC procedures.
Intended Audience
The course content is aimed at operators of devices running the Junos OS in a NOC environment.
These operators include network engineers, administrators, support personnel, and reseller
support personnel.
Course Level
Junos Troubleshooting in the NOC is an introductory-level course.

Prerequisites
Students should have basic networking knowledge and an understanding of the Open Systems
Interconnection (OSI) reference model and the TCP/IP protocol suite. Students should also attend
the Introduction to the Junos Operating System (IJOS) course and the Junos Routing Essentials
(JRE) course, or have equivalent experience prior to attending this class.

www.juniper.net Course Overview • v


Course Agenda

Day1
Chapter 1: Course Introduction
Chapter 2: Troubleshooting as a Process
Lab 1: The Troubleshooting Process
Chapter 3: Junos Product Families
Lab 2: Identifying Hardware Components
Chapter 4: Troubleshooting Toolkit
Lab 3: Monitoring Tools and Establishing a Baseline

Day2
Chapter 5: Hardware and Environmental Conditions
Lab 4: Monitoring Hardware and Environmental Conditions
Chapter 6: Control Plane
Lab 5: Control Plane Monitoring and Troubleshooting
Chapter 7: Data Plane: Interfaces
Lab 6: Monitoring and Troubleshooting Ethernet Interfaces
Chapter 8: Data Plane: Other Components
Lab 7: Isolate and Troubleshoot PFE Issues
Day3
Chapter 9: Staging and Acceptance Testing
Chapter 10: Troubleshooting Routing Protocols
Lab 8: Troubleshooting Routing Protocols
Chapter 11: High Availability
Chapter 12: Network Monitoring
Lab 9: Monitoring the Network
Chapter 13: JTAC Procedures
Appendix A: Interface Troubleshooting

vi • Course Agenda www.juniper.net


Document Conventions

CLI and GUI Text


Frequently throughout this course, we refer to text that appears in a command-line interface (CLI)
or a graphical user interface (GUI). To make the language of these documents easier to read, we
distinguish GUI and CLI text from chapter text according to the following table.

Style Description Usage Example

Franklin Gothic Normal text. Most of what you read in the Lab Guide
and Student Guide.

Courier New Console text:


commit complete
Screen captures
Noncommand-related Exiting configuration mode
syntax
GUI text elements:
Select File > Open, and then click
Menu names Configuration.confin the
Filename text box.
Text field entry

Input Text Versus Output Text


You will also frequently see cases where you must enter input text yourself. Often these instances
will be shown in the context of where you must enter them. We use bold style to distinguish text
that is input versus text that is simply displayed.

Style Description Usage Example

Normal CLI No distinguishing variant. Physical interface:fxpO,


Enabled
Normal GUI
View configuration history by clicking
Configuration > Histor�

CLI Input Text that you must enter. lab@San Jose> show route

GUI Input Select File > Save, and type


config. ini in the Filename field.

Defined and Undefined Syntax Variables


Finally, this course distinguishes between regular text and syntax variables, and it also
distinguishes between syntax variables where the value is already assigned (defined variables) and
syntax variables where you must assign the value (undefined variables). Note that these styles can
be combined with the input style as well.

Style Description Usage Example

CLl Variable Text where variable value is policy my-peers


already assigned.
GUI Variable
Click my-peers in the dialog.

CLI Undefined Text where the variable's value Type set policy po.licy-name.
is the user's discretion and text
ping 10.0.�
where the variable's value as
GUI Undefined shown in the lab guide might Select File > Save, and type
differ from the value the user £i.lename in the Filename field.
must input.

www.juniper.net Document Conventions • vii


Additional Information

Education Services Offerings


You can obtain information on the latest Education Services offerings, course dates, and class
locations from the World Wide Web by pointing your Web browser to:
http://www.juniper.net;training/education/.

About This Publication


The Junos Troubleshooting in the NOC Student Guide was developed and tested using software
Release 12.2R2.5. Previous and later versions of software might behave differently so you should
always consult the documentation and release notes for the version of code you are running before
reporting errors.
This document is written and maintained by the Juniper Networks Education Services development
team. Please send questions and suggestions for improvement to training@juniper.net.

Technical Publications
You can print technical manuals and release notes directly from the Internet in a variety of formats:
Go to http://www.juniper.net;techpubs/.
Locate the specific software or hardware release and title you need, and choose the
format in which you want to view or print the document.
Documentation sets and CDs are available through your local Juniper Networks sales office or
account representative.

Juniper Networks Support


For technical support, contact Juniper Networks at http://www.juniper.net;customers/support/, or
at 1-888-314-JTAC (within the United States) or 408-745-2121 (from outside the United States).

viii • Additional Information www.juniper.net


JUnlf2v�[
Junos Troubleshooting in the NOC

Chapter 9: Staging and Acceptance Testing


Junos Troubleshooting in the NOC

Objectives

• After successfully completing this content, you will be


able to:
• Perform the initial inspection and power-on of a Junos
device
• Perform general system checks recommended for a newly
deployed Junos device
• Determine the status of new interface connections by
performing loopback testing and monitoring

We Will Discuss:
Initial inspection and power-on of Junos devices;
General system checks for newly deployed Junos devices; and
Loopback testing of new interfaces.

Chapter 9-2 • Staging and Acceptance Testing www.juniper.net


Junos Troubleshooting in the NOC

Agenda: Staging and Acceptance Testing


7lnitial Inspection and Power-On
• General System Checks
• Interface Testing

Initial Inspection and Power-on


The slide lists the topics we will discuss. We discuss the highlighted topic first.

www.juniper.net Staging and Acceptance Testing • Chapter 9-3


Junos Troubleshooting in the NOC

Initial Installation
• Installation directions and safety guidelines for Junes
devices are provided through hardware
documentation at www.juniper.net/techpubs
• Choose hardware family and then the specific device
• Complete hardware guide available in PDF format
MX240 30 Universal Edge Router

Sen�s HarrJ',;1are- & Softv,rare Oocumentat1on Home


Supported Platforms
Overview Components Pli,nn\ng Safety Installation Maintenance Troubleshooting • r,1X240 Routers

r.,X240 Router Release t101es MX2-t0 F!ou!er Overview


Do;•.mtoads
Cutstanr.ting l::su�;; wH.n th� t.li2..!0 RNiter
• IJ.<240 Router Hamt.are Gwd:.

Errata ,v1th th� f.1�24.J Routt't D,xumentJhon
• r.1X.2JQ Router R.::-!ease
. Notes .l:.I
• 1,1X240 Router C!JiCt: St.1.ri °1;
• r.,x Series une Card Guide�

Initial Installation
Installation and safety guidelines are not in the scope of this course. However, the slide shows you how to access detailed
installation guides at the www.juniper.net;techpubs website.
There are also hardware installation computer-based training modules available at the
https://learningportal.juniper.netjjuniper/user_courses.aspx learning portal website. You can easily find the installation
trainings by performing a keyword search using the term installation.

Chapter 9-4 • Staging and Acceptance Testing www.juniper.net


Junos Troubleshooting in the NOC

What Is Staging?

• Staging a device is a procedure used to detect


hardware problems so they can be resolved before
having an impact on end users
• For example, parts damaged in transit
• Significant number of hardware failures occur within the first
48 hours of operation

What Is Staging?
As shown on the slide, staging is a process used to detect hardware issues prior to any user impacts. It is notable that most
hardware failures occur within the first 48 hours of a device's operation. Th.erefore, it is important to let the hardware
operate for at least two days before acceptance. We recommend that you perform the tests outlined in this chapter twice­
once after system power-on and again after two days of continuous operation.

www.juniper.net Staging and Acceptance Testing • Chapter 9-5


Junos Troubleshooting in the NOC

Physical Inspection and Power-On


• Physical inspection and power-on
• Physically check the chassis and system boards for any sign
of mechanical damage
• Check that all FRUs are correctly seated in the chassis
• Connect the power supplies and power them on one at a
time
• Unless otherwise noted in the hardware documentation

r�
Juniper
" .. ' to 1
.: C !!014Junlp;,Netwctl!s, ltlc.AII njJl!lsreseM><L � Worldwide Education Services WWWJU,Hpernel I 6
��d���.-t� - %1 __,
:i..

Physical Inspection and Power-on


Once the system has been installed in a suitable mounting location, and before power-on, you should physically inspect the
chassis, component boards, and other field-replaceable units (FRUs). Ensure the FRUs are seated properly and show no
signs of damage.
After physical inspection, proceed with powering up each power supply one-by-one. Please refer to the hardware
documentation because some devices require minimum amounts of time between each power supply power-on.

Chapter 9-6 • Staging and Acceptance Testing www.juniper.net


Junos Troubleshooting in the NOC

Agenda: Staging and Acceptance Testing


• Initial Inspection and Power-On
�General System Checks
• Interface Testing

General System Checks


The slide highlights the topic we discuss next.

www.juniper.net Staging and Acceptance Testing • Chapter 9- 7


Junos Troubleshooting in the NOC

Check for Alarms

• Check for alarms with the show chassis alarms


command
user@mx> show chassis alarms
1 alarm currently active
Alarm time Class Description
:013-0:-10 04:50:11 UTC Major fxpO: Link down

• An alarm will be present for the management interface if it is


not yet connected
• Alarms are configurable

Alarm Checking
Once the system has fully powered up and you are able to log into the device using either a console or management port,
check for any chassis alarms with the show chassis alarms command. Do not be alarmed if you notice the alarm
shown on the slide. It is common to see an alarm for the management interface since it may not be connected at initial
power-on.
Note that you can change the default behavior of alarms through configuration. For example, you can choose to display
alarms for any Ethernet interface for which a link is in the down state as shown below.
[edit chassis alarm]
user@mx# show
ethernet {
link-down red;

Chapter 9-8 • Staging and Acceptance Testing www.juniper.net


Junos Troubleshooting in the NOC

Collect the Serial Numbers

• Retrieve and save the output of show chassis


hardware for inventory and support purposes
• Example output from an MX80:
user@mx> show chassis hardware I no-more
Hardware inventory:
Item Version Part number Serial nw.uber Description
Chassis 04884 MX80
Midplane REV 06 711-031594 YK9006 MX80
PEM D Rev 03 740-028288 UG00881 AC Power Entry Module
Routing Engine BUILTIN BUILT IN Routing Engine
TFEB O BUILT IN BUILT IN Forwarding Engine Processor
QXM O REV 05 711-028408 YK6532 MPC QXM
FPC O BUILTIN BUILTIN MPC BUILTIN
MIC U BUILT IN BUILT IN 4x lOGE XFP
!?IC O BUILT IN BUILT IN 4x lOGE XFP
Xcvr 0 REV 01 740- 14289 C721XU04Y XFP-10G-SR
FPC 1 BUILTIN BUILT IN MPC BUILTIN
MIC 0 REV 22 750-028392 YJ7457 3D 20x 1GE(Llh�) SFP
PIC 0 BUILT IN BUILT IN 10x lGE(LJL�) SFP
Xcvr 0 REV 02 740-013111 11.121798 SFP-T
Xcvr 1 REV 02 740-013111 A116893 SFP-T

Collecting Serial Numbers


One of the first tasks you should complete after power-on is to collect the output of the show chassis hardware
command. Not only does this command provide an inventory of what the Junos OS believes to be installed in the chassis, but
it also provides a list of serial numbers for the chassis itself and all FRUs. (Note: Look for missing components.) You will need
the chassis serial number to open any case with the Juniper Networks Technical Assistance Center (JTAC) and you will need
the component serial number in case a replacement component is required.
This information should be stored off the device, whether it be in an external database or within an inventory control system.
Many network management platforms, such as the Junos Space platform have automated inventory discovery tools to assist
in this endeavor.

www.juniper.net Staging and Acceptance Testing • Chapter 9-9


Junes Troubleshooting in the NOC

Check Temperatures
• Sample output from an MX80:
user@mx> show chassis environment
Class Item Status Measurement
Temp ?EM 0 OK
PEM 1 Absent
RE 0 Intake OK 31 degrees c I 87 degrees F
RE 0 Front Exhaust OK 35 degrees c I 95 degrees F
RE 0 Rear Exhaust OK 34 degrees c I 93 degrees F
Rom:ing Engine OK 39 degrees c I 102 degrees F
Routing Engine CPU OK 50 degrees c I _22 degrees "'
TFEB 0 QX 0 TSen OK 36 degrees c I 96 degrees F
TFEB 0 QX 0 Chip OK 46 degrees c I 114 degrees F
TFEB 0 LU 0 TSen OK 36 degrees c I 96 degrees F
TFEB 0 LU 0 Chip OK 49 degrees c / 120 degrees F
TFEB 0 MQ 0 TSen OK 36 degrees c I 96 degrees F
TFEB 0 MQ 0 Chip OK 44 degrees c / 111 degrees F
TFEB 0 TBB PFE TSen OK 34 degrees c I 93 degrees F
TFEB 0 TBB PFE Chip OK ·"- degrees c I _Q7 degrees F
a�
TFEB 0 TFEB PCIE TSen OK 37 degrees c I 98 degrees F
TFEB 0 TFEB PCIE Chip OK 52 degrees c 125 degrees F
::ans Fan 1 OK Spinning at normal speed
Fan 2 OK Spinning at normal speed
Fan 3 OK Spinning at normal speed

Checking the Environment


If you have followed the installation and safety guidelines in the technical documentation, you should have an acceptable
temperature-controlled environment within which the Junes device can operate. To check the temperatures of the Junes
device components and the status of the cooling fans, issue the show chassis environment command.
The Status and Measurement columns displayed on the slide indicate the device is operating within expected guidelines.
Note that you can view the temperature alarm thresholds with the show chassis temperature-thresholds
command.
user@mx> show chassis temperature-thresholds
Fan speed Yellow alarm Red alarm Fire Shutdown
(degrees C) (degrees C) (degrees C) (degrees C)
Item Normal High Normal Bad fan Normal Bad fan Normal
Chassis default 48 54 65 55 75 75 100
Routing Engine 65 70 85 75 95 80 100

Chapter 9-10 • Staging and Acceptance Testing www.juniper.net


Junes Troubleshooting in the NOC

Check Voltages

• Command can vary by platform


• Sample output from an MX80=
user@mx> show chassis environment cb
CB O status:
State Online Master
Temperature 31 degrees c I 87 degrees F
Power 1
1.0 V l008 mV
1.0 V MQ 1076 mV
1.0 V LU 1086 mV
1.2 V 1214 mV
1.5 V 1524 mv
1. 8 V 1817 mv
2.5 V 2568 mv
3.3 V 3296 mV
5.0 V 5201 mV
5.0 v bias 5066 mV
12.0 V 12084 mv

• Anomalies will be flagged in the output

Checking the Voltages


The slide illustrates the show chassis environment cb command and output from an MX80 device. Note that the
command to check voltages varies by platform and is usually dependent upon the type of system switching board present in
the Junes device. Use the show chassis environment ? command to determine the possible completions for your
Junes device. Further details can be found in the hardware documentation specific to your device, including expected power
levels. However, major anomalies will be reported in the output.

www.juniper.net Staging and Acceptance Testing • Chapter 9-11


Junos Troubleshooting in the NOC

Checking Individual Components


• Components will vary by chassis
user@mx> show chassis?
Possible completions:
alarms Show alarm status
craft-inter::ace Show craft interface status
environment Show component status and temperature, cooling system speeds
fan Sho-1v fan and fan tray information
firmware Show firnn...rare and operating system version for components
fpc Show Flexible PIC Concentrator status

pie Show Physical Interface Card state, type, and uptime

tfeb Show Compact Forwarding Engine Board status

user@mx> show chassis fpc 1 detail


Slot 1 information:
!State Online
Temperature 37 (C)
Total CPU D?-AM 1024 MB
Total SRl'.M 403 MB
Total SDRk� 1316 MB
Start time 2013-02-05 20:51:16 UTC
!UptJ.me 5 days, 8 hours, 16 minutes, 34 seconds!

Checking Individual Components


As part of the staging process you should check the status of individual components installed in your Junos device. Use
show chassis commands to determine the current state of each component, paying special attention to the State field.
To ensure none of the components suffer outages during the staging period, be sure to check the uptime after 48 hours.

Chapter 9-12 • Staging and Acceptance Testing www.juniper.net


Junes Troubleshooting in the NOC

Checking the Routing Engine


(master)
user@mx> show chassis routing-engine
Routing En·Jine status:
Slot 0:
current state Master
Election priority Mast.er (default)
Temperature 42 degrees c I 107 degrees r·
CPU temperature 47 degrees c I 116 degrees F
DRl'.M 3584 MB
Memory utilization 19 percent
CPU utilization:

Routing Engine status:


Slot 1:
Current state Backup
Election priority Backup (default)
Temperature 40 degrees c I 104 degrees F
CPU temperature L�
.L. degrees c I 107 degrees F

Checking Routing Engine Status


Use the show chassis routing engine command to display the status of a system's routing engines. In systems with
redundant routing engines, be sure that one routing engine is listed as Master and one routing engine is listed as Backup
as shown on the slide.

www.juniper.net Staging and Acceptance Testing • Chapter 9-13


Junos Troubleshooting in the NOC

Checking Storage Media


• Check storage media
•Ensure/var, /config, and/ (root) are properly mounted
user@mx> show syste.'I\ storage
Filesystem size Used Avail capacity Mounted on
/dev/daOsla 885� 125M 688M 15% I
devfs 1.OK 1. OK OB 100% /dev
/dev/mdO 42M 42M OB 100% /packages/mnt/jbase
/dev/mdl 184M 184M OB 100% /packages/mnt/jkernel-
ppc-12.2R2.5
/dev/md2 33M 33 M OB 100% /packages/mnt/jpfe-
MX80-12. 2R2 .5
/dev/md3 9.lM 9. lM OB 100% /packages/mnt/jdocs-
12.2!<2.5
/dev/md4 55M SSM OB 100% /packages/mnt/jroute-
ppc-12.2R2.5
/dev/md5 12M 12M OB 100% /packages/mnt/jcrypto-
ppc-·2.2R2.5
/dev/md6 2 . 8G 8.0K 2.6G 0% /wnp
/dev/md7 2. BG 1. 6M 2.CG 0% /mfs
/dev/daOsle 98M 98K 90M 0% /config
procfs 4. OK 4.0K OB 100% /proc
/dev/dalslf 2. 8G l.SG 1.lG 58% /var

Checking the Storage Media


Use the show system storage command to display the mounted directories and file systems. The output will vary
depending on your Junos device. For example, you might have both a hard disk (typically adO or daO) and flash drive
(typically adl or dal) or the output might list only a hard disk drive. For more information about media naming browse to the
http://kb.juniper.net;lnfoCenter/index?page =content&id= KB11224 Knowledge Base note.
Verify the I. or root directory, /var, and I con fig directories are mounted properly.

Chapter 9-14 • Staging and Acceptance Testing www.juniper.net


Junos Troubleshooting in the NOC

Agenda: Staging and Acceptance Testing

• Initial Inspection and Power-On


• General System Checks
7lnterface Testing

Interface Testing
The slide highlights the topic we discuss next.

www.juniper.net Staging and Acceptance Testing • Chapter 9-15


Junos Troubleshooting in the NOC

Testing Optical Interfaces (1. of 4)

• It is recommended to test all optical interfaces using


simple loopback testing
• Physical loopback testing is preferred
• Tests the optics as well as the interface's software operation
• Requires loopback cable with a single fiber looped from the
transmit port to the receive port
Rx Tx

• www.juniper_netjtechpubs/en_US/junos12.3/information­
products/topic-collections/nog-interfaces/index.html

Loopback Testing on Optical Interfaces: Part 1


To ensure all your device's interfaces are operational, weren't damaged in transit, and are not dead-on-arrival {DOA), it is
recommended that you perform interface testing on all your device's installed interfaces.
In this section we will focus on optical Ethernet loopback testing. Like other optical interfaces, it is recommended you use a
physical loop to test the interfaces. While software loopback testing is available, physical loopback testing is more
comprehensive by also testing the interface's optics. An optical loopback cable is merely a single fiber which runs from the
transmit port to the receive port on the same interface.

· Chapter 9-16 • Staging and Acceptance Testing www.juniper.net


Junos Troubleshooting in the NOC

Testing Optical Interfaces (2 of 4)

• Ethernet optical interface loopback test example


1. Configure the interface with a user-defined MAC address
and use this same MAC address for responding to ARP
requests destined to the remote IP address

[edit interfaces xe-0/0/0]


user@mx# show
mac 00:00:00:00:00:01;
unit O {
family inet
address 10.0.0.1/30
arp 10.0.0.2 mac 00:00:00:00:00:01;

Loopback Testing on Optical Interfaces: Part 2


To perform loopback testing on an Ethernet interface, you will send Internet Control Message Protocol (ICMP) ping requests
out of the interface. To make the test meaningful, you must statically associate a MAC address with the interface under test.
You must also configure a local IP address and a simulated remote IP address with a static Address Resolution Protocol
(ARP) entry. You associate the simulated remote IP address with the same static MAC address as configured for the local IP
address. The simulated IP address is required because an Ethernet interface will not ping its own address. It is expected that
the ping test will fail, however, you will use interface counters to confirm proper operation.

www.juniper.net Staging and Acceptance Testing • Chapter 9-17


Junos Troubleshooting in the NOC

Testing Optical Interfaces (3 of 4)

• Example continued
2. Clear the interface statistics for the interface under test
user@mx> clear interfaces statistics xe-0/0/0

3. Run a ping test to the remote IP address, bypass routing,


and specify a TTL
user@mx> ping interface xe-0/0/0 10.0.0.2 bypass-routing count 10 ttl 255
PING 10.0.0.2 (10.0.0.2): 56 data bytes
36 bytes from 10.0.0.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 ed42 0 0000 01 01 b864 10.0.0.1 10.0.0.2

36 by.:es from 10.0.0.1: Time to live exceeded


Vr HL TOS Len ID Flg off TTL Pro cks src Dst
4 5 00 0054 edde O 0000 01 01 b7c8 10.0.0.1 10.0.0.2

10.0.0.2 ping statistics


10 packets transmitted, 0 packets received, 100% packet loss

Loopback Testing on Optical Interfaces: Part 3


To create an accurate baseline, be sure to clear the interface's statistics prior to running the ping test with the clear
interfaces statistics command as shown on the slide.
Next, perform a ping test specifying the interface under test as the outgoing interface and the simulated remote IP address
as the destination.Use the bypass-routing option so that the Junos OS will not perform a routing lookup on the
destination, ensuring the proper interface is used.For this test, we also specify a time to live (TTL) of 255 hops.

Chapter 9-18 • Staging and Acceptance Testing www.juniper.net


Junos Troubleshooting in the NOC

Testing Optical Interfaces (4 of 4)

• Example continued
4_ Verify the appropriate number of packets are sent and
received on the tested interface
• Based on 10x the TIL

user@mx> show interfaces extensive xe-0/0/0 I find "Traffic


statistics"
Traffic statistics:
Input bytes 214200 O bps
Output bytes 214400 O bps
Input packets: �550 O pps
Output packets: 2550 O pps

Loopback Testing on Optical Interfaces: Part 4


To verify the results of your test, view the output of the show interfaces extensive interface-name command.
The Input packets counter and the output packets counter should list counts of 2,550 packets. The counter value
is the result of sending 10 ICMP echo requests with a TIL of 255 for each packet.
As shown on the slide, both the input and output functions of the interface appear to be operating correctly.

www.juniper.net Staging and Acceptance Testing • Chapter 9-19


Junos Troubleshooting in the NOC

Summary
• In this content, we:
• Performed the initial inspection and power-on of a Junos
device
• Performed general system checks recommended for a newly
deployed Junos device
• Determined the status of new interface connections by
performing loopback testing and monitoring

We Discussed:
Performing initial inspection and power-on of a Junos device;
How to perform general system checks on a newly deployed Junos device; and
How to perform loopback testing on an optical interface.

Chapter 9-20 • Staging and Acceptance Testing www.juniper.net


Junes Troubleshooting in the NOC

Review Questions

1. What is staging?
2. What are three critical directories to check with the
show system storage command?
3. When doing loopback testing on an Ethernet
interface, why must you configure a static ARP
entry?

Review Questions
1.

2.

3.

www.juniper.net Staging and Acceptance Testing • Chapter 9-21


Junos Troubleshooting in the NOC
Answers to Review Questions
1.
Staging is a process used to detect hardware issues prior to any user impacts.

2.
Among orhcr checks, you should verify the I (root), I config, and /var directories arc present and mounted in the output of the
show sy ste1n storage co011nand.

3.
You use a static ARP entry to associate the simulated remote destination IP address with rhe manually-configured MAC address on the
interface. This is required because an Ethernet interface will not send packets to itself and an ARP reply is required before the interface
will send the ICMP echo requests required for rhe test.

Chapter 9-22 • Staging and Acceptance Testing www.juniper.net


JUnlf2v�f
Junos Troubleshooting in the NOC

Chapter 10: Troubleshooting Routing Protocols


Junos Troubleshooting in the NOC

Objectives

• After successfully completing this content, you will be


able to:
• Explain how to troubleshoot OSPF
• Describe how to troubleshoot BGP
• Demonstrate how to troubleshoot routing loops
• Discuss how to troubleshoot route oscillation

We Will Discuss:
How to troubleshoot OSPF;
How to troubleshoot BGP;
How to troubleshoot routing loops; and
How to troubleshoot route oscillation.

Chapter 10-2 • Troubleshooting Routing Protocols www.juniper.net


Junes Troubleshooting in the NOC

Agenda: Troubleshooting Routing Protocols

�Troubleshooting OSPF
• Troubleshooting BGP
• Troubleshooting Routing Loops and Route Oscillation

Troubleshooting OSPF
The slide lists the topics we will discuss. We discuss the highlighted topic first.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-3


Junos Troubleshooting in the NOC

OSPF Protocol Overview


• An overview of OSPF
• OSPF is a link-state IGP that routes packets within a single
AS
• OSPF reliably floods LSAs to distribute link-state
information once an adjacency is formed
• Each router uses these LSAs to create a complete
database for the network
• OSPF uses the SPF algorithm within the database to
calculate the best route to every node in the network
• OSPF is defined in:
• RFC 2328, OSPF Version 2
• RFC 1587. The OSPF NSSA Option

OSPF Protocol Overview


OSPF is a link-state routing protocol designed for use within an autonomous system (AS). It is considered an interior gateway
protocol (IGP). Link-state protocols allow for faster reconvergence, support larger internetworks, and are less susceptible to bad
routing information than distance vector protocols.
Routers running OSPF send out information about their network links and the state of those links to other routers in the AS. This
information is transmitted reliably to all other routers in the AS through link-state advertisements (LSAs). The other routers
receive this information and store it locally. This total set of information now contains all possible links in the network.
In addition to flooding LSAs and discovering neighbors, a third major task of the link-state routing protocol is establishing the
link-state database (LSDB). The link-state (or topological) database stores the LSAs as a series of records. The important
information for the shortest-path determination process is the advertising router's ID, its attached networks and neighboring
routers, and the cost associated with those networks or neighbors.
OSPF uses the shortest-path-first (SPF) algorithm or Dijkstra algorithm to calculate all at once the shortest paths to all
destinations. It does this calculation by calculating a tree of shortest paths incrementally and picking the best candidate from
that tree.
For more information about OSPF, please refer to RFC 2328 which defines OSPF version 2, and RFC 1587 which defines the
OSPF NSSA option.

Chapter 10-4 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

OSPF Areas

• Areas
• Single AS can be divided into smaller groups called areas
• Reduces the LSDB because LSA flooding is now constrained
to the area
• Routers maintain a separate LSDB on a per-area basis
• Each LSDB within an area still must be identical on all routers
• Special OSPF area called the backbone area
• Backbone area (0.0.0.0) distributes routing information
between areas
• All other OSPF areas must connect to the backbone area
• All user traffic from one area to another must traverse the
backbone

OSPFAreas
Using areas achieves the OSPF hierarchy. As mentioned previously, areas reduce the size of the LSDB on an individual router.
Now, each router maintains a separate LSDB for each area to which it is connected.

Backbone Area
To ensure correct routing knowledge and connectivity, OSPF maintains a special area called the backbone area. This backbone
area is designated as area 0. All other OSPF areas must connect themselves to the backbone for connectivity. All data traffic
between OSPF areas must transit the backbone.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-5


Junos Troubleshooting in the NOC

OSPF Router Terminology


• Important OSPF router terminology
• Internal router has all OSPF links in the same area
• Within area o. also called a backbone router
• Backbone router
• Any router with a link to area O
• ABRs
• Routers that belong to more than one area are called area border
routers
• Connect OSPF areas to the backbone area O
• ASBRs
• Routers that injects routing information from outside the OSPF
domain are called AS boundary routers

OSPF Router Terminology


An OSPF router with all its links within an area is known as an internal router. If that router is located within the backbone area
(0.0.0.0), it is also known as a backbone router.
Any OSPF router with a link to Area O (the backbone) is considered to be a backbone router. This router can also be an internal
or area border router, depending on whether it has links to other, non-backbone areas.
An OSPF router with links in two areas is called an area border router (ABR). The ABR is responsible for connecting OSPF areas
to the backbone. It transmits network information between the backbone and the its attached areas.
An OSPF router that injects routing information from outside the OSPF AS is known as an autonomous system boundary
router (ASBR). Typically, an ASBR is located in the backbone, but the OSPF specification allows an ASBR in other areas as
welt.

Chapter 10-6 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

OSPF Neighbors Versus Adjacencies


user@router> show ospf neighbor extensive
Address Intf State ID Pri Dead
172.16.30.254 ge-0/0/0.0 Full 10.250.240.8 128 30
area 0.0.0.5, opt Ox42, DR 172.16.30.254, BDR 172.16.30.253
Up 00:10:50, adjacent 00:10:50
Topology default (ID 0) -> Bidirectional

172.16.30.253 ge-0/0/0.0 Full 10.'.:50.240.35 128 30


area 0.0.0.5, opt Ox42, DR 17'.:.16.30.254, BDR 172.16.30.'.:53
Up 00:10:50, adjacent 00:10:52
Topology default (ID 0) -> Bidirectional

172.16.30.252 ge-0/0/0.0 2Way 10.'.:50.240.3'.: 64 38


area 0.0.0.5, opt Ox42, DR 172.16.30.254, BDR 172.16.30.'.:53
Up 00:08:10
Topology default (ID 0) -> Bidirectional

OSPF Neighbor Relationship


As soon as an OSPF router sees a hello packet on an interface, it starts to retain knowledge of that neighbor. You can display
this information with the operational command-line interface (CU) command show ospf neighbor.
On the slide, this router has three neighbors on the ge-0/0/0.0 interface. Two of the three routers are the designated router (DR)
and the backup designated router (BDR); full adjacencies exist with them. Each of the hello packets received from all three
routers lists their addresses.
The router that is in a two-way state is a neighbor on the link, but it is not the DR or BDR. This router reaches the two-way state
because the DR and BDR can see its hello packets, and this router's own router ID (RID) is located in the received hello. For
broadcast media, it is acceptable to have some neighbors in the two-way state.
The address column is the interface IP address of the neighboring router. The ID column is the RID of the neighboring router.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10- 7


Junes Troubleshooting in the NOC

Displaying OSPF Route Information


• The show ospf route command display routes
learned from and advertised to OSPF
• Includes routes for interfaces running OSPF
user@router> show ospf route ?
<[£nte=J> Execute this command
<destination> IP address and optional prefix length of destination
abr Display routes to area border routers
asbr Display routes to AS border routers
brief Display brief output (default)
detail Display detailed output
extensive Display extensive output
extern Display external routes
instance Name of OSPF instance
inter Display inter-area routes
intra Display intra-area routes
logical-system Name of logical system, or 'all'
network Display routes to networks
no-backup-coverage Display routes with no backup coverage
router Display routes to 211 routers
�apology Name of topology
I Pipe through a command
user@router> show ospf route detail
Prefix Route/Path/NextHop Type Metric Next hop i/f NH addr/Label
192.168.0.1/32 Intra Router IP 1 so-0/1/2.0
area 0.0.0.0, options OxOxO, origin 192.168.0.1

,-,-�---""""
1 ., ' �
:'c:w�; Jun1pe,���.ridttS�
��-11.:i-.. ""'�- .c!&� t -- -
JLJnf�
-
WorldwideEducationServices
A
www,un,pernet Is

Displaying OSPF Route Information


The show ospf route command displays the routes in the unicast routing table, inet.0, that were installed by OSPF. The use
of additional keywords allows you to display only OSPF routes learned by specific LSA types. The output fields of the show ospf
route command are the following:
Prefix: Displays the destination of the route.
Route/Path Type: Displays how the route was learned:
ABR: Route to area border router;
ASBR: Route to AS border router;
Ext: External router;
Inter: lnterarea route;
Intra: Intra-area route; or
Network: Network router.
Metric: Displays the route's metric value.
Next hop i/f: Displays the interface through which the route's next hop is reachable.
Next hop addr: Displays the address of the next hop.
Continued on the next page.

Chapter 10-8 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Displaying OSPF Route Information (cont.)


area: (detail output only) Displays the area ID of the route.
options: (detail output only) Displays the option bits from the LSA.
origin: (detail output only) Displays the router from which the route was learned.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-9


Junes Troubleshooting in the NOC

Displaying OSPF Interface Parameters

• Use show ospf interface to display the OSPF


parameters associated with an interface
user@router> show ospf interface
Interface State Area DR ID BDR ID Nbrs
at-0/1/0.100 Pt To Pt o.o.o.o o.o.o.o o.o.o.o 0
so-0/2/0.0 Pt To Pt o.o.o.o 0.0.0.0 0.0.0.0 0
ge-0/0/1.0 BDR o.o.o.o 2.2.2.2 1.1.1. l 1
ge-0/0/4.0 DP.other 0.0.0.0 0.0.0.0 0.0.0.0 0
loO.O Drother 0.0.0.0 0.0.0.0 0.0.0.0 0

Displaying OSPF Interface Parameters


The show ospf interface command displays information relating to the interfaces on which OSPF is configured to run. The
output fields are the following:
Interface: Displays the name of the interface running OSPF.
State: Displays the state of the interface. It can be BDR, Down, DR, DRother, Loop, PtToPt, or Waiting.
Area: Displays the number of the area in which the interface is located.
DR ID: Displays the address of the area's DR.
BDR ID: Displays the BDR for a particular subnet.
Nbrs: Displays the number of neighbors on this interface.
Type (detail and extensive output only): Displays the type of interface. It can be LAN, non broadcast multiaccess
(NBMA), point to multipoint (P2MP), point to point (P2P), or Virtual.
Address (detail and extensive output only): Displays the IP address of the neighbor.
Mask (detail and extensive output only): Displays the mask of the interface.
MTU (detail and extensive output only): Displays the interface's maximum transmission unit (MTU).
Continued on the next page.

Chapter 10-10 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Displaying OSPF Interface Parameters (contd.)


Cost (detail and extensive output only): Displays the interface's cost (metric).
DR addr (detail and extensive output only): Displays the address of the DR.
BDR addr: Displays the address of the BDR.
Adj count (detail and extensive output only): Displays the number of adjacent neighbors.
Dead (detail and extensive output only): Displays the configured value for the dead timer.
Hello (detail and extensive output only): Displays the configured value for the hello timer.
ReXmi t (detail and extensive output only): Displays the configured value for the retransmit timer.
Auth type: (detail and extensive output only): Displays the configure authentication type.
OSPF area type (detail and extensive output only): Displays the type of OSPF area, which can be Stub, Not Stub, or
not-so-stubby area (NSSA).

www .juniper.net Troubleshooting Routing Protocols • Chapter 10-11


Junos Troubleshooting in the NOC

Displaying Adjacency Information

• Use show ospf neighbor command to display


adjacency information
user@router> show ospf neighbor
Address Interface state ID Pri Dead
192.168.254.225 ge-1/2/0.0 2Way 10.250.240.32 1�0
Lu 36
192.168.254.230 ge-1/2/0.0 Full 10 .250.240.8 128 38
192.168.254.229 ge-1/2/0.0 Full 10.250.240.35 128 33
10.1.1.129 ge-2/0/0.0 Full 10.250.240.12 128 23
10.1.1.131 ge-2/0/0.0 Full 10.250.240.11 128 24
10.1.2.1 ge-2/1/0.0 Full 10.250.240.9 128 32
10.1.2.81 ge-2/1/0.0 Full 10.250.240.10 128 33

Displaying Adjacency Information


The show ospf neighbor command displays adjacency status for OSPF. The output fields include the following:
Address: Displays the address of the neighbor.
Interface: Displays the interface through which the neighbor is reachable.
State: Displays the state of the neighbor, which can be Attempt, Down, Exchange, ExStart, Full, lnit, Loading, or
2Way.
ID: Displays the RID of the neighbor.
Pri: Displays the priority of the neighbor to become the DR.
Dead: Displays the number of seconds until the neighbor becomes unreachable.
area (detail and extensive output only): Displays the area in which the neighbor is located.
opt (detail and extensive output only): Displays the option bits from the neighbor.
DR (detail and extensive output only): Displays the address of the DR.
BDR (detail and extensive output only): Displays the address of the BDR.
Up (detail and extensive output only): Displays the length of time since the neighbor came up.
adj acent (detail and extensive output only): Displays the length of time since the adjacency with the neighbor
was established.

Chapter 10-12 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Clearing Adjacencies
• Clearing adjacencies:
• Use the clear ospf neighbor command to clear
OSPF adjacencies:
user@router> clear ospf neighbor 192.168.254.225

Clearing Adjacencies
Use the clear ospf neighbor command to clear OSPF adjacencies. Note that in most cases the adjacency should reform
immediately.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-13


Junos Troubleshooting in the NOC

Troubleshooting OSPF Adjacencies

• OSPF Adjacencies
• No neighbor
• Physical and data-link-layer connectivity
• Mismatched IP subnet, subnet mask (on multi-access links). area
number. area type. authentication. hello or dead interval, or
network type
• Stuck in two-way state
• Normal for DR-other neighbors
• Stuck in exchange start
• Mismatched IP MTU

Troubleshooting OSPF Adjacencies


When attempting to find the reason why an OSPF adjacency does not come up to the full state, look for some of the items listed
in the slide. Note that an adjacency can only be established on a multiaccess interface-that is, Ethernet-when IP parameters
are matched. Matching IP parameters is not needed on point-to-point interface as the protocol is designed to support
unnumbered point-to-point links.

Chapter 10-14 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Displaying Database Entries

• Use the show ospf database command to


display entries in the LSDB
user@router> show ospf database

OSPF database, Area 0.0.0.0


Type ID Adv Rtr seq Age Opt Cksurn Len
Router *1.1.1.1 1.1.1.1 OxBOOOOOlb 278 Ox22 Ox9b32 60
Router 2.2.2.2 2.2.2.2 Ox80000013 442 Ox22 Ox76ca 84
Router 3.3.3.3 3.3.3.3 Ox80000016 2236 Ox22 OxeeaO 60
Router 4.4.4.4 4.4.4.4 OxBOOOOOla 355 Ox22 Ox5dlf 60
Network 10.1.1.2 2.2.2.2 Ox80000004 2691 Ox22 Ox26ee 32
Network 10.1.5.2 3.3.3.3 Ox80000008 2832 Ox22 Ox28d8 32
Network 10.1.6.1 3.3.3.3 Ox8000000e 440 Ox22 Ox7f73 32
Network 10.1.7.2 4.4.4.4 Ox80000008 2352 Ox22 Ox16e0 32
OSPF AS SCOPE link state database
Type ID Adv Rtr seq Age Opt Cksum Len
Extern 10.1.4.0 3.3.3.3 Ox8000000e 1640 Ox22 Oxdlb8 36
Extern 10.10.10.0 3.3.3.3 Ox8000000e 1039 Ox22 Ox2358 36
Extern *10.11.11.0 1.1.1.1 Ox8000023e 2143 Ox22 Oxcf82 36
Extern *10.12.12.0 1.1.1.1 Ox8000023e 1211 Ox22 Oxca82 36

Displaying OSPF Database Entries


The show ospf database command displays entries in the LSDB. The display is organized by areas and LSA types. The
command options include the following:
brief (optional): Displays a brief listing of all entries in the OSPF link-state database. This is the default setting.
detail. (optional): Displays detailed information about the entries in the OSPF link-state database.
extensive (optional): Displays extremely detailed information about the entries in the OSPF lin k s
- tate database.
LSA filters (optional): Displays one or more of the following LSA filters. If you specify more than one filter, only LSAs
that match all the filters are displayed. For example, the command show ospf database detail. router
l.sa-id l.O. O. O. 1 displays all router LSAs in all areas that have an LSA identifier of 10.0.0.1.
advertising-router address: Displays the LSAs advertised by a particular router.
area area-id: Displays the LSAs in a particular area.
l.sa-id :Lsa-id (optional): Displays the LSA with the specified LSA identifier.
LSA t ype: Displays specific types of LSAs. You can specify asbrsummary, extern, netsummary,
network, nssa, or router.
summary (optional): Displays summary information about the OSPF link-state database.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-15


Junos Troubleshooting in the NOC

Clearing Database Entries


• Use the clear ospf database command to clear
the LSDB
• Normally existing LSAs are simply reflooded over existing
adjacencies
• The purge option forces a refresh of all LSAs
user@router> clear ospf database ?
Possible completions:
<[Enter]> Execute this coro_�and
advertising-router Router ID of adver�ising router
area OSPF area ID
asbrsum."!lary Summary AS boundary router link-state advertisements
external External link-state advertisements
ins'tance Name of OSPF instance
link-local Link local link-state advertise..�ents
logical-syste:n Name of logical system, or 'all T
lsa-id Link-state advertisement ID
netsummary Summary network link-stat.e advertisements
netw·ork Network link-state advertisements
nssa Not-so-stubby area link-state advertisements
opaque-area Opaque area-scope link-state advertisements
purge Purge database entries instead of deleting them
router Router link-state advertisements
Pipe through a command

Clearing Database Entries


The c1ear ospf database command clear entries from the LSDB. After the command is entered, the router begins the
database synchronization process with its neighboring routers such that, in most cases, the database returns to its prior state.
The c1ear ospf database command supports an optional purge option. By including the purge option you force the
local router to set all LSAs in its database to maximum age. These LSAs are then reflooded according to the OSPF specification,
which states that a router must flood any LSA that it has set to maximum age, regardless of whether the LSA was generated by
the local router. All routers receive the newly flooded maximum age LSAs; the router that originated a given LSA is forced to
refresh that LSA when it receives a copy of that LSA with an indication that it has reached the maximum age.
Albeit somewhat disruptive, this procedure tends to eliminate stale or bogus database entries without having to
wait for the normal aging out process, which can take as long as 3600 seconds (one hour).

Chapter 10-16 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

OSPF Tracing
• Trace OSPF to gain insight into what the protocol is
doing
•Atypical tracing configuration:
[edit protocols ospf]
user@router# show
traceoptions {
file ospf-trace;
flag error detail;
flag hello detail;
flag lsa-update detail;

• Monitor the resulting log file using the monitor start


1.og-fil.e-name command or the show log
1.og-fil.e-name command

IGPTracing
To perform debugging functions on the OSPF routing process, use the Junos OS traceoptions function. The trace output (debug
information) is directed to the named log file, which is stored in the
/var/log directory on the router's storage space. You can view the log file using themonitor start or show l.og operational
mode commands. In addition to specifying the trace file, you must also tell the router what information you want to trace by
specifying one or more fl.ag keywords.
While you can only direct tracing to a single file, you can trace many options by using the fl.ag keyword multiple times. In
addition, you can add granularity by using the detail., receive, and send flag modifiers.
Continued on the next page.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-17


Junos Troubleshooting in the NOC
IGP Tracing (contd.)
Available tracing flags for OSPF include the following:

all Trace everything


database-description Trace database description packets
error Trace errored packets
event Trace OSPF state machine events
flooding Trace LSA flooding
general Trace general events
hello Trace hello packets
ldp-synchronization Trace synchronization between OSPF and LDP
lsa-ack Trace LSA acknowledgment packets
lsa-analysis Trace LSA analysis
lsa-request Trace LSA request packets
lsa-update Trace LSA update packets
normal Trace normal events
nsr-synchronization Trace NSR synchronization events
on-demand Trace demand circuit extensions
packet-dump Dump the contents of selected packet types
packets Trace all OSPF packets
policy Trace policy processing
restart-signaling Trace restart signaling
route Trace routing information
spf Trace SPF calculations
state Trace state transitions
task Trace routing protocol task processing
timer Trace routing protocol timer processing

Chapter 10-18 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Agenda: Troubleshooting Routing Protocols


• Troubleshooting OSPF
"'?Troubleshooting BGP
• Troubleshooting Routing Loops and Route Oscillation

Troubleshooting BGP
The slide highlights the topic we discuss next.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-19


Junos Troubleshooting in the NOC

What Is BGP?

• BGP defined
• Is an interdomain routing protocol that communicates
prefix reachability
• Is a path-vector protocol
• Views the Internet as a collection of autonomous systems
• Supports CIDR
• Exchanges routing information between peers
• Is defined in RFC 1771

What Is BGP?
BGP is an inter-AS routing protocol and is sometimes called a path-vector routing protocol because it uses an autonomous
system path, used as a vector, to prevent interdomain routing loops. The term path vector, in relation to BGP, means that BGP
routing information includes a series of AS numbers , indicating the path that a route takes through the network.
BGP exchanges routing information among autonomous systems or domains. An AS is a set of routers that operate under the
same administration. BGP routing information includes the complete route to each destination. BGP uses the routing
information to maintain a database of network layer reachability information (NLRI), which it exchanges with other BGP systems.
BGP uses the NLRI to construct a graph of AS connectivity, thus allowing BGP to remove routing loops and enforce policy
decisions at the AS level.

BGP is a classless routing protocol, which supports prefix routing, regardless of the class definitions of 1Pv4 addresses. BGP
routers exchange routing information between peers. The peers must be connected directly for inter-AS BGP routing (unless
certain configuration changes are completed). The peers are Transmission Control Protocol (TCP) peers, which is addressed
later in this chapter.
For more detail information concerning BGP, please refer to RFC 1771.

Chapter 10-20 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

BGP Fundamentals
• Fundamentals of BGP
• Each BGP update contains one path advertisement and
attributes
• Many prefixes can share the same path
• Routes consist of destination prefixes with an AS path and
other BGP-specific attributes
• BGP compares the AS path and other attributes to choose
the best path
• BGP withdraws unreachable routes

BGP Fundamentals
Routes in BGP consist of destination networks and attributes associated with those routes. Each BGP update contains one path
advertisement. However, many destinations can share the same path.
Once a connection is functioning, BGP sends routes. BGP routes consist of destination prefixes, each associated with BGP
attributes. Some of the complexities of BGP are the variety of these attributes, the order of their execution, and various rules
that can be applied to the attributes. BGP can associate one or more attributes with a route advertisement. Attributes carry
descriptive information about the route and are used in choosing the best path to a destination.
BGP attributes describe the following, as well as other items:
The next hop for a packet sent to a particular destination;
Various numeric-type attributes;
The path through autonomous systems that a routing announcement has traversed to arrive at the destination
where it is now; and
The method of generation for the prefix, or which protocol originated the route.
When two devices communicate through BGP, the receiving device assumes that a route remains active (should the BGP next
hop be accessible) until the originator explicitly withdraws it or until the session is terminated.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-21


Junos Troubleshooting in the NOC

BGP Neighbor States

• TCP connectivity:
• Idle
• Connect
• Active
• BGP connectivity:
• OpenSent
• OpenConfirm
• Established

TCP Connectivity
To exchange traffic using BGP, we first must establish a TCP connection. The following list shows the possible states for these
sessions:
Idle: This is the initial neighbor state. All BGP connections are refused. BGP waits for a start event, usually caused
by configuring a BGP session or resetting an existing session. The router also might try to initiate a start event.
Connect: In this state, BGP waits for a TCP connection to be completed.
Active: In this state, a TCP timeout has occurred, and the TCP session is not established yet. The ConnectRetry
timer has restarted, and the system continues to listen for completion of TCP session. This state is bypassed when
a TCP session is established before the ConnectRetry expires.

BGP Connectivity
After establishing a TCP session, we can now attempt to create communication at the BGP level. The following list shows the
states for BGP connectivity:
OpenSent: The TCP connection is established and an open message has been sent to the remote peer. BGP waits
to receive an open message from the peer.
OpenConfirm: An open message has been received from the remote peer, and BGP now waits for either a keepalive
message or a notification message. If a keepalive is received, the state machine transitions to established. If a
notification message is received, the state machine transitions to idle.
Established: The BGP peers are fully adjacent and can exchange keepalive, update, and notification messages.

Chapter 10-22 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

BGP Peering
• BGP sessions are established between peers
• BGP speakers
• Two types of peering sessions:
• EBGP (external) peers with different autonomous systems
• IBGP (internal) peers within the same AS
• IGP connects BGP speakers within the AS
• IGP advertises internal routes

BGP Peers
BGP is a protocol in which routing information exchanges occur between exactly two nodes, called peers. These peers can be
connected either directly or remotely.

EBGP Versus IBGP


BGP supports two different types of exchanges of routing information. Exchanges between autonomous systems are called
external BGP (EBGP) sessions and handle inter-AS routing. Exchanges within an AS are called internal BGP (IBGP) sessions, and
handle intra-AS routing.
An EBGP peer connection is between a device in one AS and another device in a different AS. The connection between the two
autonomous systems consists of a physical connection and a BGP connection. The physical connection is a shared
data-link-layer subnetwork between the two autonomous systems. On this shared subnetwork, each AS has at least one border
gateway belonging to that AS. The BGP connection exists between BGP speakers in each of the autonomous systems. This
session can communicate destinations that can be reached through the advertising AS. The EBGP connection typically is
established between immediately connected devices located in two different autonomous systems because the time to live
(TIL) value of the EBGP packets is equal to 1, by default.
An IBGP connection is typically established between loopback interfaces of the routers not immediately connected. BGP uses
the loopback interfaces for stability reasons-these interfaces are always functional, unless the router itself stops to function.
Because the IBGP connection typically exists between remotely connected routers, there is a requirement for an IGP within the
AS. TCP sessions for BGP speakers are established using regular routing tables.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-23


Junos Troubleshooting in the NOC

EBGP and IBGP


/
�--------------------- ' ,/ ISP-XAS 2 ,'

.•S•,
I \
\
'
I

l
I

e7�
I

·� [&1. �
..=----------
� -�-------'�
I-�· '��
. �
·•'

- �:�-.��- - e-...:.__
' ' .

re,· · · · · · · ®,':
1 I EBGP, ,..------------------,,
CUstomer AS 1

uses default route



-- :
i __·_-�..�
to the Internet , ISP-YAS 3
'------------------/Customer2

EBGP and IBGP


A BGP system shares network reachability information with adjacent BGP systems referred to as neighbors or peers.
BGP systems are arranged into groups. In an IBGP group, all peers in the group, called internal peers, are in the same AS.
Internal peers can be anywhere in the AS and need not be directly connected to each other. Internal groups use routes from an
IGP to resolve forwarding addresses. They also propagate external routes among all other internal routes running IBGP, and they
compute the next hop by taking the BGP next-hop attribute received with the route and resolving it using information from an IGP
or static routing.
In an EBGP peer group the peers are called external peers and are in different autonomous systems, but usually share a subnet.
In an external group, the next hop is computed with respect to the interface shared between the external peers and the local
router.

Chapter 10-24 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

IBGP Loopback Interfaces

• IBGP loopback interfaces


• IBGP peering is often done using loopback interfaces
• More stable
• Not tied to a single physical path
• The AS needs an IGP because IBGP speakers need to reach
each others' loopback addresses

=
.,, .,e�-- · · · · · =-;e ,
--------------------------------------------------
I
, /
'
\
I l

1
.. · Full-Mesh ,•'1 Router B
Router A •, I . 1BGP . ••
� v
··.. .
..1.·

,.,,,,··

Router c .�· 1o0: 19 .168


.. . . . .

2 255.3/32

�-------------------------------------------------'
AS1

IBGP Use of Loopback Interfaces


IBGP peers often use loopback interfaces. The advantage of using loopback interfaces is that they eliminate a dependency that
would otherwise occur when you use the IP address of a physical interface to configure BGP. The slide shows a network in which
using the loopback interface is advantageous.
On the slide, Routers A and Brun IBGP within AS 1. If Router A were to specify the IP address of an Ethernet interface on Router
Bin the remote neighbor with the router configuration command, and if the specified interface were to become unavailable,
Router A would not be able to establish a TCP connection with Router B. Instead, Router A specifies the IP address of the
loopback interface that Router Bdefines. When the loopback interface is used, BGP does not have to rely on the availability of a
particular interface for making TCP connections. Router A specifies the IP address of the loopback interface (192.168.255.2) of
Router B.
Note that BGP rarely uses loopback interfaces between EBGP peers because EBGP peers usually are connected directly . EBGP
peers therefore depend on a particular physical interface for connectivity, however, exceptions include parallel paths.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-25


Junes Troubleshooting in the NOC

BGP Route Advertisement ules

• BG P route advertisement rules


• Advertise only the active BGP routes to peers
• Inactive BGP routes are not advertised by default
• Hidden BGP routes cannot be advertised
• Can advertise inactive routes with the advertise­
inacti ve command
• Does not advertise hidden routes
• Advertise EBGP routes to all BGP peers
• Never advertise IBGP routes to IBGP peers
• Prevents loops

BGP Route Advertisement Rules


Prior to route advertisement, BGP always checks if the routes are active. If the BGP routes are not active they are not advertised
by default. Inactive routes still appear in the routing table for load balancing or redundancy reasons. However, hidden routes do
not appear in the routing table and cannot be used unless the reason behind them being hidden is resolved. BGP routes can be
inactive or hidden for many reasons, some of those reasons include an improperly written import policy, duplicate cluster-IDs,
protocol preference, AS path loop detection, route damping, and an unreachable next-hop attribute. There are many other
reasons for BGP routes to be hidden or inactive, but discussing those reasons is beyond the scope of this chapter.
If you need to advertise an inactive BGP route, you can use the advertise-inactive command under the (edit
protocols bgp], [edit protocols bgp bqp-qroup],or [edit protocols bgp bqp-qroup neighbor
neighbor-ip] hierarchy levels. However, you cannot advertise hidden BGP routes through the use of the
advertise-inactive command.
To prevent loops, BGP has very strict rules about route advertisement. If an EBGP speaking router receives an EBGP route, it
advertises the route to all of its EBGP and IBGP speaking neighbors, except for the EBGP speaking neighbor that it originally
received the route from. Any IBGP speaking routers that receive the route only pass it on to other EBGP speaking neighbors, if
any. By default, IBGP speaking routers never advertise IBGP routes to other IBGP speaking neighbors. This requirement is
relaxed if route reflectors or confederations are present in the BGP domain. Note that these requirements do not apply for
routing information that is redistributed into BGP from another routing protocol. For example, if OSPF routes are being
redistributed into BGP on a router that only has IBGP speaking neighbors, the router advertises the routing information to its
IBGP speaking neighbors.

Chapter 10-26 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

BGP Routes in Main Routing Table

• BGP routes in the main routing table


• BGP routes are placed in the main routing table
(inet. O)
• Routing table stores:
• Routing information learned from update messages
• Routing information that passes sanity check (for
instance, AS loop detection)
• Local routing information selected by applying local
policies to routes received in update messages

BGP Routes in the Main Routing Table


The Junos OS places BGP routes that pass the import policy in the local routing table: inet.O. The inet.0 routing table contains
entries from all unicast routing protocols.
The Junos OS stores BGP routes in the inet.O table when they are learned from the update messages, checked for their validity,
and conform with the import policy.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-27


Junos Troubleshooting in the NOC

Monitoring BGP Operation


• Several commands display a wide variety of BGP
information, either from the protocol itself or from
BGP routes

user@router> show bgp?


Possible completions:
bmp Show BGP Monitoring Protocol statistics
group Show the BGP group database
neighbor Show the BGP neighbor database
replication BGP NSR replication state between master and backup
summary Show overview of BGP information

Monitoring BGP Operation


The Junos OS has a wide variety of BGP monitoring commands. This slide displays the following commands:
show bgp bmp: Shows the BGP Monitoring Protocol statistics.
show bgp group: Shows the BGP group database.
show bgp neighbor: Shows the BGP neighbor database.
show bgp replication: Show nonstop active routing (NSR) replication state between master and backup
show bgp summary: Displays the overall BGP information, including the state of BGP peer session
establishment.

Chapter 10-28 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Displaying BGP Group Information


• View information about BGP groups:
useI@router> show bgp group
Group Type: Internal AS: 11111 Local AS: 11111
Name: int Index: 1 Flags: <Export Eval>
Export: [ bgp-out
Holdtime: 0 One peer defined. session
11
(Total oee rs: 1
2.2.2.2+179
Established:
.... ................. established
'r-..__
inet.O: 0/0iOiO

Group Type: External Local AS: 11111


Name: 1 Index: 0 Flags: <>
Haldtime: 0
i.;········...
'"!T_c_t_a_l_ _e_e_r_s_:_l____"_s_t_a_b_l_i_s_h_e_d___,
p
: O ! ····················
One peer defined. session
not established
123.133.1.1

Groups: 2 Peers: 2 External: 1 Internal: 1 Down peers: 1 Flaps: 0


Table Tot Paths Act Paths Suppressed Hist,:n:y Damp State Pending

Displaying BGP Group Information


Use the show bgp group command to view BGP group information. Some of the output fields of this command are the
following:
Group Type: Displays the type of BGP group. It can be either Internal or External.
Local AS: Displays the number of the local AS.
Export: Displays the export policies configured for the BGP group with the export statement.
Total peers: Displays the total number of peers in the group.
Established: Displays the number of peers within the group that are in the established state.
Flags: Flags associated with the BGP group. This field is used by Juniper Networks customer support.
Holdt.i me: Maximum number of seconds allowed to elapse between successive keepalive or update messages
that BGP receives from a peer in the BGP group, after which the connection to the peer is closed and routing
devices through that peer become unavailable.
ip addresses: Displays the list of peers that are members of the group. The address is followed by the peer's
port number.
Index: Unique index number of a BGP group.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-29


Junos Troubleshooting in the NOC

Displaying BGP Summary Information

• Use the show bgp summary command to view


basic information about all BGP neighbors
user@router> show bgp summary
Groups: 2 Peers: 3 Down peers: 1
! inet. o routes !
Table Tot Paths Act Paths Suppressed History Damp state Pending\
inet.O 7 7 0 0 0 0
Peer AS In Pkt Out Pkt OutQ Flaps Last Up/Dwn
State I #Active/Received/Damped.. ········································· ·························· ··
· · 1, .
.
192 168.8.1 3 8 11 0 0 2:37 5/5/0/0 £
.
0/0/0/0
10.0.8.1 10 0 0 0 0 17:17 Idle
10.0.31.1 2 6 8 0 0 0 2/2/0/0
o;o;o;ov..
··. . .
l':. ... .
·· ..
I inet. 2 routes I Established
..

Displaying BGP Summary Information


The output fields of the show bgp summary command are the following:
Groups: Displays the number of BGP groups.
Peers: Displays the number of BGP peers.
Down peers: Displays the number of unestablished BGP peers.
Peer: Displays the address of each BGP peer. Each peer has one line of output.
AS: Displays the peer's AS number.
InPkt: Displays the number of packets received from the peer.
Out Pkt: Displays the number of packets sent to the peer.
OutQ: Displays the count of the number of BGP packets queued to be transmitted to a particular neighbor. It
usually is O because the queue is emptied quickly.
Last Up/Down: Displays the last time since the neighbor transitioned to or form the established state.
Continued on the next page.

Chapter 10-30 • Troubleshooting Routing Protocols www.juniper.net


Junes Troubleshooting in the NOC

Displaying BGP Summary Information (contd.)


State I #Active/Received/Accepted/Damped: Displays either the BGP state or, if the neighbor is in the
established state, the number of paths received from the neighbor, as well as the number of these paths that have
been accepted as active, and hence are being used for forwarding, as well as the number of routes that have been
suppressed due to damping. The display always includes entries for the inet. O and inet. 2 routing tables. The
inet. 2 table holds unicast routes that are used for multicast reverse path forwarding (RPF) lookup when
multiprotocol BGP is in effect to support routes with a SAFI value of 2. For example, 8/10/10/2 2/4/4/0 indicates
the following:
8 active routes, 10 received routes, 10 accepted routes, and 2 damped routes from a BGP peer appear in
the inet.0 routing table.
2 active routes, 4 received routes, 4 accepted routes, and no damped routes from a BGP peer appear in
the inet.2 routing table.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-31


Junes Troubleshooting in the NOC

Displaying BGP Neighbors


user@router> show bgp neighbor
Peer: 10.12.12.100+179 AS 1111 Local: 10.12.12.1+56624 AS 2222
Type: External state: Established Flags: <Sync> ..o1 :.----"""'.Type and state
.,E of BGP session
Last State: Openconfirm Last Event: RecvKee Alive
Last Error: None
!options: <Preference Damping PeerAS Refresh>� Supported
Holdtime: 90 Preference: 170 options
Number of flaps: 0
Peer ID: 10.12.12.100 Local ID: 1.1.1.l Active Holdtime: 90
Keepalive Interval: 30 Group index: 1 Peer index: 0
BFD: disabled, down
Local Interface: ge-1/0/7.0
NLRI for restart configured on peer: inet-unicast
NLRI advertised by peer: inet-unicast
NLRI for this session: inet-unicast
Peer supports Refresh capability (2)
stale routes from peer are kept for: 300
Peer does not support Restarter functionality Negotiated
NLRI that restart is negotiated for: inet-unicast BGPsession
NLRI of received end-of-rib markers: inet-unicast properties
NLRI of all end-of-rib markers sent: inet-unicast
Peer supports 4 byte AS extension (peer-as 1111)
Peer does not support Addpath

Displaying BGP Neighbors


The output fields of the show bgp neighbor command are the following:
Peer: Displays the address of each BGP peer. Each peer has one line of output.
Type: Displays the type of peer (Internal or External).
State: Displays the BGP state for this neighbor.
Flags: Displays the internal peer-specific flags for this neighbor.
Last State: Displays the BGP state of this neighbor prior to the current state.

Last Event: Displays the last BGP state transition event.


Last Error: Displays the last notification message sent to the neighbor.
Options: Displays the configuration options in effect for this neighbor.
Holdtime: Displays the configured hold time for this neighbor.
Preference: Displays the configuration preference for routes learned from the neighbor.
Peer ID: Displays the neighbor's router ID.
Local ID: Displays the local system's router ID.
Continued on the next page.

Chapter 10-32 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Displaying BGP Neighbors (contd.)


Active Holdtime: Displays the hold-time value that was negotiated during the BGP open.
Table inet. O Bit: Displays the Internal bit used for the peer group.
Send state: Displays whether all peers in the group have received all their updates (in sync or out of
sync).
Active Prefixes: Displays the number of prefixes accepted as active from this neighbor.
Last traffic (seconds): Displays how recently a BGP message was sent or received between the local
system and this neighbor.
Output Queue: Displays the number of BGP update messages pending for transmission to the neighbor.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-33


Junos Troubleshooting in the NOC

Clearing BGP Neighbors


• Use the clear bgp neighbor command to clear
BGP sessions
• All sessions are cleared when a specific neighbor address is
not specified!
• Use the soft or soft-inbound options to force
readvertisement of prefixes without clearing the session
• Requires BGP refresh capability support

user@router> clear bgp neighbor


Cleared 2

user@router> clear bgp neighbor 192.168.12.1 soft-inbound'(: ...


····· .... ...·
······
A soft clear does not reset the BGP session

Clearing BGP Neighbors


Use the clear bgp neighbor command to reset a BGP session. You must use this command with caution because clearing
a BGP session can result in significant disruption while the BGP session is being reestablished. Note that when you omit a
specific BGP session address as an option, the result is a clearing of al/ BGP sessions that are currently established!
You can use the soft and soft-inbound options to force the soft reset of outbound or inbound state respectively. These
options make use of the BGP refresh capability to support the readvertisement of prefixes that have been previously advertised
(and acknowledged by the TCP transport). A hard clearing of the session is the only way to force a BGP speaker to readvertise
routes when the refresh capability is not supported because BGP is a stateful protocol that does not make use of periodic
retransmissions.

In most cases soft clears are performed to support changes in VPN policy that result in previously discarded prefixes now being
considered a match against one or more locally defined route targets. In a non-VPN context, a soft clear is used to refresh routes
that are not stored in input RIB due to input sanity checks such like AS loop detection. After enabling AS loops support under
[edit routing-options], a soft clear should result in the display of routes that were previously discarded due to an AS
loop condition, without forcing the tear down and reestablishment of the related BGP session.

Chapter 10-34 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Viewing BGP Route Details


user@rcuter> show route 200.0.3.0/24 extensive

inet.O: 12 destinations t 12 routes (12 active, 0 holddown, 0 hidden)


200.0.3.0/24 (1 entry, 1 announced)
TSI:
KRT in-kernel 200.0.3.0/24 -> {10.12.12.100)
Page O idx O Type 1 val 28b5944
Flags: Nexthop Change
Nexthcp: Self
Localpref: 100
AS path: [2222) 1111 I
Communities:
Path 200.0.3.0 from 10.12.12.100 Vector len 4. Yal: 0
*BGP Preference: 170/-101
Next hop type: Router, Next hop index: 737
Address: Ox2945698
Next-hop reference count: 3
Source: 10.12.12.100
Next hop: 10.12.12.100 via ge-1/0/7.0, selected
Session Id: Ox102
State: <Active Ext>
Local AS: 2222 Peer AS: 1111
Age: 8:36
Validation State: unverified
Task: BGP_llll.10.12.12.100+179
Announcement bits (3): 0-KRT 2-BGP RT_Backgrcund 4-Resolve tiee 1
AS path: 1111 I
Accept:.ed
Localpref: 100
Router ID: 10.12.12.100

Viewing Specific Routes


To see all information associated with a given prefix (for example, BGP prefixes information such as the AS path, origin, local
preference, and MED attributes as well as any community strings associated with the prefix), use the show route
extensive command. You also can use this command to determine the reason why the prefix is hidden (for example, an
unusable next hop).

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-35


Junos Troubleshooting in the NOC

BGP Route Advertisement Commands


• BGP route advertisement commands
• show route receive-protocol bgp address
• Displays routes received by a peer before policy is applied
user@router> show route receive-protocol bgp 10.123.123.1

inet.O: 22 destinations, 27 routes (22 active, O holddown, 0 hidden)


Prefix Next hop MED Lclpref AS path
10.10.10.0/24 10.123.123.1 12345 I
* 10.11.11.0/30 10.123.123.1 12345 I
• show route advertising-protocol bgp
address
• Displays routes advertised to a specific peer
user@router> show route advertising-protocol bgp 10.123.123.1

inet.O: 22 destinations, 27 routes (22 active, O holddown, O hidden)


Prefix Nexthop MED Lclpref AS path
* 1.1.1.1/32 Self I
* 2.2.2.2/32 Self 1 I

Displaying Routes Received or Advertised to a Peer


The show route receive-protocol. bgp address command displays the routing information as it was received
through a particular neighbor of a particular dynamic routing protocol. This information includes the routes that the local router
advertised to the neighbor. The information reflects the routes before they are filtered by that protocol's import policy
statements.
The show route advertising-protocol. bgp address command displays the routing information as it has been
prepared for advertisement to a particular neighbor of a particular dynamic routing protocol. The information reflects the routes
that the routing table exported into the routing protocol and that were filtered by that protocol's export routing policy statements.

Chapter 10-36 • Troubleshooting Routing Protocols www.juniper.net


Junes Troubleshooting in the NOC

Hidden BGP Routes (1 of 2)

v:
··.... ., ,
·· ... · ···········........
·
200_0_3_0 24. / ------
NH=10_0_16_2

•show route hiddenandshow route


resolution unresolved displays hidden routes
user@R2> show route 200.0.3.0/24 hidden

inet.0: 4E. destinations, 61 routes (41 active, 0 holddown, 20 hidden)


+ = Active Route, - = Last Active, * = Both

200.0.3.0/24 100, from 192.168.16.1


unverified

Hidden BGP Routes


Routes that fail certain sanity checks are hidden by the Junos OS. A common reason for a hidden route is an EBGP speaking
router that does not overwrite the next hop before readvertising the routes to IBGP peers. This situation creates a next hop
reachability problem, as shown on the slide, because in most cases the addressing used to support EBGP peerings is not
carried by the IGP within that AS. Note that the EBGP speaking router has no problem resolving the EBGP next hop by virtue of
its direct attachment to the EBGP peering link. This next hop resolution occurs without effort because EBGP speaking routers
change the next hop attribute to their outgoing interface IP address by default.
This condition is shown on the slide where the 200.0.3.0/24 route is advertised to EBGP speaker Rl . Rl has no trouble
resolving the 10.0.16.2 next hop, and it prefers this route so it is installed as an active route. The rules of IBGP advertisement
cause Rl to advertise the 200.0.3.0/24 route to its IBGP neighbor, R2. With the default behaviors in place, the advertisement
sent to R2 contains the original EBGP next hop of 10.0.16.2. This is where the problem occurs because R2 has no knowledge of
how to reach the 10.0.16.2 address (unlike Rl, R2 is not attached to the 10.0.16.0/25 subnet).
The show route resolution unresolved command provides additional information that helps illuminate the problem:
user@R2> show route resolution unresolved I find 200. 0.3.0/24
200.0.3.0/24
Protocol Nexthop: 10.0.16.2
Indirect nexthop: 0 -

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-37


Junos Troubleshooting in the NOC

Hidden BGP Routes (2 of 2)

-- ... ....
... '
Y..
··.. ··.. ., I

., ,,,
···············
200.0.3.0/24,
-------
NH=10.0.16.2

• Turning unresolved routes into resolvable routes


• Place R1's external facing interface into the IGP
• Or, apply a policy on R1 that changes the next-hop attribute
to an IP address that is resolvable by R2

Resolving Previously Unresolvable Routes


To resolve the previously unresolvable route on R2, there are a few different approaches you can take. The simplest method
is to take Rl's external facing interface (the interface that points to R3) and enable the IGP on it. For example, if Rl and R2
are using OSPF as the IGP, you can place Rl's external facing interface into OSPF with the passive option. This method
places the prefix that is associated with Rl's external facing interface into OSPF, which allows RL to use R3's interface
address as the next hop of the BGP route. However, this method can be undesirable because it introduces more routes into
the IGP that are used for nothing more then BGP next hop resolution. The other preferred, and more complicated, method is
to configure a routing policy the changes the next hop attribute of the BGP routes that Rl advertises to R2.
The following output is an example of a routing policy that can be applied to Rl as an export policy for the IBGP group to
change the next hop attribute of all prefixes that Rl is advertising to R2 through IBGP. The policy uses the next-hop self
statement that changes the next hop attribute to Rl's loopback IP address.
user@Rl> show configuration policy-options
policy-statement next-hop-self-policy {
term 1 (
then {
next-hop self;

Chapter 10-38 • Troubleshooting Routing P rotocols www.juniper.net


Junos Troubleshooting in the NOC

Damping and BGP Routes


• BGP routes can also be hidden because of damping
• Restore with clear bgp damping command
user@router> show route 200.0.3.0/24 hidden

inet.O: 12 de stin t ions, 12 routes (11 acti-..re, 0 holddcwn, hidden)


+=Active Route, a = Last Active, w = Both

200.IJ.3.0/24 [BGP 00:00:05, localoref 100


AS path: 1111 I, validation-state: unverified
> to 10.12.12.100 via ge-1/0/7.0

user@router> show route protocol bgp 200.0.3.0/24 damping suppressed detail

inet.0: 12 destinations, 12 routes (11 acti"'J"e, 0 holddown, 1 hidd en)


200.0.3.0/24 (1 entry, 0 announced)
BGP /-101
Next hop ty-pe: Router, Next hop index: 737

Source: 10.12.12.100 Damping


i e s el e statistics
a
/
_· ;_� �-�_:_� lO_ _O v_ g - l / O / �_. _· O_, _ _ _e_r_- '"_- _u_·
,.;�...;�...;:_;_;_;_� :_ir
_, 1_
_· :�_
-,
,,, ,.. ,. ...,.. ..., ,,.,,.. "' '" "' "' "" "" '" _
Merit (last update/now): 5765/5720 ----
Default damping parameters u s ed
Last update: 00:00:17 First update: G0:01:50
Flaps: 6
Suppressed. Reusable in: 00:':14:00
Preference will be: 170

Route Damping
EBGP routes that experience too many state transitions (flaps) in a given period of time are hidden when route damping is
enabled. These routes are displayed with a show route hidden command, along with all other hidden routes. You can add
the detail option to the show route hidden command to determine if BGP damping has caused the route to be hidden.
The show route protocol bgp damping command in conjunction with the suppressed option is even better because
this command displays only those routes that are hidden due to BGP damping.
Use the clear bgp damping command to activate BGP routes that have been hidden because of damping when you believe
that the flapping condition has been corrected, and you do not want to wait for the route's figure of merit to decay by itself.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-39


Junos Troubleshooting in the NOC

Tracing BGP
•Atypical BGP tracing configuration:
[edit protocols bgp group ext]
user@router# show
type external;
traceoptions {
file ebgp-trace;
flag open detail;
flag update detail;

peer-as 10;
neighbor 10.0.8.1;
neighbor 10.0.31.1
peer-as 2;
}

• Sample output:
user@router> monitor start ebgp-trace
ebgp-trace ***
Apr 6 16:45:50 bgp_send: sending 45 bytes to 10.0.31.1 (External AS 2)
Apr 6 16:45:50
Apr 6 16:45:50 BGP SEND 10.0.31.2+2541 -> 10.0.31.1+179
Apr 6 16:45:50 BGP SEND message type 1 (Open) length 45
.F.::;>r 6 16:45:50 BGP SEND version 4 as 3 holdtime 90 id 192.168.12.1 parmlen 16
Apr 6 16:45:50 BGP SEND MP capability AFI=1, SAFI=l

BGPTracing
To perform debugging functions on the BGP routing process, use the Junos OS traceoptions feature. The trace output (debug
information) is directed to the named log file, which is located in the
/var I log/ directory on the router's hard drive. You can view the log file using the monitor start or show log
operational-mode commands. In addition to specifying the trace file, you also must tell the router what information you want to
trace. You accomplish this by specifying one or more flag keywords.
While you can only direct tracing to a single file, you can trace many options by using the flag keyword multiple times. In
addition, you can add granularity by using the detail, receive, and send flag modifiers.
Continued on the next page.

Chapter 10-40 • Troubleshooting Routing Protocols www.juniper.net


Junes Troubleshooting in the NOC

BGP Tracing (contd.)


Available tracing flags for BGP include the following:

4byte-as Trace 4 byte AS events


add-path Trace add-path events
al.l. Trace everything
bfd Trace BFD events
damping Trace BGP damping information
general. Trace general events
grace-restart Trace Graceful Restart events
keepal.ive Trace BGP keepalive packets
normal. Trace normal events
nsr-synchronization Trace NSR synchronization events

open Trace BGP open packets


packets Trace all BGP protocol packets
pol.icy Trace policy processing
refresh Trace BGP refresh packets
route Trace routing information
state Trace state transitions
task Trace routing protocol task processing
timer Trace routing protocol timer processing
update Trace BGP update packets

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-41


Junos Troubleshooting in the NOC

BGP Session Troubleshooting Checklist


• Checklist for BGP session problems
• Verify IP connectivity between BGP peers
• Is the interface up and working?
• Is there a route and can you reach the peer address?
• Check the configuration parameters
• Are the appropriate address families enabled?
• Verify the peer and local AS number as well as IP addresses
• Check the authentication settings
• Consider temporarily disabling the authentication
• Monitor the syslog file for BGP related messages
• Check for firewall filters
• Do firewall filters block the TCP port 179?
• Does the problem still exist?
• Use traceoptions or the monitor traffic interface command

Checklist for BGP Session Problems


To verify IP connectivity using the session IP address, use ping utility:
user@router> ping 172.27.255.3 source 172.27.255.1
If the ping fails, check IP routing:
user@router> show route 172.27.255.3
Verify the parameter settings using the show protocols bgp group command:
[edit]
user@router# show protocols bgp group IBGP
type internal;
local-address 172.27.255.1;
authentication-key "$9$I56hyKX7V4aUM8aUjH5TRhS"; ## SECRET-DATA
export NHS;
neighbor 172.27.255.3;
neighbor 172.27.255.4;
Continued on the next page.

Chapter 10-42 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC
Checklist for BGP Session Problems (contd.)
Check the authentication settings:
lab@Rl> show bgp neighbor 172.27.255.3
Peer: 172.27.255.3 AS 3895077211 Local: 172.27.255.1 AS 3895077211
Type: Internal State: Active Flags: <>
Last State: Idle Last Event: Start
Last Error: None
Export: [ NHS ]
Options: <Preference LocalAddress AuthKey LogUpDown AddressFamily Refresh>
Authentication key is configured
Address families configured: inet-unicast inet6-labeled-unicast
Local Address: 172.27.255.1 Holdtime: 90 Preference: 170
NI.RI inet6-labeled-unicast: ExplicitNull
Number of flaps: 0
Consider temporarily disabling the authentication to eliminate the potential problem.
Syslog can provide an invaluable source of information for troubleshooting.
lab@R4> show log messages I match NOTIFICATION
Jul 29 00:05:29 R4 rpd[l068]: bgp_rt_maxprefixes_check_common:6856: NOTIFICATION
sent to 2008:4498:0:1.::2 (External AS 65432): code 6 (Cease) subcode 1 (Maximum
Number of Prefixes Reached) AFI: 2 SAFI: 1 prefix limit 12
Jul 29 00:06:01 R4 rpd[1068]: bgp_pp_recv:2961: NOTIFICATION sent to
2008:4498:0:1::2+64490 (proto): code 2 (Open Message Error) subcode 5
(authentication failure), Reason: no group for 2008:4498:0:1::2+64490 (proto)
from AS 65432 found (peer idled due to prefix-limit violation), dropping him

Check Firewall Filters


In the exam, you might be asked to configure firewall filters. Check the BGP session status after the filters are implemented
to make sure that the filters do not block BGP communication.
If you still cannot find the source of problem with BGP, use traceoptions to or themonitor traffic interface
command debug BGP sessions.
lab@R4> show log bgp-trace.log
Feb 11 12:52:31.750571 BGP RECV 201.201. 0.1+179 -> 10.0.3.4+3291
Feb 11 12:52:31.750622 BGP RECV message type 1 (Open) length 45
Feb 11 12:52:31.750668 BGP RECV version 4 as 65011 holdtime 90 id 201.201.0.1 parmlen
16
Feb 11 12:52:31.750692 BGP RECV MP capability AFI=l, SAFI=l
Feb 11 12:52:31.750707 BGP RECV Refresh capability, code=l28
Feb 11 12:52:31.750718 BGP RECV Refresh capability, code= 2
Feb 11 12:52:31.751108 bgp_process_open: NOTIFICATION sent to 201.201.0.1 (External
AS 65010): code 2 (Open Message Error) subcode 2 (bad peer AS number), Reason:
peer 201.201.0.1 (External AS 65010) ciaims 65oii, 650io configured
Feb 11 12:52:31.751156 bgp send: sending 21 bytes to 201.201.0.1 (External AS 65010)
Feb 11 12:52:31.751175
Feb 11 12:52:31.751175 BGP SEND 10.0.3.4+3291 -> 201.201.0.1+179
Feb 11 12:52:31.751194 BGP SEND message type 3 (Notification) length 21
Feb 11 12:52:31.751208 BGP SEND Notification code 2 (Open Message Error) subcode 2
(bad peer AS number)

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-43


Junos Troubleshooting in the NOC

Troubleshooting EBGP Specifics

• Remember the EBGP has unique requirements


• mul tihop is required for loopback peering
• A route must exist to the remote address
• peer-as configuration
• Troubleshooting can be more difficult because it
might not be able to verify the peer's configuration
•Usetraceoptionsormonitor traffic interface
• Can also be used to determine a remote device's BGP parameters
that are unknown or undefined

EBGP Configuration Specifics


Recall that EBGP sessions take TIL value of 1 by default, hence for a loopback-to-loopback peering session, mul tihop
configuration is mandatory.
Watch for mismatched peer AS configuration.

EBGP Troubleshooting Specifics


EBGP sessions are often harder to troubleshoot because you might not be able to look at the peer router's configuration. Use
traceoptions if you cannot find a source of problem by checking your router's configuration. You can also monitor the control
traffic using the monitor traffic interface interface name command.

Chapter 10-44 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Agenda: Troubleshooting Routing Protocols

• Troubleshooting OSPF
• Troubleshooting BGP
7Troubleshooting Routing Loops and Route Oscillation

Troubleshooting Routing Loops and Route Oscillation


The slide highlights the topic we discuss next.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-45


Junes Troubleshooting in the NOC

Defining a Routing Loop

• What is a routing loop?


• Anytime a packet passes through a routing instance more
than once
• Packets will loop until the TIL expires
• Typically occurs at points of route redistribution

A Routing Loop Defined


Any time a packet passes through a routing instance, typically the master routing instance, more than once, a routing loop
has formed. Link-state IGPs have mechanisms that make them very resistant to forming routing loops. Because of the
routing loop resistant behavior of link-state IGPs, routing loops typically form at points where route redistribution is taking
place. The IGP currently employed in the network is not privy to the routing information contained within another routing
protocol. An unwittingly written routing policy has the ability to inject routing information that causes a routing loop.

Chapter 10-46 • Troubleshooting Routing Protocols www.juniper.net


Junes Troubleshooting in the NOC

Routing Loop Occurrence (1 of 2)

RIP-1 R}-�2
�-······················.>� Areao
� �-·-·-·-��
,,r, /�.....I ··
. ''
I . '"'
I

Cif . . . . . . . . . . ��-�[J
i
.
v
/. .',
'.'
I
I

Host-2

RIP-2 R3 R4

0 Directly connected route on R4 is sent to the OSPF routers as an external OSPF route

0 R3 redistributes the external OSPF route into RIP and sends it to RIP-2

G) RIP-2 sends the route to RIP-1. then RIP-1 sends the route to R1

0 R1 redistributes the RIP route into OSPF as an external type 1 OSPF route. R2 and R3
now attempt to forward traffic through R1 to reach Host-2

How a Routing Loop Forms


The slide depicts how a routing loop can form. First, R4 sends a directly connected route, which tells the other routers how to
reach Host-2, into OSPF. This route is an type 2 external OSPF route, which is the default external OSPF route type. Next, R3
redistributes this route into RIP and sends it to the RIP-2 router. Then, the RIP-2 router sends it through RIP to the RIP-1
router. The RIP-1 router then sends the route to R1. Next, R1 uses a policy to redistribute this route into OSPF as an external
type 1 OSPF route. As you might already know, external type 1 OSPF routes are more preferred than their external type 2
counter parts. This behavior causes R2 and R3 to prefer the version of the route that R1 is redistributing into the OSPF
domain.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-4 7


Junes Troubleshooting in the NOC

Routing Loop Occurrence (2 of 2)

Host-2

(0 Host-1 attempts to send traffic to Host-2


G) The traffic goes to R1. to RIP-1. to RIP-2. to R3. to R2. and back to R1 again
© All traffic sent from Host-1 to Host-2 continues to loop on the previous path

A Routing Loop in Action


Now that the routing information has been improperly distributed among the routers, a routing loop is present. The routing
loop is evident when Host-1 attempts to send traffic to Host-2. The traffic first arrives at R1, which sends the traffic to the
RIP-1 router. Then the RIP-1 router sends the traffic to the RIP-2 router. Next, R3 receives the traffic and passes it on to R2
which then forwards it to R1. R1 has no idea that it has seen this traffic before. so it examines the destination address and
forwards it to the RIP-1 router. This routing loop results in traffic that is sent from Host-1 that can never reach Host-2.

Chapter 10-48 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Troubleshooting Routing Loops


• How to troubleshoot a routing loop
• traceroute
• Helps to determine where the routing loop is occurring
• Examine routing tables
• Once you have found the point at which the packet loops, the
routing tables on the involved routers can give valuable
information

Troubleshooting a Routing Loop


When troubleshooting a routing loop, you must quickly determine where the loop is actually taking place. Use the
traceroute command, from multiple routers if necessary, to help you narrow down the point at which the loop is
occurring. Once you have determined where the routing loop is occurring, examine the routing tables on those routers. This
will provide you with the information that is necessary to correct the routing loop. Solving a routing loop might require you to
change the preference value of a routing protocol, block a certain route, or change a routing policy.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-49


Junos Troubleshooting in the NOC

Defining Route Oscillation


• What is route oscillation?
• Routing information is constantly added to and removed
from the routing table
• Typically is caused by improper route redistribution
• Might cause a routing loop or suboptimal routing

Route Oscillation Defined


Route oscillation occurs when routing information is constantly added, then removed, then added again. Route oscillation
situations, much like routing loops, are typically caused by improper route redistribution. The insidious nature of route
oscillation is that you typically see inconsistent traffic patterns. For example, some traffic might take the optimal path to the
destination, some traffic might form a routing loop, and other traffic might take a sub-optimal path to the destination.

Chapter 10-50 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Route Oscillation Occurrence

Route Oscillation in Action


The slide depicts a scenario in which route oscillation is occurring. To begin, the Internet router sends Rl a route through
EBGP, then Rl passes this route on to R2 through IBGP. R2 currently has a policy that redistributes all BGP routes into OSPF.
This behavior causes the OSPF version of the route to appear in the routing tables of Rl, R3, and R4. When Rl receives the
OSPF version of the route it becomes active over the BGP version of the route, due to a better preference value of the OSPF
route. This causes Rl to send a BGP message to R2 through IBGP to withdraw the BGP version of the route. This action by Rl
causes R2 to withdraw the OSPF advertisement of the route. These route withdrawals cause the BGP version of the route to
become active again on Rl, and Rl passes the BGP version of the route through IBGP to R2. Then, the route oscillation cycle
begins again.

www.juniper.net Troubleshooting Routing P rotocols • Chapter 10-51


Junos Troubleshooting in the NOC

Troubleshooting Route Oscillation


• How to troubleshoot route oscillation
• Use multiple traceroutes to determine if route oscillation is
occurring
• One traceroute might show suboptimal routing, another might
show optimal routing. and another might show a routing loop
• Examine routing tables for the age of routes in question
• Ensure that the version of the route that is entering the
network is preferred

Troubleshooting Route Oscillation


If you suspect that route oscillation is occurring, be sure to issue multiple traceroutes. As one version of the route propagates
throughout the network over another version, the forwarding path for the traffic changes. You might have a couple of
traceroutes that show optimal routing, then another traceroute might show suboptimal routing, and you might even see a
routing loop briefly appear.
One clue to look for while discovering a route oscillation problem is the age of routes in the routing table. Route oscillation
results in routes constantly being advertised and withdrawn from the routing table of the routers in your network. If you
notice that the routes in a routing table are always very new, possibly less than 10 seconds old, route oscillation might be
occurring.
To fix a route oscillation problem, you must first discover where the route is initially entering the network. Once you have
found this original routing information, ensure that the protocol that is receiving it into the network is more preferred for the
specific routing information. This adjustment will break the route oscillation cycle and allow the route to properly be passed
through the network.

Chapter 10-52 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Routing Loop Case Study (1 of 8)


• Traffic sent from Host-1 to Host-2 never reaches
Host-2. Issuing a traceroute shows that a routing loop
is occurring. You must troubleshoot the problem and
fix the routing loop.
�D I

e·· · · · · · · ·�e-- e
Host-1 �RPtoOSPF:
RIP-1 ''\R:t�2

� � -0->
r a


:
:
.
· ',, I ',
""" ·. '
/:,.

,..®·. --Q�.-,-iD·
i osPFto_RIP_j ' - .. - '' -
. i c� �o os�� j
,!, "'-� , I 1. � ::_

· ·.�
. ..················.....
...........1------
. . .2
Host-2
RIP-2 R3 R4
·� · • 1721617

Routing Loop Case Study


In this case study, Host-1 is attempting to send traffic to Host-2, but the traffic never arrives at its intended destinatior..
Issuing a traceroute reveals a routing loop that you must troubleshoot. Note that R3 is redistributing OSPF routes into RIP
and R1 is redistributing RIP routes into OSPF.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-53


Junos Troubleshooting in the NOC

Routing Loop Case Study (2 of 8)


• Examining the traceroute output from Host-1
user@Host-1> traceroute 172.16.17.2 E j Host-2saddress I

traceroute to 172.16.17.2 (172.16.17.2), 30 hops max, 40 byte packets

1 172.16.16.1 (172.16.16.1) 18.749 ms 4.598ms 6.105 ms�

2 10.1.2.2 (10.1.2.2) 11.969ms 15.538 ms 11.876ms�

3 10.1.3.2 (10.1.3.2) 11.981 ms 11.633ms 11.984 ms�

4 10.1.4.2 (10.1.4.2) 12.013ms 11.544 ms 12.040 ms�

5 10.1.5.1 (10.1.5.1) 12.253ms 11.127ms 11.959ms�

6 10.1.1.1 (10.1.1.1) 12.022ms 11.551ms 12.040m5�

7 10.1.2.2 (10.1.2.2) 18.044ms 17.569ms 17.887ms

8 10.1.3.2 (10.1.3.2) 17. 964ms 17.632ms 17.988ms

9 10.1.4.2 (10.1.4.2) 16.122 ms 15.314 ms 19.941 ms

10 10.1.5.1 (10.1.5.1) 17.995ms 17.546ms 18. 084m5

11 10.1.1.1 (10.1.1.1) 18.029ms 18.391ms 17.131ms

Examining the Traceroute


Examining the traceroute from Host-1 reveals vital clues in determining why the routing loop is occurring. As you can see
from the output on the slide, the traceroute path goes through R1, RIP-1, RIP-2, R3, R2, and back to R1. This traceroute
reveals two problems, first you can see that a routing loop is occurring, second you the routing loop is causing some very
sub-optimal routing to occur. You will want to fix both issues.

Chapter 10-54 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Routing Loop Case Study (3 of 8)


•Whereto look
• Begin by examining R1's routing table
• The routing loop is going through R1
• R1 is a point of route redistribution

user@Rl> show route 172 .16 .17.2 ( I Host-2"s address I


inet.O: 21 destinations, 34 routes (21 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, " Both

172.16.17.0/24 *[RIP/100] 01:22:57, metric 4, tag O


> to 10.1.2.2 via ge-0/0/2.0
[OSPF/150] 01:18:15, metric 0, tag O
> to 10.1.1.2 via ge-0/0/1.0

Where to Look
Routing loops are usually caused by improper route redistribution. Rl is redistributing OSPF routes into RIP which makes it a
prime candidate for problems . Examining the 172.16.17.0/24 route on Rl reveals that Rl has an OSPF version of the route
and a RIP version of the route. The output also reveals that Rl is preferring the RIP version of the route.
Also, you might need to examine the routing table on R3 if manipulating Rl"s configuration does not resolve the issue. Just
like Rl, R3 is redistributing routing information between OSPF and RIP.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-55


Junos Troubleshooting in the NOC

Routing Loop Case Study (4 of 8)

• Examine the OSPF routing policy


• Improper route redistribution the likely cause for most
routing loops

[edit]
user@Rl* show protocols ospf �
export ospf-out;

[edit]
user@Rl* show policy-options policy-statement ospf-out
then {
external
type l;

accept;

Examining the Routing Policy


Examining the routing policy on router R1 reveals that it is taking all non-OSPF routes and redistributing them into OSPF.
Although the 172.16.17.0/24 route was an OSPF route, it is now appearing on R1 as a RIP route, and R1 is redistributing all
RIP routes into OSPF. Any routing policy that redistributes routes into another routing protocol should never be broadly written
as this example.

Chapter 10-56 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Routing Loop Case Study (5 of 8)


• How to fix the problem
• Blocking RIP routes from being installed in R1's routing
table
• Not the best option. the RIP route can be used as a backup path
• Change the RIP or OSPF preference
• Better option. OSPF route is preferred
• Change routing policy
• Best option. improper routing policy is the problem
• Least disruptive of the options

JLJnm
"
i;;;,�:.ietwcrb, ...;;_All ilio,ts......-.
� - "'"'l""' "".-:.3'' "'Js '""' : 7 . ;;' ,.,.,.,
'.' �·2014 . .:c , Worldwide Education Services www4un;pe,.net I 57
��-�-.::.:de.,���:.�:;=-+-< ::..Ji--�-:, -- Afi �,= �

Fixing the Problem


Once you have discovered the source of the routing loop, there are many ways that you can go about fixing it. Some of these
methods are better than others, but remember, the best fixes are the most specific fixes. Implementing a board fix, such as
blocking al/ RIP routes from entering R1's routing table might have unintended consequences. The best option in our case
study is to change the routing policy on R1 to redistribute all routes into OSPF, except the offending RIP route.
If this were a live network scenario, it would be best to configure the routing policy on R1 to only redistribute the routes that
originate from the RIP routers into the OSPF domain. For for the sake of simplicity, we are only concerned about the route
that represents the IP address of Host-2.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-57


Junos Troubleshooting in the NOC

Routing Loop Case Study (6 of S)


• Change the routing policy
• Exclude the prefix from routing policy
[edit]
user@Rl* show policy-options policy-statement ospf-out
from {
route-filter 172.16.17.0/24 exact reject;
route-filter 0.0.0.0/0 orlonger;

then
external
type l;

accept;

Changing the Routing Policy


The slide depicts the change to the routing policy on R1 that excludes the route that represents Host-2, but still allows all
other routes to be redistributed into OSPF.

Chapter 10-58 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Routing Loop Case Study (7 of 8)

• Preventing sub-optimal routing


• R1 still uses RIP-1 to forward the traffic to Host-2
• Changing the preference of OSPF external routes fixes the
problem
user@Rl> show configuration protocols ospf external-preference
external-preference 99;

user@Rl> show route 172.16.17/24

inet. 0: 21 dest:inatic,ns, 34 routes (21 active, 0 holddown, 0 hidden)


+ = Acti Ye Route, - = Last Activ·e, Both

172.16.17.0/24 *[OSPF/99] 00:02:43, metric O, tag O


> to 10.1.1.2 via ge-0/0/1.0
[RIP/100] 01:45:01, mecric 4, tag O
> to 10.1.2.2 via ge-0/0/2.D

One Problem Down


However, fixing the routing loop problem didn't fix the sub-optimal routing problem. The R1 router still forwards the traffic to
the RIP-1 router because the RIP version of the route has a better preference value than the OSPF version of the route. The
slide depicts changing the OSPF external preference value. This action causes Rl to prefer the OSPF version of the route and
fixes the sub-optimal routing problem.

www.juniper.net Troubleshooting Routing Prot ocols • Chapter 10-59


Junos Troubleshooting in the NOC

Routing Loop Case Study (8 of 8)

• Verifying the fix


user@Host-1> traceroute 172.16.17.2
traceroute to 172.16.17.2 (172.16.17.2), 30 hops max, 40 byte packets
1 172.16.16.1 (172.16.16.1) 21.973 ms 7.427 ms 6.006 ms�
2 10.1.1.2 (10.1.1.2) 5.995 ms 7.563 ms 7.970 ms�
3 10.1.7.2 (10.1.7.2) 12.048 ms 7.723 ms 12.001 ms�
4 172.16.17.2 (172.16.17.2) 9.778 ms 11.620 ms 9.984 ms

Verifying the Fix


As with any troubleshooting issue, you must verify the fix to ensure that the traffic is being forwarded properly. As the slide
depicts, the routing loop is gone and optimal routing for traffic sent between Host-1 and Host-2 is present.

Chapter 10-60 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Summary
• In this content, we:
• Explained how to troubleshoot OSPF
• Described how to troubleshoot BGP
• Demonstrated how to troubleshoot routing loops
• Discussed how to troubleshoot route oscillation

We Discussed:
How to troubleshoot OSPF;
How to troubleshoot BGP;
How to troubleshoot routing loops; and
How to troubleshoot route oscillation.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-61


Junos Troubleshooting in the NOC

Review Questions

1. How are the entries in the OSPF LSDB organized?


2. What are two tools that can help you troubleshoot
BGP adjacency issues?
3. What is a tool that you can use to discover the path
of a routing loop?

t c;.;,_4;unip... lletwml!s,lnc.AI;�- JUnl�r Worldwide Education Services WWW.JU11ij)efnel I 62


� �l 1t. I � ,.:L "' .....

Review Questions
1.

2.

3.

Chapter 10-62 • Troubleshooting Routing Protocols www.juniper.net


Junos Troubleshooting in the NOC

Troubleshooting Routing Protocols Lab

• Troubleshooting OSPF issues.


• Troubleshooting BGP issues.
• Troubleshoot route loop issues.

Troubleshooting Routing Protocols Lab


The slide provides the objectives for this lab.

www.juniper.net Troubleshooting Routing Protocols • Chapter 10-63


Junos Troubleshooting in the NOC

Answers to Review Questions


l.
The OSPF LSDB is organized by area and LSA types.
2.
Two tools that can help you troubleshoot BGP adjacency issues arc traceoptions and the monitor traffic interface command.
3.
Using the traceroute tool can help you discover the path of a routing loop.

Chapter 10-64 • Troubleshooting Routing Protocols www.juniper.net


JUnlf;v�[
Junos Troubleshooting in the NOC

Chapter 11: High Availability


Junos Troubleshooting in the NOC

Objectives
• After successfully completing this content, you will be
able to:
• Discuss graceful routing engine switchover
• Explain graceful restart
• Discuss nonstop active routing and bridging
• Explain unified in-service software upgrade

We Will Discuss:
Graceful Routing Engine switchover (GRES);
Graceful restart;
Nonstop active routing (NSR) and bridging; and
Unified in-service software upgrade (ISSU).

Chapter 11-2 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

Agenda: High Availability


�High Availability Overview
• Graceful Routing Engine Switchover
• Graceful Restart
• Nonstop Active Routing and Bridging
• Unified In-Service Software Upgrade

High Availability Overview


The slide lists the topics we will discuss. We discuss the highlighted topic first.

www.juniper.net High Availability • Chapter 11-3


Junos Troubleshooting in the NOC

High Availability Overview

• High demand for minimal down time


• Era of the five 9s
• Routing engine redundancy
• Provides hardware high availability
• Provides software high availability

Hard drive on RPO is thrashing


master RE failed, on master RE,
switching to switching to
backup RE backup RE

High Demand
The networks of yesteryear were not vital to the day-to-day workings of a business. During these times, network outages were
common. Small or large network downtimes were the norm. If a network outage were to occur, workers could simply continue
doing there jobs with a slight downgrade in their efficiency. Fast forward to present-day where every second of network
downtime can cost a business millions of dollars. Welcome to the era of five 9s, or 99.999% system uptime. Many providers
and enterprises have service-level agreements (SLAs) that require them provide 99.999% system uptime for their
customers, any less results in a breach of contract and lost money and time.

Routing Engine Redundancy


To provide solutions in the era of the five 9s, the Junos OS provides methods of high availability (HA) through features that
work in conjunction with dual routing engines in a routing system. These HA features, which we will discuss in great detail
later in this chapter, provide hardware and software HA.

Chapter 11-4 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

Agenda: High Availability


• High Availability Overview
�Graceful Routing Engine Switchover
• Graceful Restart
• Nonstop Active Routing and Bridging
• Unified In-Service Software Upgrade

Graceful Routing Engine Switchover


The slide highlights the topic we discuss next.

www.juniper.net High Availability • Chapter 11-5


Junos Troubleshooting in the NOC

Graceful Routing Engine Overview

• What is GRES?
• Dual routing engines required
• Interface and kernel information preserved
• Preservation of the forwarding plane
• Control plane is not preserved
• Main components synchronization, switchover, recovery
Hard drive on Detected EBGP
master RE failed, session failure.
switching to removing BGP
backup RE routes

;,: :::i

GRES Defined
The GRES feature in the Junos OS enables a routing platform with redundant routing engines to continue forwarding packets,
even if one routing engine fails. GRES preserves interface and kernel information. Traffic is not interrupted, however, GRES
does not preserve the control plane. Neighboring routers detect that the router has experienced a restart event and react to
the event in a manner prescribed by individual routing protocol specifications. To preserve routing during a switchover, GRES
must be combined with either graceful restart protocol extensions or NSR. Any updates to the master routing engine are
replicated to the backup routing engine as soon as they occur. If the kernel on the master routing engine stops operating, the
master routing engine experiences a hardware failure, or the administrator initiates a manual switchover, mastership
switches to the backup routing engine.
If the backup routing engine does not receive a keepalive from the master routing engine after 2 seconds, it determines that
the master routing engine has failed and takes mastership. The Packet Forwarding Engine (PFE) seamlessly disconnects
from the old master routing engine and reconnects to the new master routing engine. The PFE does not reboot, and traffic is
not interrupted. The new master routing engine and the PFE then become synchronized. If the new master routing engine
detects that the PFE state is not up to date, it resends state update messages.
Successive routing engine switchover events must be a minimum of 240 seconds (4 minutes) apart after both routing
engines have come up. If the router displays a warning message similar to Standby Routing Engine is not ready
for graceful switchover . Packet Forwarding Engines that are not ready for graceful
switchover might be reset, do not attempt the switchover. If you choose to proceed with the switchover, the PFEs
that were not ready for graceful switchover are reset. We recommend that you wait until the warning no longer appears and
then proceed with the switchover.

Chapter 11-6 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

GRES Synchronization
Routing Engine O (Master) Routing Engine 1 (Backup)

rpd and other


processes

cl1assisd
® ---- cl1assisd

ksyncd

w ...................... ........................ .
Kernel f'i"

Packet Forwarding Engine

GRES Synchronization
The switchover preparation process for GRES follows these steps:
1. The master routing engine starts.
2. The routing platform processes, such as the chassis process (chassisd), start.
3. The PFE starts and connects to the master routing engine.
4. All state information is updated in the system.
5. The backup routing engine starts.
6. The system determines whether GRES has been enabled.
7. The kernel synchronization process (ksyncd) synchronizes the backup routing engine with the master routing
engine.
8. After ksyncd completes the synchronization, all state information and the forwarding table are updated.

www.juniper.net High Availability • Chapter 11- 7


Junos Troubleshooting in the NOC

GRES Switchover Process

Routing Engine O (Old Master) Routing Engine 1 (New Master)

rpd and oth�.


processes

rpd and otl1er processes


Kernel

Helper chassisd
Router
4

Kernel

Packet Forwarding Engine

GRES Switchover Process


When a routing engine switchover occurs, the switchover process follows these steps:
1. When keepalives from the master routing engine are lost, the system switches over gracefully to the backup
routing engine.
2. The PFE connects to the backup routing engine, which becomes the new master.
3. Routing platform processes that are not part of GRES, such as the routing protocol process (rpd), restart.
4. State information learned from the point of the switchover is updated in the system.
5. If configured, graceful restart protocol extensions collect and restore routing information from neighboring peer
helper routers. Alternatively, you can configure NSR to handle the HA of the control plane.

Chapter 11-8 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

Routing Engine Switchover Results

• Routing engine switchover scenarios


• Dual routing engines - No other HA features
• Control plane and forwarding plane are not preserved
•GRES
• Forwarding plane preserved. control plane reset
•GRES and GR
• Control plane and forwarding plane are preserved
• Requires helper routers that must support GR
•GRES and NSR
• Control plane and forwarding plane are preserved
• No helper routers required

Routing Engine Switchover Results


The slide describes the effects of a routing engine switchover when no HA features are enabled and when GRES, graceful
restart, and NSR features are enabled.
Dual routing engines (no other HA features): When the switchover to the new master routing engine is complete,
routing convergence takes place and traffic is resumed. All physical interfaces are taken offline, PFEs restart,
the backup routing engine restarts the rpd process, and all hardware and interfaces are discovered by the new
master routing engine. The switchover takes several minutes and all of the router's adjacencies are aware of
the hardware and routing change.
GRES only: During the switchover, interface and kernel information is preserved. The switchover is faster
because the PFEs are not restarted. The new master routing engine restarts the rpd process. All hardware and
interfaces are acquired by a process that is similar to a warm restart. All adjacencies are aware of the router's
change in state.
GRES and GR: Traffic is not interrupted during the switchover. Interface and kernel information is preserved.
Graceful restart protocol extensions quickly collect and restore routing information from the neighboring
routers. Neighbors are required to support graceful restart and a wait interval is required. The rpd process
restarts. For certain protocols, a significant change in the network can cause graceful restart to stop.
GRES and NSR: Traffic is not interrupted during the switchover. Interface, kernel, and routing protocol
information is preserved. Unsupported protocols must be refreshed using the normal recovery mechanisms
inherent in each protocol.
Continued on the next page.

www.juniper.net High Availability • Chapter 11-9


Junos Troubleshooting in the NOC

Routing Engine Switchover Results (contd.)


When given the choice, we recommend that you always deploy NSR to provide control plane HA, and only employ graceful
restart when you have to. Because graceful restart relies heavily on outside routers for support, you are less likely to have a
hitless routing engine switchover. NSR is self-contained and does not rely on external routers to provide control plane HA,
which results in better routing engine switchover scenarios. Note that you cannot configure NSR and graceful restart at the
same time on a single routing platform.

Chapter 11-10 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

Configuring GRES
• Configuring GRES
• Configure chassis redundancy

[edit]
user@routeri set chassis redundancy graceful-switchover

• Configure commit synchronize


• Not strictly required. but highly recommended

[edit]
user@router# commit
warning: graceful-switchover is enabled, commit synchronize should be used
commit complete

{master} [edit]
user@routert set system commit synchronize

Configuring GRES
Configuring GRES is a straight forward process in which you simply enable the graceful-switchover option under the
[ edit chassis redundancy] hierarchy level. Once you apply the GRES configuration, notice that the banner changes
to tell you which routing engine you are currently logged in to. The slide depicts that banner for the master routing engine. If
you log into the command-line interface (CU) of the backup routing engine, the banner displays the output of {backup) to
alert you that you are in the backup routing engine. Note that you can log into the backup routing engine from the master
routing engine by using the request routing-engine login other-routing-engine operational command.
Although it is not strictly required, it is highly recommended that you use the commit synchronize command, as noted
by the CU warning, when enabling GRES. Using the commit synchronize command ensures that both routing engines
have the same configuration, which is very important during a routing engine switchover. The slide depicts the use of the
set system commit synchronize command. This command effectively turns the commit command into a commit
synchronize command by synchronizing the configuration between both routing engines when you issue the commit
command.
Note that when you configure GRES, you can bring the backup routing engine online after the master routing engine is
already running. There is no requirement to start the two routing engines simultaneously

www.juniper.net High Availability • Chapter 11-11


Junos Troubleshooting in the NOC

Verifying GRES

• How to verify GRES is functioning properly


• Check GRES from the backup RE

{backup)
user@router> show system switchover
Graceful switchover: On
Configuration database: Ready
Kernel database: Ready
Peer state: steady state

• show system switchover command not supported


from master RE

Verifying GRES
To verify whether GRES is enabled and ready, issue the show system switchover command on the backup routing
engine-this command is not supported on the master routing engine. When the output of the command indicates that the
Graceful switchover field is set to on, GRES is operational. The status of the kernel database and configuration
database synchronization between routing engines is also provided.

Chapter 11-12 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

Agenda: High Availability


• High Availability Overview
• Graceful Routing Engine Switchover
�Graceful Restart
• Nonstop Active Routing and Bridging
• Unified In-Service Software Upgrade

Graceful Restart
The slide highlights the topic we discuss next.

www.juniper.net High Availability • Chapter 11-13


Junos Troubleshooting in the NOC

Graceful Restart Overview

• What is graceful restart?


• Uninterrupted packet forwarding
• Temporary routing protocol update suppression
• Support for:
• Routing protocols
• MPLS related protocols
• Layer 2 and Layer 3 VPNs
• Two main components - Helper router and restarting router
• Restarting router - Requires rapid restoration of forwarding state
• Helper router - Assists restarting router
• Great HA solution for routers with only one routing engine

What Is Graceful Restart?


With routing protocols, any service interruption requires that an affected router recalculate adjacencies with neighboring
routers, restore routing table entries, and update other protocol-specific information. An unprotected restart of a router can
result in forwarding delays, route flapping, wait times stemming from protocol reconvergence, and even dropped packets.
The main benefits of graceful restart are uninterrupted packet forwarding and temporary suppression of all routing protocol
updates. Graceful restart enables a router to pass through intermediate convergence states that are hidden from the rest of
the network.
Three main types of graceful restart are available.
Graceful restart for aggregate and static routes and for routing protocols-Provides protection for aggregate and
static routes and for BGP, End System-to-Intermediate System (ES-IS), Intermediate System-to-Intermediate
System (IS-IS), OSPF, RIP, Routing Information Protocol next generation (RIPng), and Protocol Independent
Multicast (PIM) sparse mode routing protocols.
Graceful restart for MPLS-related protocols-Provides protection for LDP, RSVP, circuit cross-connect (CCC), and
translational cross-connect (TCC).
Graceful restart for virtual private networks (VPNs)-Provides protection for Layer 2 and Layer 3 VPNs.
Continued on the next page.

Chapter 11-14 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

What Is Graceful Restart? (contd.)


Graceful restart works similarly for routing protocols and MPLS protocols and combines components of these protocol types
to enable graceful restart in VPNs.
Most graceful restart implementations define two types of routers-the restarting router and the helper router. The restarting
router requires rapid restoration of forwarding state information so it can resume the forwarding of network traffic. The
helper router assists the restarting router in this process. Graceful restart configuration statements typically affect either the
restarting router or the helper router.
Graceful Restart not only helps provide HA in routing platforms with dual routing engines, but you can also use it as a method
to provide HA for routing platforms with only a single routing engine. For example, many of the branch routing platforms only
have one routing engine. If a disruptive event, such as the rpd process flapping, occurs, you can employ graceful restart to
minimize the impact to the control plane of the router that was struck by the disruptive event. Minimizing the disruptive
effect to the control plane in this manner also minimizes the negative effects that can ripple through the rest of the network.

www.juniper.net High Availability • Chapter 11-15


Junos Troubleshooting in the NOC

Enabling Graceful Restart


• Turning on graceful restart
[edit]
user@router# set routing-options graceful-restart

• Must disable per protocol

[edit]
user@router# set protocols ospf graceful-restart disable

[edit]
user@router# set protocols 1dp graceful-restart disable

Enabling Graceful Restart


Graceful restart is disabled by default. You must configure the graceful.-restart option at the
[edit routing-options J hierarchy level to enable the feature. To enable graceful restart for protocols contained in a
routing instance, you must enable graceful restart under the [edit routing-instance instance-name
routing-options J hierarchy level.
Once you enable graceful restart, it is enabled for every protocol that you configure for that routing instance-which is
typically the main routing instance. If you wish to configure graceful restart for some protocols and disable it for others, you
must manually disable it for each protocol that you do not want graceful restart applied to. The slide depicts enabling
graceful restart for the main routing instance, then disabling it for the LDP and OSPF protocols.
Note that if you configure graceful restart after a BGP or LDP session has been established, the BGP or LDP session restarts
and the peers negotiate graceful restart capabilities.

Chapter 11-16 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

Agenda: High Availability

• High Availability Overview


• Graceful Routing Engine Switchover
II
Graceful Restart
7Nonstop Active Routing and Bridging
• Unified In-Service Software Upgrade

Nonstop Active Routing and Bridging


The slide highlights the topic we discuss next.

www.juniper.net High Availability • Chapter 11-17


Junos Troubleshooting in the NOC

Nonstop Active Routing

• Nonstop active routing concepts


• Same infrastructure as GRES
• Preserves interface and kernel information
• Saves routing information on backup RE
• rpd process is active on backup RE
• Self contained
• Does not need external "helper" routers
• Must first enable GRES
• Both routing engines must have the same version of
Junos OS

Nonstop Active Routing Concepts


NSR uses the same infrastructure as GRES to preserve interface and kernel information. However, NSR also saves routing
protocol information by running the rpd process on the backup routing engine. By saving this additional information, NSR is
self-contained and does not rely on helper routers to assist the routing platform in restoring routing protocol information.
NSR is advantageous in networks where neighbor routers do not support graceful restart protocol extensions. As a result of
this enhanced functionality, NSR is a natural replacement for graceful restart.
Note that to use NSR, you must first enable GRES on your routing platform and both routing engines must be on the same
Junos OS version.

Chapter 11-18 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

NSR During a Switchover


Routing Engine O (Old Master) Routing Engine 1 (New Master)
rpd and ott1er

I I
processes

cl1assisd -
" 1 - rpd
Kernel

cl1assisd

@
External .� 3
Router
" Kernel

'I'

<p cp cp ·�
I

ii ··-··
Packet Forwarding Engine
It-'<'. t> - ' '. • ,0, •
2<1«��.!nc,AllrigMs� 1'* '. JUnm WorldwldeEducatlonServices

www-1uniper..net I 19
2���JE±l"�1')ffjt·"i--::---v?/-"- _ w,m,;,=�= �=·�

NSR During a Switchover


When NSR is enabled and a routing engine switchover event occurs, such as hardware failure of the master routing engine.
the switchover process follows these steps:
1. When keepalives from the master routing engine are lost, the system switches over gracefully to the backup
routing engine. This process is due to GRES already being enabled.
2. The PFE connects to the backup routing engine, which becomes the new master. Because the rpd process and
the chassisd process are already running on the new master routing engine, these processes do not need to
restart.
3. State information learned from the point of the switchover is updated in the system. Forwarding and routing are
continued during the switchover, resulting in minimal packet loss.
4. Peer routers continue to interact with the routing platform as if no change had occurred. Routing adjacencies
and session state relying on underlying routing information are preserved and not reset.

www.juniper.net High Availability • Chapter 11-19


Junes Troubleshooting in the NOC

Configuring NSR

• Configuring NSR
• Enable GRES
[edit]
user@router# set chassis redundancy gracefu1-switchover

• Enable NSR
[edit]
user@router# set routing-options nonstop-routing

• Enable commit synchronization


[edit]
user@router# set system commit synchronize

.. ·;,; . .-
',C2!>14Juniper!f�·hlc.AJl"..u,,ts�rved.
� . , Jl:Jnm Worldwide Education Services www;umpernel I 20
f...,.t.,.,� • N,.," ��;i,, t�� ��£,,.,;!--',.' ,,,,z � � � �- �'- � _

Configuring NSR
Configuring NSR is simple and straight forward. First, you must configure GRES, as we discussed earlier. Next, you must
configure NSR by enabling the nonstop-routing option under the [edit routing-options J hierarchy level.
Finally, it is a requirement that you configure the Junos OS to synchronize the configuration whenever you activate the
configuration with the commit synchronize command under the [edit system] hierarchy level.
If you try to commit the NSR configuration without including the commit synchronize statement, the commit fails.
However, if the backup routing engine is down when you issue the commit, the Junos OS displays a warning and commits
the candidate configuration in the master routing engine. When the backup routing engine comes up, its configuration will
automatically be synchronized with the master routing engine.
Although it is not shown on the slide, you can enable the routing platform to switch over to the backup routing engine when
the rpd process fails rapidly three times in succession by including the other-routing-engine statement at the [edit
system processes routing failover] hierarchy level.

Chapter 11-20 • High Availability www.juniper.net


Ju nos Troubleshooting in the NOC

Verifying NSR

• Verifying NSR
• Master routing engine
{master}
user@rcuter> show task replication
Stateful Replica�ion: Enabled!+-------�
RE mode: Master

I
Protocol Synchronization Status
Configured protocol
CSPF I
NSR states
Complete
RIP Complete ,..--------ii
BGP Complete
IS-IS Complete
LDP Compiete

• Backup routing engine


{backup)
user@router> show task replication
Stateful Replication: Enabled
RE mode: Backup

Verifying NSR
You can verify NSR operation by issuing the show task replication command on the master and backup routing
engines. Note that the command shows the current state of GRES and the NSR state of the protocols that are configured on
the router. The Synchronization Status field can display the state of NotStarted, InProgress, and Complete
for any configured protocol. If any protocol is not in the Complete state when a switchover event occurs, network topology
and traffic forwarding disruptions occur for the protocols that are not in the Complete state.
You can examine the replication state of BGP in more detail by issuing the show bgp replication command. The
following output displays an example of the command for the master routing engine.
user@router> show bgp replication

Synchronization master:
Session state: Up, Since: 44:07
Flaps: 0
Protocol state: Idle, Since: 14
Synchronization state: Complete
Number of peers waiting: AckWait: 0, SoWait: 0, Scheduled: 0
Messages sent: Open 1, Establish 924, Update 381, Error 60, Complete 114
Messages received: Open 1, Request 1 wildcard 113 targeted, EstablishAck 924,
CompleteAck 114

www.juniper.net High Availability • Chapter 11-21


Junos Troubleshooting in the NOC

NSRand BFD

• NSR support for BFD


• Routing engine based BFD session failure time must be
greater than routing engine switchover time
• Multihop sessions
• Tunnel encapsulated sessions
• Sessions over IRB interfaces
• Minimum interval of greater than 2500 ms for routing
engine based BFD sessions
• Minimum interval of greater than 10 ms for distributed BFD
sessions

NSR Support for BFD


NSR supports the Bidirectional Forwarding Detection (BFD) protocol, which uses the topology discovered by routing protocols
to monitor neighbors. The BFD protocol is a simple hello mechanism that detects failures in a network. Because BFD is
streamlined to be efficient at fast liveness detection, when it is used in conjunction with routing protocols, routing recovery
times are improved. With NSR enabled, BFD session states are not restarted when a routing engine switchover occurs.
When a BFD session is distributed to the PFE, BFD packets continue to be sent during a routing engine switchover. If
nondistributed, or routing engine based, BFD sessions are to be kept alive during a switchover, you must ensure that the
session failure detection time is greater than the routing engine switchover time, which in most situations is 2 seconds. The
following BFD sessions are not distributed to the PFE: multihop sessions, tunnel-encapsulated sessions, and sessions over
integrated routing and bridging (IRB) interfaces. An example of a multihop session is a standard IBGP peering or a multihop
EBGP peering.
For BFD sessions to remain up during a routing engine switchover event when NSR is configured, specify a minimum interval
of 2500 ms or greater for routing engine based sessions. For distributed BFD sessions with NSR configured, we recommend
that you set the a minimum interval of 10 ms or greater.

Chapter 11-22 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

NSRand BGP

• NSR support for BG P


• Address families supported in main routing instance
• Only family unicast is supported in VRF instances
• Include the path-selection external-router-ID
statement at the [edit protocols bgp] hierarchy
level
• Ensures consistent path selection between master and backup
routing engines
• BGP route damping does not work on backup routing engine

NSR Support for BGP


NSR BGP support is subject to the following conditions:
Because of BGP's nature to use the version of a BGP route that arrives first over the version of the same BGP
route that arrives only seconds later, you must include the path-selection external-router-ID
statement at the [edit protocols bgp] hierarchy level to ensure consistent path selection between the
master and backup routing engines during and after the NSR switchover.
If the BGP peer in the master routing engine has negotiated address-family capabilities that are not supported
for NSR, then the corresponding BGP neighbor state on the backup routing engine shows as idle. On switchover,
the BGP session is reestablished from the new master routing engine. Effectively, NSR is disabled for the BGP
session that uses unsupported address families.
BGP route dampening does not work on the backup routing engine when NSR is enabled.
Continued on the next page.

www.juniper.net High Availability • Chapter 11-23


Junos Troubleshooting in the NOC

NSR Support for BGP (contd.)


The following address families are supported for NSR and BGP.
inet unicast
inet labeled-unicast
inet multicast
inet6 labeled-unicast
inet6 multicast
inet6 unicast
route-target
12vpn signaling
inet6-vpn unicast
inet-vpn unicast
inet-mdt
iso-vpn

Chapter 11-24 • High Availability www.juniper.net


Junes Troubleshooting in the NOC

NSR and RSVP-TE LSPs


• NSR support for RSVP-TE LSPs
• Support for P2P and P2MP LSPs
• Ingress, transit. and egress
• Unsupported RSVP-TE LSP features
• Dynamic P2MP LSPs
• GMPLS
• lnterdomain or loose-hop expansion LSPs
• BFD liveness detection

NSR Support for RSVP-TE LSPs


Junes OS extends NSR support to label-switching routers (LS Rs) and Layer 2 Circuits that are part of an RSVP-TE
label-switched path (LSP). NSR support on LSRs ensures that routing engine switchover on an LSR remains transparent to
the network neighbors and that the LSP information remains unaltered during and after the switchover.
You can use the show rsvp version command to view the NSR mode and state on an LSR. Similarly, you can use the
show mpls lsp and show rsvp session commands on the backup routing engine to view the state recreated on the
backup routing engine.
NSR currently supports RSVP-TE point-to-point (P2P) and point-to-multipoint (P2MP) ingress, transit, and egress LSPs. During
the switchover, the LSP comes up on the backup routing engine that shares and synchronizes the state information with the
master routing engine before and after the switchover. NSR support for RSVP-TE LSPs ensures that the switchover remains
transparent to the network neighbors, and preserves the LSP information across the switchover.
The Junes OS does not support NSR for the following RSVP-TE LSP features:
Dynamic P2MP LSPs;
Generalized MPLS (GMPLS) and LSP hierarchy;
lnterdomain or loose-hop expansion LSPs; and
BFD liveness detection.

www.juniper.net High Availability • Chapter 11-25


Junos Troubleshooting in the NOC

Nonstop Bridging

• Nonstop bridging for MX 30 devices


• Same infrastructure as GRES - GRES must be enabled
• Preserves interface and kernel information
• Saves L2CP (12cpd) information on backup RE
• Self contained
• Does not need external devices
• Both routing engines must have the same revision of
Junos OS
• Support for STP, RSTP, and MSTP control protocols
• Verify by issuing L2CP commands on backup routing engine

Nonstop Bridging Concepts


Nonstop bridging (NSB) uses the same infrastructure as GRES to preserve interface and kernel information. This reliance on
GRES requires that GRES is enabled before you can enable NSB. NSB saves Layer 2 Control Protocol (L2CP) information by
running the Layer 2 Control Protocol process (12cpd) on the backup routing engine. By running the 12cpd process on the
backup routing engine, the router is able to maintain the state of the Spanning Tree Protocol (STP), Rapid Spanning Tree
Protocol (RSTP), and Multiple Spanning Tree Protocol (MSTP) control protocols through a routing engine switchover without
the help of an external device.
When you enable NSB, you can issue L2CP related operational mode commands on the backup routing engine to verify NSB.
However, the output of the commands might not match the output of the same commands issued on the master routing
engine.
Note that both routing engines must be running the same Junos OS release.

Chapter 11-26 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

NSB During a Switchover


Routing Engine O (Old Master) Routing Engine 1 (New Master)
Layer 2 control protocol process
(12cpd) and other processes

cl1assisd
-- 1 - Layer 2 control protocol process
(12cpd)

cl1assisd

" 3

I Kernel
I Kernel

cp � '
Packet Forwarding Engine
ir;::::�; k-�;��::::���
JUn!Ee[ Worldwide Education Services www,iurriper.net I 27
t-'::iill!ixi:!i!i�:\zlt,&li&\;;a«R, •;; =s!L� -""- �

NSB During a Switchover


When NSB is enabled and a switchover event occurs, such as hardware failure of the master routing engine. the switchover
process follows these steps:
1. When keepalives from the master routing engine are lost, the system switches over gracefully to the backup
routing engine.
2. The PFE connects to the backup routing engine, which becomes the new master. Because the 12cpd process
and chassisd process are already running on the new master routing engine, these processes do not need to
restart.
3. State information learned from the point of the switchover is updated in the system. Forwarding and bridging
are continued during the switchover, resulting in minimal packet loss.

www.juniper.net High Availability • Chapter 11-27


Junos Troubleshooting in the NOC

Configuring NSB

• Configuring NSB
• Enable GRES
[edit]
user@router# set chassis redundancy gracefu1-switchover

• Enable NSB
[edit]
user@router# set protocois 1ayer2-controi nonstop-bridging

• Enable commit synchronization


[edit]
user@router# set system collllllit synchronize

Configuring NSB
Configuring NSB is simple and straight forward. First, you must configure GRES, as we discussed earlier. Next, you must
configure NSB by enabling the nonstop-bridging option under the [edit protocols layer2-control]
hierarchy level. Finally, it is a requirement that you configure the Junos OS to synchronize the configuration whenever you
activate the configuration with the commit synchronize command under the [edit system] hierarchy level.
If you try to commit the NSB configuration without including the commit synchronize statement, the commit fails.
However, if the backup routing engine is down when you issue the commit, the Junos OS displays a warning and commits
the candidate configuration on the master routing engine. When the backup routing engine comes up, its configuration will
automatically be synchronized with the master routing engine.

Note that a newly inserted backup routing engine automatically synchronizes its configuration with the master routing engine
configuration.When you configure NSB, you can bring the backup routing engine online after the master routing engine is
already running. There is no requirement to start the two routing engines simultaneously.
Note that if you are using an EX Series Ethernet switch, you enable NSB by configuring the nonstop-bridging option
under the [edit ethernet-switching-options] hierarchy level.

Chapter 11-28 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

Agenda: High Availability

• High Availability Overview


• Graceful Routing Engine Switchover
• Graceful Restart
• Nonstop Active Routing and Bridging
7Unified In-Service Software Upgrade

.. I " .
JUnJJ;ef
�M'<-·-_>c -'-- - - - _..., -
• �20.1�'h<nlpo,Networio>,"!';�'�-- . Worldwide Education Services www;un;pe,.net 1 29
-�k�d,..( ""- _,; A ,�= � -

Unified In-Service Software Upgrade


The slide highlights the topic we discuss next.

www.juniper.net High Availability • Chapter 11-29


Junos Troubleshooting in the NOC

Unified ISSU Overview


• What is unified ISSU?
• Upgrade routing engines with no control plane disruption
and minimal forwarding plane disruption
• Must enable GRES and NSR
• Enable NSB if Layer 2 control protocols are in use
• Primary and backup REs must be running the same versions
of the Junos OS
• Confirm compatibility of software, configuration, and
hardware
• request system software validate
in-service-upgrade

Unified ISSU Overview


A unified ISSU enables you to upgrade between two different Junos OS Releases with no disruption on the control plane and
with minimal disruption of traffic forwarding. Unified ISSU is only supported on dual routing engine platforms. GRES must be
enabled along with a combination of NSR and NSB, depending on the Layer 3 and Layer 2 protocols in use. Unified ISSU
allows you to eliminate network downtime during software upgrades, reduces operating costs while delivering higher service
levels, and allows you to implement new features at a faster pace.
Before you can begin the unified ISSU process. you must ensure that both routing engines are on the same Junos OS
version. Note that during the unified ISSU process, you cannot take any PIC online or offline.
You can verify the unified ISSU-compatibility of the software, hardware, and the configuration on a device by issuing the
request system software validate in-service-upgrade command. This command runs the validation
checks, and shows whether the operating system, device components, and configuration are unified ISSU compatible.

Chapter 11-30 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

Unified ISSU Process (1 of 5)

PFEO
Previous
Master RE Junos OS version
(REO) New
Junos OS version
PFE1

Backup RE
(RE1)
PFE2

Master RE checks for disk space on both REs,


unsupported configurations. and unsuppor ted PICs
ksyncd synchronizes the kernel on the backup RE
with the master RE

Unified ISSU Process: Part 1


The following notes detail steps 1 and 2 of the unified ISSU process.
1. The master routing engine checks for disk space available for the /var file system on both routing engines,
unsupported configurations, and for unsupported PlCs. If there is not sufficient disk space available on either of
the routing engines, the unified ISSU process fails and returns an error message saying that the routing engine
does not have enough disk space available. However, unsupported PICs do not prevent a unified ISSU. The
software issues a warning to indicate that these PICs will restart during the upgrade. Similarly, an unsupported
protocol configuration does not prevent a unified ISSU. The software issues a warning that packet loss may
occur for the protocol during the upgrade.
2. When the validation succeeds, the ksyncd process synchronizes the kernel on the backup routing engine with
the master routing engine.

www.juniper.net High Availability • Chapter 11-31


Junos Troubleshooting in the NOC

Unified ISSU Process (2 of 5)

PFEO
Previous
Junos OS version
New
Junos OS version
PFEl
,.-. - - .....,
I Backup RE I
E
L_� �_J PFE2

Backup RE is upgraded with the new software and


® resynchronizes with the master RE
When software processes are ready for unified ISSU,
master RE sends ISSU_PREPARE messages to FPCs

Unified ISSU Process: Part 2


The following notes detail steps 3 and 4 of the unified ISSU process.
3. The backup routing engine is upgraded with the new software image. Before being upgraded, the backup
routing engine gets the configuration file from the master routing engine and validates the configuration to
ensure that it can be committed using the new software version. After being upgraded, it is resynchronized with
the master routing engine.
4. The chassisd process on the master routing engine prepares other software processes for the unified ISSU.
When all the processes are ready, the chassisd process sends an ISSU_PREPARE message to the FPCs
installed in the router.

Chapter 11-32 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

Unified ISSU Process (3 of 5)


·
· · ··
·········· ·· · ·· P F EO
..-- --,,1,:· ········ ..___ _ _ __.., Previous
----
Master RE Junos OS version
(REO) New
Junos OS version
PFE1

I Backup RE
_�E1)
L _J PFE2

® Every PFE on each FPC downloads new software from backup RE


@ Each PFE sends an ISSU_READY messages to chassisd
on the master RE

Unified ISSU Process: Part 3


The following notes detail steps 5 and 6 of the unified ISSU process.
5. The PFE on each FPC saves its state and downloads the new software image from the backup routing engine.
6. Each PFE sends an ISSU_READY message to chassisd process on the master routing engine.

www.juniper.net High Availability • Chapter 11-33


Junos Troubleshooting in the NOC

Unified ISSU Process (4 of 5)

PFEO I
L ____ J Previous
Junos OS version

New
Junos OS version

I Backup RE
E
I
L_� �_J PFE2 I
L ____ J
h\ chassisd sends an ISSU_REBOOT message to the FPCs;
0 the FPCs reboot with the new Junos OS image
® 8
After the FPCs reboot, the FPCs send a READY message to the
chassisd process on the master RE; Other processes prepare
for RE switchover
..C2014J"!!lp... �.lnc.AllrialdS"'50fflld..
"'�.J,;t;"' £�· t '� ...
Junm WorldwideEducationServlces
__;,==-----�· � �
WWW.jUll�net 134
I

Unified ISSU Process: Part 4


The following notes detail steps 7 and 8 of the unified ISSU process.
7. After receiving an ISSU_READY message from a PFE, the chassisd process sends an ISSU_REBOOT message to
the FPC on which the PFE resides. The FPC reboots with the new software image. After the FPC is rebooted, the
PFE restores the FPC state and a high-speed internal link is established with the backup routing engine running
the new software. The chassisd process is also reestablished with the master routing engine.
8. After all PFEs have sent a READY message using the chassisd process on the master routing engine, other
software processes are prepared for a routing engine switchover. The system is ready for a routing engine
switchover at this point.

Chapter 11-34 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

U11ified ISSU Process (5 of 5)


,---·-----,
I PFEO I
l ____ J Previous
Backup RE j Junos OS version
(REO)
I ,...------, New

I PFE1 I Junos OS version

l ____ J
,--- ----.
I Master RE
I ,-------,
l_�E�_J I PFE2 I
l ____J

® RE switchover occurs, RE1 is the new master RE


@ REO, the new backup RE, is upgraded
@ Unified ISSU is now complete

Unified ISSU Process: Part 5


The following notes detail steps 9, 10, and 11 of the unified ISSU process.
9. The routing engine switchover occurs and the backup routing engine becomes the new master routing engine.
10. The new backup routing engine is now upgraded to the new software image. (This step is skipped if the
no-old-master-upgrade option is specified.)
11. When the backup routing engine has been successfully upgraded, the unified ISSU is complete.
Note that in the example on the slide, after the unified ISSU process completes RE1 is still the master routing engine. For
REO to become the master routing engine, you must manually switch mastership back to REO. However, we recommend that
you wait until GRES and NSR are completely ready before attempting the mastership switchover, or traffic disruption might
ensue.

www.juniper.net High Availability • Chapter 11-35


Junos Troubleshooting in the NOC

Summary
• In this content, we:
• Discussed graceful routing engine switchover
• Explained graceful restart
• Discussed nonstop active routing and bridging
• Explained unified in-service software upgrade

We Discussed:
GRES;
Graceful restart;
NSR and bridging; and

Unified ISSU.

Chapter 11-36 • High Availability www.juniper.net


Junos Troubleshooting in the NOC

Review Questions

1. What are the three main components of GRES?


2. Which features can you combine GRES with to
provide control plane high availability?
3. What happens if you perform a unified ISSU and an
unsupported PIC is present in a router?

Review Questions
1.

2.

3.

www.juniper.net High Availability • Chapter 11-37


Junos Troubleshooting in the NOC

Answers to Review Questions


1.
'fhc three main components of GRES are synchronization, switchover, and recovery.
2.
You can combine GRES with NSR or graceful restart to provide control plane HA. You can combine GRES and NSB if Layer 2
control protocols are in use.
3.
If you perform a unified ISSU and an unsupported PIC is present in the router, the unsupported PIC is taken offline and the unified
ISSU proceeds.

Chapter 11-38 • High Availability www.juniper.net


JUntf2v�f
Junos Troubleshooting in the NOC

Chapter 12: Network Monitoring


Junes Troubleshooting in the NOC

Objectives

• After successfully completing this content, you will be


able to:
• Explain how to configure and monitor SNMP
• Discuss how to configure and monitor RMON
• Understand how to use flow monitoring

We Will Discuss:
How to configure and monitor Simple Network Management Protocol (SNMP);
How to configure and monitor Remote Monitoring (RMON); and
How to use flow monitoring.

Chapter 12-2 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Agenda: Network Monitoring


�SNMP
•RMON
• Flow Monitoring

SNMP
The slide lists the topics we will discuss. We discuss the highlighted topic first.

www.juniper.net Network Monitoring • Chapter 12-3


Junos Troubleshooting in the NOC

SNMP Overview
• What is SNMP?
• Application Layer protocol
• Designed to monitor and manage TCP/IP network devices
• Set of standards for network management
• Protocol
• Database structure specification
• Data Objects
• Monitors various parameters of managed devices
• CPU utilization
• Memory utilization
• Temperature
• Many other items

What is SNMP?
SNMP is an Application Layer protocol that is designed to monitor and manage TCP/IP network devices. SNMP manages
network devices through a set of protocols and data objects, which we discuss in detail in later slides. SNMP also monitors
managed devices for parameters such as CPU utilization, memory utilization, component temperature, and many other
items.

Chapter 12-4 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

SNMP Architecture

• Consists of three key components


• Managed device
• SNMP agent
• SNMP software that runs on the managed device

-..
•NMS
Managed Device
• Contains SNMP manager software
••••••••••••••
NMS SNMPAgent

SNMP Architecture
The SNMP agent. which runs on the managed device, exchanges network management information with the SNMP manager
software running on a network management system (NMS), or host. The agent responds to requests for information and
actions from the manager. The agent also controls access to the agent's MIB, which is the collection of objects that can be
viewed or changed by the SNMP manager.

www.juniper.net Network Monitoring • Chapter 12-5


Junos Troubleshooting in the NOC

Managed Devices
• What is a managed device?
• Network node that resides on a managed network
• Routers, firewaiis, switches, servers, and much more
• Contains SNMP Agent
• Collects and stores management information

Routers

&---· --
Firewalls Switches servers
;......:...... ...... ......ij.J
" ·: .. ::::::::::=;
.... :
;......:...... .'."""""6

"' r �� r " ;;; ; ' '

'C:1014J'.""p'.';,�rn?M�reseM!ld.. JLJnm WorldwideEducationServlces www1un,pernet I 6


LA-- ....iA....JL.'li: - ... , i>---.iF""i� -�-!- ' -- -

Managed Devices
A managed device is a network node that contains an SNMP agent and resides on a managed network. Managed devices
collect and store management information and make this information available to NMS devices using SNMP. Managed
devices, which are sometimes called network elements, can be routers, access servers, switches, computer hosts, printers,
and many other network elements.

Chapter 12-6 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

SNMPAgent

• What is the SNMP Agent?


• Resides on a managed device
• Translates management information into an SNMP
compatible format

-..
• Communicates with the NMS
Managed Device

..............
SNMPAgent

SNMPAgent
An SNMP agent is a network management software module that resides on a managed device. The agent has local
knowledge of management information and translates that information into a form that is compatible with SNMP. In the
Junos OS, the SNMP agent is the SNMP process (snmpd), which runs on the routing engine. It is responsible for collecting
MIB information from various sub-agents and responding to requests from NMS devices. SNMPD is also the process that
sends trap and inform messages to an NMS station.

www.juniper.net Network Monitoring • Chapter 12- 7


Junos Troubleshooting in the NOC

Network Management Station

• What is the NMS?


• Computer that runs applications that monitor and control
the managed devices
• Trap collection. MIB browser. utilization monitoring

NMS

What Is the NMS?


An NMS is generally a computer with SNMP applications installed. These applications can include an SNMP MIB browser
that is used to view information from a managed device, a trap collector that listens for unsolicited traps from a managed
device, network monitor tools that poll managed devices for utilization, and many other similar applications. There can also
be multiple NMS stations in a managed network for redundancy or specific tasks.

Chapter 12-8 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Management Information Base


• What is the Management
Information Base?
• Collectively referred to as the
MIB database
• Describes the structure of the private
management data of a enterprises
--.--c ..........
managed device ......- juniperMIB
jnxProducts
• Hierarchical namespace jnxMibs
depicted as a tree
• Contains managed objects
identified by OIDs

Management Information Base


A MIB is a hierarchy of information used to define managed objects in a network device. The MIB structure is based on a tree
structure, which defines a grouping of objects into related sets. Each object in the MIB is associated with an object identifier
(OID), which names the object. The leaf in the tree structure is the actual managed object instance, which represents a
resource, event, or activity that occurs in your managed device.
MIBs are either standard or enterprise-specific. Standard MIBs are created by the Internet Engineering Task Force (IETF) and
documented in various RFCs. Depending on the vendor, many standard MIBs are delivered with the NMS software. You can
also download the standard MIBs from the IETF website, http://www.ietf.org, and compile them into your NMS, if necessary.
Enterprise-specific MIBs are developed and supported by a specific equipment manufacturer. If your network contains
devices that have enterprise-specific MIBs, you must obtain them from the manufacturer and compile them into your
network management software.
For a list of Juniper Networks enterprise-specific supported MIBs refer to the Junos OS SNMP MIBs and Traps Reference PDF
document located in the Juniper TechPubs,
http://www.juniper.net/techpubs.

www.juniper.net Network Monitoring • Chapter 12-9


Junes Troubleshooting in the NOC

Object Identifiers

• Purpose of 01 Ds
• Identifies a managed object within the MIB hierarchy
OMIBTree
0iso{1)
0org{1-3)
D do<I {1-3-6J
D mtemet c1-3_5_1J
D private {1-3.6_uJ
D enterpr;ses {1-3.6_1-4_1J
user@router> show snmp mib get jnxBoxserialNo.O DiuniperMIB{1-3_6_1 -4_1-2636)
DinxMi!Js {1-3.6 1 _ - 4 1_2636
_ _ 3J
jnxBoxserialNo.O = D4889 DinxBoxAnatomy <1-3_5_1_4_1-2636_3_1J
DinxBoxaass c1-3_5_1_4_1-2636.3_1-1i
DinxBoxDescr c1-3_5_1_4_1_2636_3_1-2i
user@router> show snmp mib walk 1. 3. 6 .1. 4 .1. 2636. 3 .1. 3 O l'lXBollSerialNo <1-3_6 _1 1_2636 3
_ _1-3i
i _4_
jnxBoxSerialNo.O = D4889

user@router> show chassis hardware I match chassis


Chassis D4889 MXBO

Purpose of OIDs
An 010 identifies a managed object within the MIB hierarchy. The slide gives a physical representation of the MIB tree and
the OIOs that relate to each specific part of the tree as it branches off to the jnxBoxSerialNo 010. The 010 is represented by
its name or its numeric instance value. The slide depicts the usage of the show srunp mib get and show srunp mib
walk commands to acquire the chassis serial number of an MX80 device.

Chapter 12-10 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Structure of Management Information


• Purpose of the SMI database
• Defines the rules of describing management information
• Uses Abstract Syntax Notation One -ASN.1
• Standard and flexible notation for describing data structures
• Allows managed device vendors and NMS vendors to work
together indirectly

Purpose of the SMI


SNMP agents store information in a hierarchical database called the Structure of Management Information (SMI). The SMI
resembles a file system; information is stored in individual files that are hierarchically arranged in the database. These SMI
files use the Abstract Syntax Notation One (ASN.1) standard when describing data structures. This standard format allows
different managed device vendors and NMS vendors to work together indirectly.

www.juniper.net Network Monitoring • Chapter 12-11


Junes Troubleshooting in the NOC

SNMP Versions

• Three major versions of SNMP


•SNMPv1
• The initial implementation of SNMP that defines the architecture
and framework for SNMP
•SNMPv2c
• The revised protocol with improvements to performance and
manager-to-manager communications
• Not backwards compatible with SNMPv1
•SNMPv3
• The most up-to-date protocol that focuses on security
• Backwards compatible with SNMPv1 and SNMPv2c

SNMP Versions
The Junos OS supports the three major versions of SNMP; SNMP version 1 (SNMPvl), SNMP version 2c (SNMPv2c), and
SNMP version 3 (SNMPv3). However, the main focus of this section is SNMPv2c and SNMPv3.
SNMPvl is the initial implementation of SNMP that defines the architecture and framework for SNMP.
SNMPv2c is the revised protocol with improvements to performance and manager-to-manager communications. Specifically,
SNMPv2c implements community strings, which acts as passwords when determining who, what, and how the SNMP clients
can access the data in the SNMP agent . The community string is contained in SNMP Get, GetBulk, GetNext, and Set
requests. The agent may require a different community string for Get, GetBulk, and GetNext requests (read-only access) than
it does for Set requests (read-write access). SNMPv2c is not backward compatible with SNMPvl.

SNMPv3 is the most up-to-date protocol and focuses on security. SNMPv3 defines a security model, user-based security
model (USM), and a view-based access control model (VACM). SNMPv3 USM provides data integrity, data origin
authentication, message replay protection, and protection against disclosure of the message payload. SNMPv3 VACM
provides access control to determine whether a specific type of access (read or write) to the management information is
allowed. SNMPv3 is backward compatible with SNMPvl and SNMPv2c.

Chapter 12-12 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

SNMPv3 Authentication and Encryption

• User-based Security Model - known as USM


• Concept for a user for which security parameters are
configured
• Levels of security
• Authentication - MD5 or SHA
• Encryption protocols - AES128, DES. or 3DES
• Password hash
• Provides data integrity, data origin authentication, message
replay protection, and protection against disclosure of the
message payload
• Contains local device and remote NMS configuration

USM
USM uses the concept of a user for which security parameters are configured for both the agent and the manager. Messages
sent using USM are better protected than messages sent with community strings, where passwords are sent in clear-text.
With USM, messages exchanged between the manager and the agent can have data integrity checking and data origin
authentication through Message Digest 5 (MD5) or Secure Hash Algorithm (SHA). USM protects against message delays and
message replays by using time indicators and request IDs. Encryption is also available through AES128, Data Encryption
Standard (DES), or triple Data Encryption Standard (3DES).

www.juniper.net Network Monitoring • Chapter 12-13


Junos Troubleshooting in the NOC

SNMP v3 Access Control

• View-Based Access Control Model - known as VACM


• Provides access control by determining if the requested
access is allowed based on user group permissions
• Assigns users to groups - security-to-group
• Assigns views to groups - access
• Read-view
• Write-view
• Notify-view
• Assigns security levels to groups - access
• No security
• Authentication
• Privacy - Authentication and encryption

-, ' ' ' ' '


lc2014Junlpe,Netwmlol,lne,AU�-' JUn�J WorldwideEducationServlces www,un-net I 14
�k•.wA Jt::•..&,l�itf:.;..._h; ...;:, - -- ---

VACM
To complement the USM, SNMPv3 uses the VACM, a highly granular access-control model for SNMPv3 applications, Based
on the concept of applying security policies to the name of the groups querying the agent, the agent decides whether the
group is allowed to view or change specific MIB objects, VACM defines collections of data, groups of data users, and access
statements that define which views a particular group of users can use for reading, writing, or receiving traps and informs.
VACM consists of two configuration trees: security-to-group and access. Security-to-group is where users are assigned to
groups based on security-model. USM, v2c, and vl are supported security-models. USM uses the USM security model. V2c
and vl use a simple community string as the user name. Users are assigned by using the user from the USM configuration
under the security-name option. Each user can only be assigned to a single group.
Access defines groups which contain a single security model, multiple security levels and a single read, write and notify view
for each security level.

Chapter 12-14 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Basic SNMP Operations


• Basic operations of SNMP
• SNMP messages
• Read - Monitors a managed device - UDP 161
• Write - Controls a managed device - UDP 161
• Trap - Alert sent by managed device to NMS - UDP 162
• Inform - Alert sent by the managed device to the NMS, managed
device expects an acknowledgment from the NMS - UDP 162
• Each version of SNMP supports read, write, and trap
messages
• Only SNMPv3 supports inform messages

Basic Operations of SNMP


There are four basic types of SNMP messages; read, write, trap, and inform. The read operation allows the NMS to poll
information from the managed devices, such as CPU utilization. The write message allows the NMS to control the device,
such as the PING MIB. The trap operation allows the managed device to send unsolicited SNMP messages to the NMS, such
as when an interface transitions into the down state. The inform operation is very similar to the trap operation, in which the
managed device sends an unsolicited SNMP message to the NMS device to alert it of a change on the managed device.
However, the inform operation differs from the trap operation because the inform operation requires an acknowledgment
from the managed device.
As the slide notes, read and write messages occur on User Datagram Protocol (UDP) port 161, and trap and inform
messages occur on UDP port 162.
All versions of SNMP support read, write, and trap messages. However, only SNMPv3 supports inform messages.

www.juniper.net Network Monitoring • Chapter 12-15


Junos Troubleshooting in the NOC

Spoof Trap
• Spoofing SNMP traps
• The Junos OS allows for trap messages to be spoofed to test
configuration and NMS applications
user@router> request S�lflP spoof-trap ?
Possible comple�ions:
<trap> The name of the trap to spoof
adslAtucinitFailureTrap
adslAtucPerfESsThreshTrap

• You can also set specific values and instances of the objects
in the trap message
user@router> requ.est smnp spoof-trap linkup variable-bindings "ifindex[so-1/1/0. OJ = so-
1/1/0. 0, ifAdmiilStatus[so-1/1/0.0] = 1, ifOperStatus[so-1/1/0.0] = 2"

Spoofing a Trap
The Junos OS supports the ability to spoof an SNMP trap. Spoofing a trap is useful in testing your SNMP configuration
against the NMS. Without the spoof trap operation, a technician would have to go on site and manually remove
field-replaceable units (FRUs) in a managed device to correctly test the SNMP configuration. Now, with the request snmp
spoof trap command, you can simply name the trap you wish to spoof, and the Junos OS sends the requested trap to the
NMS device.
Also, as the slide depicts, you can add variables to a spoofed trap by using the variable-bindings option. This option
allows you to add in specific information into the trap message which might be helpful in testing your NMS applications.

Chapter 12-16 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

SNMP Inform
• Details of SNMP inform messages
• Trap messages can be lost
• Inform message is the same as a trap message, but
includes reliability
• NMS returns an acknowledgment for the inform request
• Multiple retries by the agent if no acknowledgment is received
Manag�d Device
NMS
Inform Request

Inform Acknowledgment

SNMP Inform Message


The Junos OS supports two types of notifications: traps and informs. With traps, the receiver does not send any
acknowledgment when it receives a trap. Therefore, the sender cannot determine if the trap was received. A trap might be
lost because a problem occurred during transmission. To increase reliability, an inform is similar to a trap except that the
inform is stored and retransmitted at regular intervals until one of the following conditions occurs:
The receiver of the inform message returns an acknowledgment to the SNMP agent.
A specified number of unsuccessful retransmissions have been attempted and the agent discards the inform
message.
If the sender never receives a response, the inform can be sent again. Thus, informs are more likely to reach their intended
destination than traps are. Informs use the same communications channel as traps, same socket and port, but have
different protocol data unit (PDU) types.
Informs are more reliable than traps, but they consume more network resources. Unlike a trap, an inform is held in memory
until a response is received or the timeout is reached. Also, traps are sent only once, whereas the transmission of an inform
may be retried several times. Use informs when it is important that the SNMP manager receives all notifications. However, if
you are more concerned about network traffic, or managed device memory, use traps.

www.juniper.net Network Monitoring • Chapter 12-17


Junos Troubleshooting in the NOC

SNMP Community Strings

• SNMP Community Strings


• Community name - String used as a password to
authenticate with an NMS
• Provides a weak form of security - passed as clear text
• Public - common community name for read access
• Private - common community name for write access
• Community names must be unique
• Communities are case sensitive
• _eublic is different than g_ublic
• Authorization - Defines access to MIBs
• read-only - read access to MIB
• read-write - read and write access to the MIB

Lc2014Junl�«�ln�Allrti,,ts-.. JUnl�
"1C;
WorldwideEducationServlces WWW.j\nlOJ)ernet J 18
'l:..:.J...�»:...J:!.JI,...:_.,£ L,,,__,__ � :.;._r-

SNMP Community Strings


The SNMP community string defines the relationship between an SNMP agent and the NMS. This string acts like a password
to control the NMS' access to the SNMP agent. To configure a community string in a Junos OS configuration, include the
community statement at the [edit snmp] hierarchy level.
Community names must be unique. You cannot configure the same community at the [edit snmp community] and
[edit snmp v3 snmp-community conununi ty-index] hierarchy levels. Community names are also case sensitive,
for example, the Public community is different than the public community.
The default authorization level for a community is read-only. To allow Set requests within a community, you need to define
that community with the authorization of read-write. For Set requests, you also need to include the specific MIB objects that
are accessible with read-write privileges using the view statement. The default view includes all supported MIB objects that
are accessible with read-only privileges; no MIB objects are accessible with read-write privileges by default.

Chapter 12-18 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

SNMPv2c Community and Traps


Configuration

• Configuring the community


[edit)
user@router# show snmp
comm nity public;

• Configuring the traps


[edit)
lab@mxA-1# show snmp

trap-group new-group
categories
link;

targets
10.1.1.10;

Configuring a Community
The only required statement when configuring an SNMPv2c community string is the community statement with the
community name. If the community name contains spaces, you can enclose the community name in quotation marks.

Configuring SNMP Traps


You can create and name a group of one or more types of SNMP traps and then define which systems receive the group of
SNMP traps. The trap group must be configured for SNMP traps to be sent to the NMS. To create an SNMP trap group,
include the trap-group statement at the [edit snmp] hierarchy level.
The trap group name can be any string and is embedded in the community name field of the trap. To configure your own trap
group port, include the destination-port statement. The default destination port is UDP port 162.
For each trap group that you define, you must include the target statement to define at least one NMS as the recipient of
the SNMP traps in the trap group. Specify the IP version 4 (1Pv4) or IP version 6 (1Pv6) address of each NMS, do not use its
hostname.
Specify the types of traps the trap group can receive in the categories statement. For information about which category
traps belong to, refer to the Juniper TechPubs at
http://www.juniper.net;techpubs.

www.juniper.net Network Monitoring • Chapter 12-19


Junos Troubleshooting in the NOC

SNMPv3 Community Configuration

• Configuring SNMPv3 community strings


• Configure community index and security name

[edit snmp v3]


lab@mxA-li!' show

snmp-community 1 {
community-name SECRET-DATA
security-na.�e jtnoc; Matches security name
Wag community-tag;! with the target
parameters

Tag referenced in
target-address
configuration

SNMPv3 Community Configuration


Configuring a community in SNMPv3 is slightly different than configuring a community string in SNMPv2c. You must
configure a security community index and a security name. The security name in the community must match the security
name configured at the [ edit snmp v3 target-parameters target-parameters-name parameters]
hierarchy level when you configure SNMPv3 traps.
The tag configuration parameter is optional, but you can reference it later in a target address to identify the address of the
NMSs that are allowed to use the community string.

Chapter 12-20 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

SNMPv3 Notifications and OID Properties

• Configure notification type and filter


[edit snmp v3]
lab(lmxA-lt show Tag referenced in
target-address
notify jtnoc-notify
configuration
tvos infcrm;
It�� notify-t:ag:I

notifv-filter nfl { and informs


!oid .1 include}
Sends only enterprise
notify-filter nf2 f specific traps and informs
loid 1.3.C.1.4.l include:)

• Configure 010 view properties


[edit snmp]
lab@mxA-1# show

view jnxltlarms {
oid 1.3.6.1.4.1.2636.3.4 include;

SNMPv3 Notifications and OID Properties


The tag option under the [edit snmp v3 notify notification-name] is referenced under a specific target
address and allows you to specify which NMS receives the notification. The type configuration option allows you to send
traps or inform messages. The slide depicts the configuration that is necessary to send inform messages.
The notify filter allows you to specify which traps, or informs, are sent to, or excluded from, the NMS. Specifying a certain OID
allows you to include, or exclude, certain parts of the MIB in the trap or inform messages. Then, you must reference the
notify filter under the [edit snmp v3 target-parameters notify-filter notify-filter-name], which is
shown on the next slide.

www.juniper.net Network Monitoring • Chapter 12-21


Junos Troubleshooting in the NOC

SNMPv3 Target Address and Target


Parameters

• Configure the target address


[edit snmp v3]
lab@mxA-li show target-address tal
address 10.10.20.123;
tag-list "notify-tag community-tag";
target-parameters tpl;

• Configure the target parameters


[edit snmp v3]
lab@mxA-1# show

target-parameters tpl {
parameters {
message-processing-model v3;
security-model usn;
security-level privacy;
security-name j�noc;

notify-fil�er nf2;

Configuring the Target Address


The target address is the address of the NMS. As the slide depicts, the address of the NMS is present, as well as the two
tags seen on previous slides. Then, you must specify which target parameters, with the target-parameters statement,
the target address must use. Note that the slide depicts multiple tags in use by the target address tal. You can specify
multiple tags by including the tags in quotations.

Configuring the Target Parameters


In the target parameters, you must specify the message processing type, the security model, the security level, and the
security name. Remember that the security name that is being specified at this point was originally defined in the SNMPv3
community. Then, you must reference the necessary notify filter. As with the tag list under the target address, you can specify
multiple notify filters by including the notify filters in quotations.

Chapter 12-22 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

SNMPv3 USM Configuration - SNMP Users

• USM configuration - SN MP users


[edit]
user@route�i show snmp
v3
usm
local-engine
user User3
authentication-sha
authentication-key "S 9Sy!l} r8X7-V2gJcylMr.'7X-
dk. mTn/p0BhyluORSleW8Ndb:·14aJGDj qroM8X­
Vboa36/9A0Blhl,•WcSs2goGUj!iqmfz/CtuBICAvW8Xbwz3n6uOEcyeKScSVw2gUDApuOhSLxNsYolRbsgoGU69C.1>.
pBRhSrvWyr8 7 -'VY2 69CpIEhSreM8 z3/ tOBSyaZGDqrn if SECRET-DATA
11;

privacy-des {
privacy-key
1
'$9S6rhU/0BIEceK8369puBEhVws4GDqmfF69.mQn9Cu0R.hSyvW8X7Nbsp0BEcSMWZUDjHmf5F9tu3nreKMXx.Ndb
s2aDik.fTiHtuOBSyaZGU.Pz36CA03ncyeKx7Hq.PFn01RrlM5QSrKMXxUjiHqfQFn/tu6/0IEcleUjiqTzFn/Cp
0aZDkPfn6WLX7bs�•; ii SECRET-DJ!.�TA

USM Configuration-SNMP Users


For each SNMPv3 user, you can specify the username, authentication type, authentication password, privacy type, ar.d
privacy password. After a user enters a password, a key based on the engine ID and password is generated and written to the
configuration file. After the generation of the key, the password is deleted from this configuration file.
The configuration on the slide shows authentication using SHA and encryption using DES for user User 3.

www.juniper.net Network Monitoring • Chapter 12-23


Junes Troubleshooting in the NOC

SNMPv3 USM Remote Engine

• USM remote engine configuration


• Required for SNMP inform message processing
[edit)
user@router# show snmp
v3
usm

remote-engine 666A61736B66313233316A666A61736C
user Userl {
authentication-none;
privacy-none;

SNMPv3 USM Remote Engine


To send inform messages to an SNMPv3 user on a remote device, you must first specify the engine identifier for the SNMP
agent on the remote device where the user resides. The remote engine ID is used to compute the security digest for
authenticating and encrypting packets sent to a user on the remote host. When sending an inform message, the agent uses
the credentials of the user configured on the remote engine.
For informs, remote-engine engine-id is the identifier for the SNMP agent on the remote device where the user
resides. For informs, user username is the user on a remote SNMP engine who receives the informs. Informs generated
can be unauthenticated, authenticated, or authenticated_and_encrypted, depending on the security level
of the SNMPv3 user configured on the remote engine (the inform receiver). The authentication key is used for generating
Message Authentication Code (MAC). The privacy key is used to encrypt the inform POU part of the message.
The configuration on the slide shows a configured remote engine with no authentication or encryption for user User 1.

Chapter 12-24 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

SNMPv3 VACM - Security-to-Group

• VACM - security-to-group configuration


[edit snmp v3]
user@router# show vacm
security-to-group {
security-model usm
security-name Userl
group system;

security-name none
group system;

security-model v2c {
security-name public
group v2;

Assigning Security Model and Security Name to a Group


For SNMPv3, the security-name is the username configured at the [edit snmp v3 usm local-engine user
username] hierarchy level. For SNMPvl and SNMPv2c, the security name is the community string configured at the [edit
snmp v3 snmp-communi ty community-index] hierarchy level. Note that the USM security name is separate from
the SNMPvl and SNMPv2c security names. If you support SNMPvl and SNMPv2c in addition to SNMPv3 in your network,
you must configure separate security names within the security-to-group configuration at the [edit snmp v3 vacm
access) hierarchy level.
The above configuration defines a USM security model and assigns Userl to the group called system and user none to the
system group as well. It also defines a v2c security model and assigns user public to the group called v2.

www.juniper.net Network Monitoring • Chapter 12-25


Junos Troubleshooting in the NOC

SNMPv3 VACM - Access

•VACM access configuration


[edit srunp v3 vacm]
lab@mxA-li show access group system
default-context-prefix
security-model usm
security-level privacy
read-view all;
notify-view all;

security-level none
read-view box;
notify-view all;

Configuring the Access Privileges Granted to a Group


To configure the access privileges granted to a group, include the group statement at the [edit snmp v3 vacm
access] hierarchy level. The group name is a collection of SNMP users that belong to a common SNMP list that defines an
access policy. Users belonging to a particular SNMP group inherit all access privileges granted to that group. Next, you must
configure a security model. The options for the security model are any, usm, vl, v2c. Then, you must configure the security
level. The slide depicts the security levels of privacy and none. The privacy option provides authentication and
encryption, the authentication option, which is not shown, provides authentication but no encryption, the none option
provides no authentication and no encryption.
Finally, you must associate MIB views with an SNMP user group. The slide depicts the read-view and notify-view
options. Note that you must associate at least one view option at this configuration level.

Chapter 12-26 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Helpful SNMP Commands

• Some helpful SNMP commands


•show snmp mib (getlwalk)
• Display local SNMP MIB OIDs
•show snmp statistics
• Display SNMP statistics about packets send and received from the
managed device
•show snmp inform-statistics
• Display information about SNMP inform requests
•show snmp v3
• Display SNMPv3 specific information

Helpful SNMP Commands


The slide displays helpful SNMP operational commands. The show snmp mib get command allows you to retrieve and
display one or more SNMP object values. The show snmp mib walk command allows you to retrieve and display the
SNMP object values that are associated with the requested OID. When you use this option, the Junos OS displays the objects
below the subtree that you specify. The following output displays examples of the two previous commands.
user@router> show snmp mib get ?sysObjectID.0 sysUpTime.O?
sysObjectID.O = jnxProductNarneM20
sysUpTirne.O = 1640992

user@router> show snmp mib walk system


sysDescr.O = Juniper Networks, Inc. rn20 internet router, kernel
Junos OS Release #0: 2004-1 Build date: build date UTC Copyright (c) 1996-2004
Juniper Networks, Inc.
sysObjectID.O = jnxProductNarneM20
sysUpTirne.O = 1640992
sysContact.O = Your contact
sysNarne.O = my router
sysLocation.O = building 1
sysServices.O = 4
Continued on the next page.

www.juniper.net Network Monitoring • Chapter 12-27


Junos Troubleshooting in the NOC
Helpful SNMP Commands (Contd.)
The show snmp statistics command displays statistics about SNMP packets sent and received by the managed
device. The show snmp inform-statistics displays information about SNMP inform requests. The following output
displays examples of the two previous commands.
user@router> show snmp statistics
SNMP statistics:
Input:
Packets: 246213, Bad versions: 12, Bad community names: 12,
Bad community uses: 0, ASN parse errors: 96,
Too bigs: 0, No such names: 0, Bad values: 0,
Read onlys: 0, General errors: 0,
Total request varbinds: 227084, Total set varbinds: 67,
Get requests: 44942, Get nexts: 190371, Set requests: 10712,
Get responses: 0, Traps: 0,
Silent drops: 0, Proxy drops: 0, Commit pending drops: 0,
Throttle drops: 0,
V3 Input:
Unknown security models: 0, Invalid messages: 0
Unknown pdu handlers: 0, Unavailable contexts: 0
Unknown contexts: 0, Unsupported security levels: 1
Not in time windows: 0, Unknown user names: 0
Unknown engine ids: 44, Wrong digests: 23, Decryption errors: 0
Output:
Packets: 246093, Too bigs: 0, No such names: 31561,
Bad values: 0, General errors: 2,
Get requests: 0, Get nexts: 0, Set requests: 0,
Get responses: 246025, Traps: 0

user@router> show snmp inform-statistics


Inform Request Statistics:
Target Name: TA1_v3_md5_none Address: 172.17.20.184
Sent: 176, Pending: 0
Discarded: 0, Timeouts: 0, Probe Failures: 0
Target Name: TA2_v3_sha_none Address: 192.168.110.59
Sent: 0, Pending: 4
Discarded: 84, Timeouts: 0, Probe Failures: 258
Target Name: TA5_v2_none Address: 172.17.20.184
Sent: 0, Pending: 0
Discarded: 2, Timeouts: 10, Probe Failures: 0
Continued on the next page.

Chapter 12-28 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Helpful SNMP Commands (contd.)


The show snmp v3 command displays the SNMPv3 operating configuration. The following output displays an example of
the show snmp v3 command.
user@router> show snmp v3

Local engine ID: 80 00 Oa 4c e04 31 32 33 34


Engine boots: 38
Engine time: 64583 seconds
Max msg size: 2048 bytes

Engine ID: local


User Auth/Priv Storage Status
userl md5/des nonvolatile active
user2 sha/none nonvolatile active
user3 none/none nonvolatile active

Engine ID: 81 00 Oa 4c 04 64 64 64 64
User Auth/Priv Storage Status
UNEW md5/none nonvolatile active
Group name Security Security Storage Status
model name type
gl usm userl nonvolatile active
g2 usm user2 nonvolatile active
g3 usm user3 nonvolatile active

Access control:
Group Context Security Read Write Notify
prefix model/level view view view
gl usm/privacy vl vl
g2 usm/authent vl vl
g3 usm/none vl vl

www.juniper.net Network Monitoring • Chapter 12-29


Junos Troubleshooting in the NOC

Agenda: Network Monitoring

•SNMP
7RMON
• Flow Monitoring

RMON
The slide highlights the topic we discuss next.

Chapter 12-30 • Network Monitoring www.juniper.net


Junes Troubleshooting in the NOC

RMON

• What is RMON?
• Monitors objects on managed device
• Warning sent if value exceeds a defined range
• Uses MIS to store OIDs
Bandwidth for an
interface
exceeded
Managed Device configured value.
NMS Sending RMON

[:J
-RM _ONA_larm_

I M I

What Is RMON?
The Junos OS supports the RMON MIB (RFC 2819), which allows a management device to monitor the values of MIB objacts,
or variables, against configured thresholds. When the value of a variable crosses a threshold, an alarm and its
corresponding event are generated. The event can be logged and the SNMP agent can generate a trap.
An operations support system (OSS) or a fault-monitoring system can be used to automatically monitor events that track
many different metrics, including performance, availability, faults, and environmental data. For example, an administrator
might want to know when the internal temperature of a chassis has risen above a configured threshold, which might indicate
that a chassis fan tray is faulty, the chassis air flow is impeded, or the facility cooling system in the vicinity of the chassis is
not operating normally.

www.juniper.net Network Monitoring • Chapter 12-31


Junos Troubleshooting in the NOC

Configuring RMON

• Configuring RMON
[edit srunp rmon]
user@router# show
alam 1 {
variable
sample-type absolute-value;
rising-threshold 800000000; Index number correlate
with event index
rising-event-index 1; numbers
falling-event-index

Options include:
event 1 {
log
ltyFe log-and-trap; I
log-and-trap
none
event 2 {
snmptrap
tyFe log;

Configuring RMON
To enable RMON alarms, you must perform the following steps:
Configure SNMP, including trap groups. You configure SNMP at the [edit snmp] hierarchy level.
Configure events using the CLI at the [edit snmp rmon event] hierarchy level.
Configure alarms using the CLI at the [edit snmp rmon alarm] hierarchy level. These alarms include the
variables to monitor rising and falling thresholds, the sampling types and intervals, and the corresponding
events to generate when alarms occur.
The slide depicts an alarm that monitors the variable ifHCOutlSecRate, which counts the number of bits per second
(bps) that egresses an interface. Then, the interface SNMP index is referenced after the variable. You can determine the
interface SNMP index by simply using the show interface command for the interface in question. Then, the rising
threshold of 800000000 bps, which is about 800 megabits per second (Mbps), is set with a falling threshold of
200000000, which is about 200 Mbps. Then, the rising event index is set to 1 and the failing event index is set to 2, which
relates to event 1 and event 2 respectively. Event 1 is set to record a log message and send a trap whenever the interface
output reaches the rising threshold. Event 2 is set to only record a log message when the falling threshold is passed after the
rising threshold has previously been reached.

Chapter 12-32 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Helpful RMON Commands

• Some helpful RMON commands


•show srunp rmon alarms
• Displays information about current RMON alarms
•show srunp rmon events
• Displays information about triggered RMON events
•show srunp rmon logs
• Displays information about RMON logs
• Event type log or log-and-trap

Helpful RMON Commands


The slide lists helpful RMON commands. The show snmp rmon alarms command displays information about the current
RMON alarms. The show snmp rmon events command displays information about RMON events that have already
been triggered. The show snmp rmon logs command displays information about RMON logs.
The following output displays examples of the previous commands.
user@router> show snmp rmon alarms

Alarm
Index Variable description Value State

1 monitor
ifHCOutlSecRate.579 765136192 rising threshold
Continued on the next page.

www.juniper.net Network Monitoring • Chapter 12-33


Junos Troubleshooting in the NOC

Helpful RMON Commands (contd.)

user@router> show snmp rmon alarms

Alarm
Index Variable description Value State

1 monitor
ifHCOutlSecRate.579 O falling threshold

user@router> show snmp rmon events

Event
Index Type Last Event
1 log and trap 2013-01-18 14:55:07 MST
2 log and trap 2013-01-18 14:55:52 MST

user@router> show snmp rmon logs


Event Index: 1
Description: Event 1 triggered by Alarm 1, rising threshold (800000000)
crossed, (variable: ifHCOutlSecRate.579, value: 893717712)
Time: 2013-01-18 14:54:27 MST
Description: Event 1 triggered by Alarm 1, rising threshold (800000000)
crossed, (variable: ifHCOutlSecRate.579, value: 921619704)
Time: 2013-01-18 14:55:07 MST

Event Index: 2
Description: Event 2 triggered by Alarm 1, falling threshold (200000000)
crossed, (variable: ifHCOutlSecRate.579, value: 0)
Time: 2013-01-18 14:53:07 MST
Description: Event 2 triggered by Alarm 1, falling threshold (200000000)
crossed, (variable: ifHCOutlSecRate.579, value: 0)
Time: 2013-01-18 14:54:57 MST
Description: Event 2 triggered by Alarm 1, falling threshold (200000000)
crossed, (variable: ifHCOutlSecRate.579, value: 0)
Time: 2013-01-18 14:55:52 MST

Chapter 12-34 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Agenda: Network Monitoring

•SNMP
•RMON
7Flow Monitoring

Flow Monitoring
The slide highlights the topic we discuss next.

www.juniper.net Network Monitoring • Chapter 12-35


Junes Troubleshooting in the NOC

JFlow
• An overview of J Flow
• Keeps track of sampled packets
• Records of sampled packets are aggregated and reported
• Reports are sent to a JFlow collector
• Support for JFlow v5, JFlow v8, JFlow v9, lnline JFlow (v10)

[] JFlow Collector

Packet Header I
Samples I

iO�n��="_<:
I

II! · J
I

-------
1
!

What Is JFlow?
JFlow is the sampling service that is available on Juniper devices. The JFlow service keeps track of the packets treated by the
router on any particular interface and the details of the traffic flow, such as the source address, the destination address,
packets, and byte counts, are aggregated and reported using the JFlow record. The JFlow reporting and the sampling service
will not hinder the traffic forwarded and processed by the router, prior to reporting the JFlow records, the router will sample
the incoming traffic as such eliminating any network delay to jitter introduced on the original flows. An integral part of the
JFlow sampling solution is the JFlow collector which is situated outside of the Juniper device as a separate appliance.
JFlow v5 and v8 are the most common standards today that are supported by most collectors worldwide. Two options of
JFlow v5 and v8 reporting are available, sampling by routing engine or alternatively, by the MS-PIG or MS-OPC. Routing
engine based sampling requires no additional hardware, but it poses resource implications on the routing engine.
Alternatively, using a dedicated MS-OPC or a MS-PIG allocates a separate services plane that eliminates any performance
implications. JFlow v5 and v8 records and templates are slightly different in content, mainly lacking support for 1Pv6 and
MPLS reporting.
With the limitations of JFlow v5 and v8, JFlow v9 introduces a new template to bridge this gap. The templates are backward
compatible across the versions. With flexibly introduced, the Juniper devices must use a MS-DPC or a MS-PIC. JFlow v9 does
not support routing engine based sampling.
Unique to Juniper MX 30 Trio forwarding line cards is inline JFlow, or JFlow v10. Inline JFlow provides increased sampling
capacity with limited forwarding impact. There is no need for additional hardware which transforms any MX 30 based router
into a carrier grade sampling node. Once the MX 30 router samples the traffic using the in line JFlow process, the records are
sent to the JFlow collector using the industry standard IPFIX version 10 format.

Chapter 12-36 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Configuring lnline JFlow (1 of 4)

• Configuration of the template


• Template under the [edit services J hierarchy
• Sampling instance
• Define as version-ipfix
• Set template type - 1Pv4 or 1Pv6

[edit services]
user@router# show
flow-monitoring {
version-ipfix
template sample-template
!ipv4-template; !
Use ipv6-template
for 1Pv6 traffic

Configuring the Template


To begin configuring inline JFlow, you must first configure the template. The template is important because the JFlow
collector decodes the data records based on the template it receives. You can configure the template under the [edit
services] hierarchy level. From that hierarchy level, you must specify the IPFIX format as the version, name the template,
and specify a protocol family. Currently, for inline JFlow, you can specify the 1Pv4 family or 1Pv6 family in a template.
Additional parameters that are optional that are not shown on the slide, such as the active flow timeout and the inactive flow
timeout, can be added to the template.

www.juniper.net Network Monitoring • Chapter 12-37


Junos Troubleshooting in the NOC

Configuring lnline JFlow (2 of 4)

• Configuration of the sampling instance input


properties
• Sampling instance under the [edit forwarding­
options sampling] hierarchy level

[edit forwarding-options sampling]


user@router# show instance instance-1
input {
rate 10;
run-length 5;
max-packets-per-second 30000;

- _·�� 1��JP�--World�eEducationServlces www,1umpernet 1 38

Configuring the Input Properties


The sampling instance input parameters defines how traffic is sampled.
The rate statement specifies the ratio of packets to be sampled. For example, if you configure a rate of 10, 1 out of every
10 packets is sampled. Alternatively, setting the rate option to a value of 1 results in every packet in the flow being sampled.
The run-length statement specifies the number of matching packets to sample following the initial one-packet trigger
event. Configuring a run length greater than O allows you to sample packets following those already being sampled.
The max-packets-per-second statement specifies the maximum number of packets to sample in a given flow. Once a
flow reaches this limit, the sampling mechanism begins dropping packets. If you do not specify a value for the
max-packet-per-second parameter, the Junes OS supplies a default value of 1000. If you specify a value of 0, the
packet forwarding engine does not sample any packets.
If you do not include the input statement, sampling is disabled.

Chapter 12-38 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Configuring lnline JFlow (3 of 4)

• Configuration of the sampling instance ouput


properties
[edit forwarding-options sampling]
user@routeri show instance instance-1

family inet {
out.put {
flow-server 10.10.10.100
port 2055;
version-ipf:...x {
template {
lsa..rnple-templat.e; I
Referenced
template
inline-jflcw
source-address 10.10.10.1;

Configuring Output Properties


When configuring the sampling instance output properties you must first specify which protocol family you wish to sample.
You can sample both 1Pv4 and 1Pv6, however, if you sample both protocol families in a single sampling instance, you must
specify how much memory you wish to dedicate to each protocol family. We discuss how to allocate flow hash tables in the
next slide.
Once you have chosen which protocol families you want to sample, you must configure the flow server properties and the
local source information for the inline JFlow service. The slide depicts that the Junos OS is sending sampled packets to the
JFlow collector at the 10.10.10.100 address on port 2055. Then, the IPFIX format is in use and the sample-template is
being referenced. Finally, the Junos OS is sourcing the sampled packets from the 10.10.10.1 address. Remember that the
sample-template was created earlier when we discussed template creation.

www.juniper.net Network Monitoring • Chapter 12-39


Junos Troubleshooting in the NOC

Configuring lnline JFlow (4 of 4)


• Chassis configuration
• Bind the JFlow sampling instance to the MX 30 Trio FPC
• Change the flow table size if 1Pv4 and 1Pv6, or just 1Pv6, are
being sampled on the same FPC
• For MX80 routers, you must bind the TFEB to a sampling
instance
• set chassis tfeb slot O sampling-instance
instance

• Firewall filter configuration


• Firewall filter with the actions of sample and accept
• Apply firewall filter to interface
• Interface must reside in the FPC specified in chassis configuration

Chassis Configuration
Binding the sampling instance to the FPC is an important step. This action tells the Junos OS to use the memory contained in
the MX 3D Trio FPC for the sampling service. To bind an MX 3D Trio FPC to a sampling instance, issue the set fpc
FPC-number sampling-instance instance-name command at the [edit chassis] hierarchy level.
If you are sampling 1Pv4 and 1Pv6 traffic on the same MX 3D Trio FPC, or only 1Pv6, you must allocate MX 3D Trio FPC
memory using the sampling hash tables. You can specify the sampling hash tables by issuing the set
ipv4-flow-table-size va.lue or set ipv6-flow-table-size va.lue commands under the [edit chassis
fpc FPC-number inline-services flow-table-size] hierarchy level. The total number of units used for both
1Pv4 and 1Pv6 cannot exceed 15. Each unit represents 256k of MX 3D Trio FPC memory. If you are only sampling 1Pv4 traffic,
it is not necessary to configure any values for the sampling hash table; all memory resources are dedicated to 1Pv4 sampling.
Note that any change in the configured size of the flow hash table initiates an automatic reboot of the MX 3D Trio FPC.
If you are configuring inline sampling for an MX80 3D router, you do not bind the sampling instance to an MX 3D Trio FPC,
you must bind the sampling instance to the TFEB. The example on the slide shows a sampling instance that is being bound
to the TFEB in slot 0. On an MX80 3D router, the slot option always receives a value of O because there can only be one
TFEB installed in an MX80 3D router.
Continued on the next page.

Chapter 12-40 • Network Monitoring www.juniper.net


Junes Troubleshooting in the NOC

Chassis Configuration (contd.)


The following output displays an example of a chassis configuration in which 1Pv4 and 1Pv6 sampling is occurring. Note how
10 flow table units are designated for 1Pv4 traffic and 5 flow table units are designated for 1Pv6 traffic. This output means
that 2560k of the TFEB memory is reserved for 1Pv4 traffic sampling and 1280k of the FPC memory is reserved for 1Pv6
traffic.
[edit chassis]
user@router# show
tfeb {
slot O {
sampling-instance instance-1;
inline-services {
flow-table-size {
ipv4-flow-table-size 10;
ipv6-flow-table-size 5;

Firewall Filter Configuration


You must define a firewall filter and then apply it to the relevant interfaces in which you wish to sample traffic. Note that you
must apply the firewall filter on interfaces to sample traffic as it enters the router. You cannot apply the firewall filter on
interfaces to sample traffic as the traffic leaves the router. You can use the from statement in a firewall filter term to
selectively specify traffic that you wish to sample. If you do not use the from statement in this manner, the Junes OS
samples all traffic for a specific protocol family that enters an interface. Finally, you must configure the then statement with
an action of sample and accept. If you do not apply the accept action to the firewall filter, the filter blocks all
nonconforming flows.

www.juniper.net Network Monitoring • Chapter 12-41


Junos Troubleshooting in the NOC

Helpful JFlow Commands

• Some helpful JFlow commands


•show services accounting status inline­
jflow
• Displays information about service accounting parameters for
inline JFlow
•show services accounting flow inline-jflow
• Displays information about packets sampled by inline JFlow

Helpful JFlow Commands


The slide lists inline JFlow commands that are helpful in determining the status of inline Jflow on the router. The show
services accounting status inl.ine-jfl.ow command displays information about the accounting parameters for
inline JFlow. The show services accounting fl.ow inl.ine-jfl.ow command displays information about packets
sampled by inline Jflow.
The following output displays examples of the two previous commands.
user@router> show services accounting status inl.ine-jfl.ow
Status information
TFEB Slot: 0
Export format: IP-FIX
IPv4 Route Record Count: 861171, IPv6 Route Record Count: 3
Route Record Count: 861175, AS Record Count: 46117
Route-Records Set: Yes, Config Set: Yes

user@router> show services accounting fl.ow inl.ine-jfl.ow


Flow information
TFEB Slot: 0
Flow Packets: 650511, Flow Bytes: 232438816
Active Flows: 1197, Total Flows: 35211
Flows Exported: 32414, Flow Packets Exported: 21583
Flows Inactive Timed Out: 29181, Flows Active Timed Out: 4110

Chapter 12-42 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Port Mirroring
• Port mirroring overview
• Entire packet is sent to an external host or packet analyzer
• Port mirroring takes precedence over sampling

Packet Analyzer

Copy of
Packets

What Is Port Mirroring?


Port mirroring is different from traffic sampling. With traffic sampling, a sampling key based on the packet header is sent to
the services hardware. There, sampled packets can be sent to a collection server. With port mirroring, the entire packet is
copied and sent out through a next hop interface.
You can configure simultaneous use of sampling and port mirroring, and set an independent sampling rate and run-length
for port-mirrored packets. However, if a packet is selected for both sampling and port mirroring, only one action can be
performed and port mirroring takes precedence. For example, if you configure a firewall filter to sample every packet
entering an interface, and the firewall filter also selects the packet to be port mirrored to another interface, only the port
mirroring takes effect. All other packets not matching the explicit filter port mirroring criteria continue to be sampled when
forwarded to their final destination.

www.juniper.net Network Monitoring • Chapter 12-43


Junos Troubleshooting in the NOC

Configuring Port Mirroring

• When configuring port mirroring


• Configure under [edit forwarding-options port­
rnirroring] hierarchy
• Specify input and output parameters
• Input parameters are global, output parameters are specific to
protocol family
[edit forwarding-options port-mirroring]
lab@mxA-141 show
input (
rate l;

family inet
output I
interface ge-0/0/1.0 I
next-hop 10.10.10.100;

Junm
"' ' 't - ·" "'"*' � 11" "
''C2014Juniper�ln�M·-·-. . WorldwideEducationServlces ---""' I 44
�L����""'--··-,..........._�,.� �- � - � � -

Configuring Port Mirroring


To begin configuring port mirroring, you must navigate to the [edit forwarding-options port-mirroring]
hierarchy level. Then, you can specify the input parameters that applies to all the protocol families that are configured to
participate in port mirroring. Next, specify the output parameters, which are specific to each configured protocol family. The
slide depicts an input rate of 1, which means every packet will be port mirrored, and the output parameters of the interface
that points to the analyzer and the IP address of the next hop to reach the analyzer, which typically is the analyzer itself.

Chapter 12-44 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Configure Port Mirroring Firewall Filter


• Configuring a firewall filter to send traffic to the port
mirroring process
[edit firewall family inet]
user@router# show
filter port-mirror {
term 1 {
then port-mirror;

• Apply firewall filter to an interface


• Ingress or egress

Port Mirroring Firewall Filter


You must configure a firewall filter that has the action of port-mirror to act on packets that require port mirroring.
Although it is not shown on the slide, you can add match criteria with the from option in the firewall filter to selectively
choose traffic to port mirror. Then, you must apply the firewall filter to the necessary interface. Unlike sampling, which only
allows you to sample traffic that is entering an interface, you can apply the firewall filter that is port mirroring traffic on both
the ingress and egress interfaces of a flow.

www.juniper.net Network Monitoring • Chapter 12-45


Junos Troubleshooting in the NOC

static ARP for 1Pv4 Port Mirroring

• Static ARP for the analyzer is required to port mirror


1Pv4 traffic
Interface that points
=� r
�:;-;-1
[edit interfaces!ge-0/0/1]1 --::::::::::: ::_ towards to analyzer
_,......���������
lab@mxA-lif show
unit O {
family inet
address 10.10.10.1/24
arp!10.10.10.1oo!rnac ������
.._ ,,-.....

MAC address
IP address of analyzer
of analyzer

Static ARP for 1Pv4 Port Mirroring


It is necessary to configure a static Address Resolution Protocol (ARP) entry for the analyzer on the interface that points
towards the analyzer. Adding a static ARP entry in this manner ensures that the mirrored packets reach the analyzer.

Chapter 12-46 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

lnline Port Mirroring


• lnline port mirroring aims to provide a large number of
port mirroring instances per chassis
• Achieved by decoupling port mirroring destination from input
parameters
• Port mirroring child instance can inherit input parameters
from port mirroring parent instance
• Only available on MX 30 Trio based line cards

lnline Port Mirroring


The implementation of port mirroring that has been discussed in the previous slides has a limitation in that the port
mirroring configuration is a global configuration, which affects all traffic that passes through the device. Inline port mirroring
aims to overcome this limitation by providing port mirroring instances.
This goal is achieved by decoupling the port mirroring destination from the input parameters. Then, you can configure
multiple child port mirroring instances that inherit input parameters from a parent port mirroring instance.
Inline port mirroring is only available for MX 30 platforms with Trio based line cards.

www.juniper.net Network Monitoring • Chapter 12-4 7


Junos Troubleshooting in the NOC

Configuring Port Mirroring Instances


[edit forwarding-options port-mirroring instance]
user@route�i show
paren t-I {
input {
Parent instance
rate l;
contains input
fa'Ilily inet arameters
output {
interface ge-0/0/1. 0 {
next-hop 10.10.10.100;

child-1
- _ _u_t __p_
�µn _e_r_s- --n
_ a_r-am-,et- p_a_r_e_n_t__l_,;
1-· _s_t_a_n_c_e__
_ ,
p
fa..:rnily inet { Child instance
output {
refers to parent
interface ge-0/0/2.0 {
next-hop 10.20.20.100;
instance for input
parameters

Configuring Port Mirroring Instances


You can begin configuring port mirroring instances under the [edit forwarding-options port-mirroring
instances] hierarchy level. As the slide depicts the parent instance contains input parameters, then the child instance
uses the input parameters of the parent instance. Note that the parent instance on the slide also contains output
parameters because you can use the parent instance to port mirror traffic. However, the output parameters for the parent
instance are optional if you do not plan to use the parent instance to port mirror traffic.

Chapter 12-48 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

lnline Port Mirroring Chassis and Firewall


Filter
• Configure a firewall filter with
port-mirror-instance action
• Then, apply firewall filter to necessary interface
[edit firewall family inet]
lab@mxA-li show
filter child-filter {
term 1 {
then port-mirror-instance child-1;

• Bind parent instance to an MX 30 Trio FPC


[edit chassis]
lab@rnxA-li show
fpc O (
port-mirror-instance parent-1;

Chassis and Firewall Configuration


You must configure a firewall filter that has the action of port-mirror-instance. Although it is not shown on the slide,
you can add match criteria with the from option to selectively choose traffic to port mirror. Then, you must apply the firewall
filter to the necessary interface.
Finally, you must attach the parent instance to an MX 3D Trio FPC. The slide depicts the parent port mirroring instance
parent-1 being bound to FPC 0. This action allows the child-1 port mirroring instance to use the input parameters of
the parent-1 port mirroring instance.
Note that although it is not shown on the slide, you must still configure a static ARP entry for the analyzer, as we discussed in
a previous slide, when using inline port mirroring.

www.juniper.net Network Monitoring • Chapter 12-49


Junos Troubleshooting in the NOC

Verifying Port Mirroring


• Displaying the current state of the port mirroring
instance
user@router> show forwarding-options port-mirroring
Instance Name: cl
Instance Id: 2
Input parameters:
Rate 1
Run-length O
Maximum-packet-length O
output parameters:
Family State Destination Next-hop
inet up ge-1/0/9.0 10.10.10.100

Verifying Port Mirroring


To verify the state and operation of a port mirroring instance, issue the show forwarding-options
port-mirroring command as the slide depicts.

Chapter 12-50 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Summary
• In this content, we:
• Explained how to configure and monitor SNMP
• Discussed how to configure and monitor RMON
• Described how to use flow monitoring

We Discussed:
How to configure and monitor SNMP;
How to configure and monitor RMON; and
How to use flow monitoring.

www.juniper.net Network Monitoring • Chapter 12-51


Junos Troubleshooting in the NOC

Review Questions

1. Under which configuration hierarchy level is port


mirroring configured?
2. When packet sampling is employed on the router,
which packet information is sent to the JFlow
collector?
3. How is an SNMP inform message different than an
SNMP trap message?

Review Questions
1.

2.

3.

Chapter 12-52 • Network Monitoring www.juniper.net


Junos Troubleshooting in the NOC

Monitoring the Network Lab

• Configure and monitor SNMP.


• Implement RMON.
• Configure and monitor packet sampling.
• Implement inline port mirroring.

Monitoring the Network Lab


The slide provides the objectives for this lab.

www.juniper.net Network Monitoring • Chapter 12-53


Junos Troubleshooting in the NOC

Answers to Review Questions


1.
Port mirroring is configured under the [edit forwarding-options port-mirroring] hierarchy level.
2.

The information contained in the packet header is sent to the JF!ow collector
3.
An SNMP inform message requests a response from the NMS, whereas an SNl'vfi> trap does not request a response from the NMS.

Chapter 12-54 • Network Monitoring www.juniper.net


JUnlf2v�f
Junos Troubleshooting in the NOC

Chapter 13: JTAC Procedures


Junos Troubleshooting in the NOC

Objectives

• After successfully completing this content, you will be


able to:
• Follow recommended procedure to open a JTAC support
case
• Access support resources
• Use the customer support Web site
• Access and use customer support tools
• Use FTP to transfer large files to JTAC

We Will Discuss:
How to open Juniper Networks Technical Assistance Center (JTAC) support cases;
How to access support resources;
How to access and use customer support tools; and
How to transfer large files to JTAC.

Chapter 13-2 • JTAC Procedures www.juniper.net


Junos Troubleshooting in the NOC

Agenda: JTAC Procedures


�Opening a Support Case
• Customer Support Tools
• Transferring Files to JTAC

Opening a Support Case


The slide lists the topics we will discuss. We discuss the highlighted topic first.

www.juniper.net JTAC Procedures • Chapter 13-3


Junos Troubleshooting in the NOC

Support Entitlement
• JTAC support cases is only offered to customers with
a valid maintenance contract
• A chassis serial number is required when opening a case so
support status can be determined
• Details at: http:j/www_juniper_netjsupport/requesting­
support_html

Support Entitlement
This slide stresses that Juniper Networks offers support only to customers with valid maintenance contracts, and that you must
provide a chassis serial number when opening a support case so that your support status can be verified. Use a show
chassis hardware command to obtain your chassis serial number:
user@m7i> show chassis hardware
Hardware inventory:
Item Version Part number Serial number Description
Chassis 37079 M7i
Midplane REV 04 710-008761 CK8606 M7i Midplane
Power Supply 0 Rev 05 740-008537 5240478 AC Power Supply
Routing Engine REV 01 740-011202 1000596020 RE-850
CFEB REV 06 750-010464 CL0791 Internet Processor II
FPC O E-FPC
PIC 0 REV 11 750-002992 CL1404 4x F/E, 100 BASE-TX
PIC 1 REV 10 750-002971 CL0236 4x OC-3 SONET, MM
FPC 1 E-FPC
PIC 2 REV 07 750-009487 CL0970 ASP - Integrated (Layer-2-3)
PIC 3 REV 09 750-009098 CK7181 2x F/E, 100 BASE-TX

Chapter 13-4 • JTAC Procedures www.juniper.net


Junos Troubleshooting in the NOC

Opening a Case

• Can open cases several ways:


• Via the Web using Case Manager at:
https:j/www.juniper_netjcm/
• Via phone:
• 1-888-314-JTAC (U.S. and Canada customers)
• Phone: +1 408-745-9500 (all other customers)
• For Priority 1 cases, contact JTAC by phone
• Documentation:
• All symptoms should be illustrated with relevant command
output
• The request support information command
provides most of the information needed

Opening a Case
Customers can open support cases using the Case Manager website, or over the telephone. Note that you should always follow
up a Priority 1 case with a phone call to JTAC, even when you opened the case using Case Manager.

Documenting the Problem


When you open a case, you should provide documentation of the problem or symptom by capturing the output of relevant
command-line interface (CU) commands. You might consider also capturing the output of a request support
information command; this command is actually a macro that executes a large number of operational-mode commands.
While the output is lengthy, adding this information might prevent the need to follow up with additional command output once
your case is being analyzed.

www.juniper.net JTAC Procedures • Chapter 13-5


Junes Troubleshooting in the NOC

Case Management
• Case management details are provided at:
https:j/www.juniper.net/support/guidelines.html
• Definition of case priorities, escalation, support levels, RMA
handling, standard warranty, and gray market inspection
procedures
• Case priority is mutually agreed upon between customer and JTAC:
Priority 1 = Critical. Priority 2 = High, Priority 3 = Medium.
Priority 4 = Low
• Escalation management defines systematic escalation to
ensure that the appropriate resources within Juniper
Networks are used to resolve outstanding technical
problems as efficiently as possible
• Manual escalation available as well (KB17701)

Case Management
Case management details and procedures are provided at https://www.juniper.net;support;guidelines.html. This document
defines case priority levels, escalation management, warranty information, and so forth.
When you contact JTAC, a member of our support staff will work with you in assigning mutually agreeable priority levels to your
problem that will be reflected in the support case opened on your behalf. The priority levels are the following:
Priority 1 (Critical): Catastrophic impact to business operations. Examples of Priority 1 issues include network or
system outage that cause customers to experience a total loss of service.
Priority 2 (High): Significant impact to business operations. Examples of Priority 2 issues include network or system
events that cause intermittent impact to end customers.
Priority 3 (Medium): Limited impact to business operations. Examples of Priority 3 issues include network events
that result in only limited impact to end customers.
Priority 4 (Low): No impact to business operations. Examples of Priority 4 issues include information requests.
Juniper Networks offers systematic escalation management to customers with current service agreements. This escalation
management ensures that the appropriate resources within Juniper Networks are used to resolve outstanding technical
problems as efficiently as possible.
If you feel the problem is not being given the appropriate level of attention, you also have the option to manually escalate
your case using the Escalate This Case link on the case manager site. You will be prompted to list a reason for the
manual escalation. If the case is urgent, it is strongly recommended to call JTAC where you will be able to speak with an
escalation manager.

Chapter 13-6 • JTAC Procedures www.juniper.net


Junos Troubleshooting in the NOC

Gaining Access to Support Services


• Request a login at
www.juniper.net/customers/support
1
JOnte.�( SearCl'!
a

•·1·"
<<< Please select your product first or se-!ect the By Task tab.

Suppon Tool'5

• Requires valid maintenance contract Request an


Account Management

Requesting a Support Login


You must have a valid support login to access Juniper Networks support services over the Web. Point your browser to
www.juniper.net/customers/support to display the login screen. Note that below the login, you can click links to request an
account or to manage the passwords associated with your existing support account.

www.juniper.net JTAC Procedures • Chapter 13-7


Junos Troubleshooting in the NOC

Agenda: JTAC Procedures


• Opening a Support Case
"'?Customer Support Tools
• Transferring Files to JTAC

Customer Support Tools


The slides highlights the topic we discuss next.

Chapter 13-8 • JTAC Procedures www.juniper.net


Junos Troubleshooting in the NOC

Customer Support

• Customer support site provides access to JTAC tools

Er�;.1y,tc,;1 A;re,•-r.;:D'
SUJ)t'k)n Sih,m.:r

Customer Support Site


This slide displays the top of the main customer support site. From this page you can easily link to case management and
technical research. The primary research tools include the knowledge base, problem reports database, technical
documentation, J-Net discussion forums, and technical bulletins. The page also includes a link to download software.

www.juniper.net JTAC Procedures • Chapter 13-9


Junos Troubleshooting in the NOC

Case Manager

• Use www.juniper.net/cm for case creation and


management CASE MANAGEMENT HOME
With Case Manager you can create new cases. check the status cf existing cases. update
numbers. and your own internal case reference numbers. iNithin �.dvanced Search �-ou can query cases by date.
Search for a Caset RMA or Customer Tracking t�umber

0 Case Mumber O Rr,.1.t.. Number O Customer Tracking Number

m:,c Advancec: Case Search


P.dvanced RI.IA Searc."1
{Include ant dashes)

Create a Case Open Technical Support Cases [O


Open Customer Care Cases [Oi
Open RMAs IOJ

Technical Support Cases Customer Care Cases RMAs


Open Cases Open cases Open RJ,l•s
Open Priority-1 Cases Created Last 30 Days Opened Last 30 Days
Created Last 30 Days Closed Last 30 Days Closed Last 30 Days
Closed Last 30 Oars

Subscribe notifications

Case Manager
Many customers prefer to open and manage their cases with the point-and-click ease offered by the Case Manager (CM)
tool. In addition to opening a case, the CM provides a handy way of tracking the status of your open cases, Return Materials
Authorization (RMA) status, and so forth.

Chapter 13-10 • JTAC Procedures www.juniper.net


Junos Troubleshooting in the NOC

The Juniper Knowledge Base (1 of 2)


• Use kb.juniper.net to research your issue
Solutions Products & Ser1ices Company Partners Education

Home � Support> t<.'8 Hom�

LOGGED IN:

Proc!uctlSeries: Uy Ac:ount i Logout


My KB Proma
Question or KB 10:
.�f.���.:. 1.!.(?.�..�?�:.�:.�.�:�.��.��?.�.�-�-�..�.�-�-�:..!.�.S..t.!!.1.���-1.?.�..i�._i_�}?..�� J
query
VFll Cor.figura1icn
0·o�r��-1i'S�-�-��h'(CS"c'"�·�·;�·t��1·;·��;;;�·;�·1· KB')'"
O Include KB Archrve (for EOL Products} ALERTS
Unread:
O Search on!y KB A.rctiive Hew are we doinQ? Please take
Search Tips 111e 60-secor.d Online Suppo11
Survey

Unread:
, Junos OS Release 12.2 is not
Popular Articles Recently Updated Articles Read r-.,e supported on S Series, J
Series, LNiOOO, and \l\'XC·!Sf,l·
Status 10 Popular Articles 200
Unreao KB7963 Copy tito;s from cn,2 lcc.3t1on to Jnother on a Roullng Engine Unread:
Unread KB11615 Ra1Jting Engine cper2tion and troubleshooting Prcduct Support Tools. Feature
E."";fplorer and Content E;<plorer

Knowledge Base
The knowledge base (KB) located at http://kb.juniper.net provides a key support tool for researching issues or technical
questions. This slide provides an example of a KB search using the keywords VPN Configuration. You can see that the results
are displayed at the bottom of the page.

www.juniper.net JTAC Procedures • Chapter 13-11


Junos Troubleshooting in the NOC

The Juniper Knowledge Base (2 of 2)

• What is the Knowledge Base?


• A searchable repository of useful information including:
1. Product documentation
2. Knowledge Base articles (including troubleshooting guides)
3. Technology notes and whitepapers
• The use of the KB can be useful for troubleshooting and
problem resolution, as well as when testing new features
• Searching the Knowledge Base
• The KB database can be searched using the KB number,
keywords. or natural language queries
• Results can be narrowed down by product (such as routers,
switches, security products) or by source (such as
documentation, KB articles. marketing)

What Is the Knowledge Base?


The content of the Knowledge Base (KB) includes not only the Juniper documentation, but also custom-made articles like
configuration and troubleshooting guides, as well as technology whitepapers. It is a very useful tool when troubleshooting a
problem, or when trying out new complex features. The whitepapers and technology notes usually include full scenarios with
step-by-step configuration guides.

Searching the Knowledge Base


The Knowledge Base can be searched using the KB number, keywords, or natural language queries. Finding the right query
can take a few attempts. If too many results are found, it is very easy to restrict the result set on the basis of the platform or
the type of information requested. For troubleshooting, KB articles (specifically solution guides) are very useful. When trying
new features, both whitepapers or configuration examples from the documentation guide can save you a lot of time. Each KB
article has an ID number that can be used to refer to it.

Chapter 13-12 • JTAC Procedures www.juniper.net


Junos Troubleshooting in the NOC

PR Searches (1 of 3)
• What is a PR?
•APR is the description of a hardware or software defect
• The PR lifecycle is part of the Juniper Networks quality
assurance and software development process
1. A PR is opened and assigned a number for tracking purposes
2. JTAC makes sure Engineering has all the information to understand
and resolve the issue: if necessary. JTAC handles lab reproduction
and on-site troubleshooting
3. Engineering resolves the problem. and asks for verification from
JTAC and System Test (quality assurance). The PR's internal state is
then moved from open to feedback state
4. After the fix is verified. the PR is moved into close state. If the fix is
not complete. the PR is re-opened and sent to Engineering again

What Isa PR?

A problem report (PR) is the description of an issue, typically a software issue, that must be addressed by the Juniper
Engineering team. The lifecycle of a PR is part of the Juniper software and hardware development and quality assurance
processes. The steps shown on the slide are a subset of the full PR lifecycle, but list the major steps.

www.juniper.net JTAC Procedures • Chapter 13-13


Junos Troubleshooting in the NOC

PR Searches (2 of 3)
• Structure of a PR
•APR can be summarized into these main fields (some are
optional):
• Number: A unique ID assigned by the Juniper defect tracking
system
• Title: A title of the problem report
• Release note: A summary description for inclusion in the release
notes of affected Junos OS versions
• Severity: This value can be minor (cosmetic or no impact). major
(potential service impact). or critical (must be addressed
immediately)
• Trigger: What causes the problem to manifest itself
• Workaround: When possible, a way to bypass the issue. or prevent
it from having service impact
• Status: Open (still not fixed) or closed (resolved)
• Resolved-in: Releases under which the problem is resolved

The Content of a PR
Among the fields that compose a PR, of particular interest are trigger and workaround; together, they can help you decide
the best way to minimize the impact of a software issue until the network can be upgraded to a release that is not affected.
In addition to those shown on the slide, a PR can also contain a number of additional fields. The full list is available on the
help page of the PR search Web interface.

Chapter 13-14 • JTAC Procedures www.juniper.net


Junos Troubleshooting in the NOC

PR Searches (3 of 3)
• Searching for a known software issue
• Known PRs are accessible using the Juniper Web interface
• You can search using a PR number or keywords
• Website access requires a valid login
• Open PRs affecting a Junos OS version are included
automatically in the Junos OS release notes
• Reviewing the release notes before deploying a new release is an
excellent addition to network operation procedures
• Tips for an efficient PR search:
• Be specific: if you do not get results. remove keywords
• Use the content log messages related to the failure-even if they
are not visible in the PR
• You can use quotes to search for a specific phrase
• You can use + and - to force keyword inclusion or exclusion.
respectively

Searching for a Known Software Issue


PRs are accessible and searchable using Juniper's Web interface using either the PR number or using keywords. The list of open
PRs affecting a given Junos release are added automatically to the version's release notes, which are constantly updated as
new problems are discovered and known problems are fixed. It is best practice, before a software rollout, to carefully examine
the open issues section in the release notes. You might avoid known problems or reduce their impact before they affect you.
Keep the following tips in mind when performing a PR search:
Be specific at first: Getting too many results means having to go through all of them, trying to understand if any
of them might describe your problem. If you do not find get adequate results, you can easily remove keywords to
expand your search.
Use log entries related to the failure: The PR index includes related log entries. Do not cut-and-paste from log
files into the search form. Remove hostnames and numerical values that might be specific to your router.
Search for an exact phrase by using quotes.
Use the + and - operators to force inclusion or exclusion of keywords: The - (exclusion) operator is the most
useful because it allows you to remove unwanted entries easily.

www.juniper.net JTAC Procedures • Chapter 13-15


Junos Troubleshooting in the NOC

Additional Support Tools


sou:mous PRODUCTS & SE.RVlCES COMPANY FARTtlE.RS

Translates IOS
configurations to Junos
configurations

Search known software


bugs

Cti,3SS!S Aoal;z<::r 10s to Junes Tr,1ns1atljr


F1re�v3U SElssi0n Anai;:.:er JLIWJSe to Juna::; Tr.3ni1�t,:ir
& Get A3:sistence Proor�m Report 1PR; Search SueenOS to. Juno�> Tr=3nslcitor
Validates FPC/PIC Junos Hardware Validattin More Translation Tools...
hardware combinations Proclucts & licenses • S�cure Artet:.s Si:.':'ing C:':llcu!ator
Junos VP�J ConHaurafion Tool
(BelaJ

Generate IPsec
configurations for SRX
Series devices

Additional Support Tools


Clicking the By Task tab at the main support site provides access to a number of helpful support tools, including:
Chassis Analyzer: Paste the output of the Junos show chassis hardware command using this tool and you
are provided a table which provides a separate listing of each hardware component, including each part
number, serial number, and hardware revision number. This information can also be downloaded as a list of
comma-separated values useful for inventory control.
Firewall Session Analyzer: This tool analyzes get session outputs from ScreenOS products and returns a
firewall session report including the top ten destination and source ports in use, destination and source IP
addresses accessed and protocols in use.
Problem Report Search: Use this tool to search Juniper Networks database of known hardware and software
bugs.
Junos Hardware Validation: This tool checks the validity of Flexible PIC Concentrator (FPC) and PIC
combinations for M Series and T Series routers. You can either upload the output of show chassis
hardware or enter the FPC and PIC part numbers manually to ensure your hardware configuration is valid and
not violating system memory constraints.
Continued on the next page.

Chapter 13-16 • JTAC Procedures www.juniper.net


Junes Troubleshooting in the NOC

Additional Support Tools (contd.)


Secure Access Sizing Calculator: The Secure Sockets Layer (SSL) virtual private network (VPN) sizing calculator
provides customers and partners with a fast and convenient way to determine the overall target scalability of
SA6500 platforms.
Junos VPN Configuration Tool: This tool provides graphical interface for manually entering information used in
site-to-site IP Security (IPsec) VPN implementations using SRX Series and J Series devices. Once you enter all
required information, the tool will output a configuration file which can be pasted into your Junes security
device.
!OS to Junos Translator: Input an Internetwork Operating System (IOS) configuration and this handy tool will
translate and return a matching Junes configuration. Note that not all configuration options are supported.
Here is an example IOS firewall configuration snippet and the resulting translation:
IOS:
ip access-group 102 in
access-list 102 permit icmp host 10.194.0.14 host 220.232.29.225
access-list 102 permit ip any any
Junos:
firewall
family
inet
filter 102 {
term Tl {
from {
protocol icmp;
source-address
10.194.0.14/32;

destination-address
220.232.29.225/32;

then {
accept;

term T2
then {
accept;

JUNOSe to Junos Translator: Similar to the IOS translator, this tool translates ERX Series configurations to Junos
configurations.
ScreenOS to Junes Translator: This tool translates ScreenOS configurations to Junes configurations used on
SRX Series and J Series devices.

www.juniper.net JTAC Procedures • Chapter 13-17


Junos Troubleshooting in the NOC

Agenda: JTAC Procedures


• Opening a Support Case
• Customer Support Tools
�Transferring Files to JTAC

Transferring Files to JTAC


The slide highlights the topic we discuss next.

Chapter 13-18 • JTAC Procedures www.juniper.net


Junos Troubleshooting in the NOC

JTAC Uses SFTP for Large File Transfers


• Use SFTP when transferring files larger than 10 MB
• Attach smaller files directly to case using Case Manager
• FTP procedures are specified at:
http://kb.juniper.net/library/CUSTOMERSERVICE/GLOBAL JT
AC/Bk23337 /sftp-instructions.pdf
• KB23337
• Files are uploaded to sftp.juniper.net
• Create new directory under /pub/incoming named with case ID
• Use anonymous for user and password

Transferring Large Files to JTAC


You may have a need to transfer a large file to JTAC, such as a core dump file. Core files should always be submitted to JTAC for
fault analysis. A file larger than 10 MB cannot be attached to a case in case manager. The recommended procedures for
transferring core files to JTAC are outlined on the slide and described here:
1. First, open a support case and obtain a case number.
2. Escape to a root shell and change to the directory containing the core file.
3. Optionally, rename (or copy) the file using a name in the form of case number-core-sequence number.
4. Although not strictly necessary, we recommend that you chmod the core file with 444 to ensure that all users (root,
owner, and other) have read permissions for the file.
5. In some cases, the core file will already be compressed, as indicated by the file having a . tgz or . gz file
extension. If the file is not already compressed, you should compress the file to reduce transfer and storage
requirements. This is especially important when dealing with the vrncore. O file associated with a kernel crash
because this memory image file can be quite large.
6. Log into the Juniper Networks anonymous SFTP site at
sftp. juniper. net, and change into the /pub/ incoming directory.
7. Ensure that your FTP client is set for a binary transfer. In many cases the client defaults to the correct transfer type.
Issue a type command to confirm the current transfer setting and use the image or binary command to enable
binary transfer mode as needed.
8. Upload the compressed and renamed core or memory image file using a put or mput command.

www.juniper.net JTAC Procedures • Chapter 13-19


Junos Troubleshooting in the NOC

SFTP Example
• Uploading large files to JTAC

user@mx> sftp anonymous@sftp.juniper.net


anonyrnous@sftp.juniper.net's password:
Connected to sftp.juniper.net.
sftp> led /var/tmp
sftp> cd /pub/incoming
sftp> mkdir 2013-0120-1189
sftp> cd 2013-0120-1189
sftp> mput large-file.tgz
Uploading large-file.tgz to /pub/incoming/Z013-01Z0-
1189/large-file.tgz
large-file.tgz 100% 131MB
3.6MB/s 00:37
sftp> bye
bye

SFTP Example
The slide illustrates the procedure for uploading a large file to JTAC directly from a Junos device.

Chapter 13-20 • JTAC Procedures www.juniper.net


Junos Troubleshooting in the NOC

Summary
• In this content, we:
• Learned the recommended procedure to open a JTAC
support case
• Described support resources
• Described the customer support Web site
• Explained how to access and use customer support tools
• Used FTP to transfer large files to JTAC

We Discussed:
How to open JTAC support cases;
How to access support resources;
How to access and use customer support tools; and
How to transfer large files to JTAC.

www.juniper.net JTAC Procedures • Chapter 13-21


Junes Troubleshooting in the NOC

Review Questions

1. What information do you need to open a support


case?
2. What aspects of the customer support site can help
you research a problem?
3. Describe the purpose of the IOS to Junos translator.
4. When should you use SFTP to transfer files to JTAC?

Review Questions
1.

2.

3.

4.

Chapter 13-22 • JTAC Procedures www.juniper.net


Junos Troubleshooting in the NOC

Answers to Review Questions


l.
A customer support login is required to open a support case. At a minimum, you will need the chassis serial number of a device
involved in the problem. The ourput of request support information provides most of the information initially required by JTAC.
2.
There arc several tools available to assist in researching problems including the knowledge base, problem report search, and technical
docmnentation.
3.
The !OS to Junos translator allows you to paste or upload an !OS configuration into the tool. The tool will translate the configuration
and rcnu:n a matchingJunos device configuration which can be pasted on ajunos device or uploaded to aJw10s device.
4.
Any file larger than 10 MB should be uploaded toJTAC using SFrP. Most files of this size consist of core dump files.

www.juniper.net JTAC Procedures • Chapter 13-23


Junes Troubleshooting in the NOC

Chapter 13-24 • JTAC Procedures www.juniper.net


Junos Troubleshooting in the NOC

Resources to Help You Learn More

http//www.juniper.net;techpubs/content­
Content Explorer
applications/content-explorer
Feature Explorer http//pathfinder.juniper.netjfeature-explorer
Learning Bytes www.juniper.net;learningbytes
Installation and
www.juniper.net;courses
configuration cours
http://forums.juniper.net;t5/Training-Certification­
J-Net Forum
Certification
Certification program
Courses //www.juniper.net;training/technical_education
Translation tools http//www.juniper.net;customers/support/#task

Resources to Help You Learn More


The slide lists online resources available to learn more about Juniper Networks and technology.
These resources include the following sites:
Pathfinder: An information experience hub that provides centralized product details.
Content Explorer: Junos OS and ScreenOS software feature information to find the right
software release and hardware platform for your network.
Feature Explorer: Technical documentation for Junos OS-based products by product,
task, and software release, and downloadable documentation PDFs.
Learning Bytes: Concise tips and instructions on specific features and functions of
Juniper technologies.
Installation and configuration courses: Over 60 free Web-based training courses on
product installation and configuration Uust choose elearning under Delivery Modality).
J-Net Forum: Training, certification, and career topics to discuss with your peers.
Juniper Networks Certification Program: Complete details on the certification program,
including tracks, exam details, promotions, and how to get started.
Technical courses: A complete list of instructor-led, hands-on courses and self-paced,
elearning courses.

Translation tools: Several online translation tools to help simplify migration tasks.

www.juniper.net
Junos Troubleshooting in the NOC

www.juniper.net

You might also like