1.

Troubleshooting IP Network Connectivity

a. General process:
i. Host wants to transmit frame
ii. Checks to see if it is local or remote request
1) Local: Find host via ARP broadcast (MAC address)
2) Remote: Find gateway via ARP broadcast (MAC address)
iii. Going down the remote path:
1) Host finds default gateway address
2) Host will frame each packet-to-be-transmitted with Data
Link layer header & trailers
3) Host sends framed packet into local collision domain
4) Router receives frame and removes packet from frame (de-
encapsulate)
5) Router uses IP to parse routing table and look for exit
interface
6) Router packet-switches packet to exit interface
7) Router frames packet with new source and destination MAC
address before sending
b. Example Troubleshooting Scenario:
i. Issue: Cannot log onto Server1 (172.16.20.254) from PC1
(10.1.1.10)
ii. Procedure
1) Check cables
a) Determine if there is a faulty cable or interface
i) Verify interface statistics
2) Make sure devices are determining correct path from
source to destination
a) Check routing information
3) Verify that default gateway is correct
4) Verify DNS settings are correct
5) Verify that there are no ACLs blocking traffic
iii. Steps for checking PC1 configuration:
1) Test local IP stack is working by pinging loopback
address
2) Test local IP stack is talking to data link layer (LAN
driver) by pinging local IP address
3) Test host is working on LAN by pinging default gateway
4) Test that host can get to remote networks by pinging
remote server 1
iv. Analyze interface statistics on Cisco Router
1) Speed and duplex settings
a) Mismatched duplex or speed
i) Mismatched speed: wont connect
b) Mismatched duplex: input/output errors and late
collisions
2) Errors:
a) Input queue drops
i) More traffic is being delivered to the
router than it can process
ii) See why counter is increasing and how
events relate to CPU usage
iii) Throttle and ignore counters will also
increment
b) Output queue drops
i) Packets dropped to interference or
congestion, which leads to queuing delays
ii) Affects applications like like VOIP
iii) Can implement QoS if constantly
incrememnting
 c) Input errors
i) High errors such as CRC
ii) Cabling problems, hardware issues or duplex
mismatches
d) Output errors
i) Total number of frames that port tried to
transmit when an issue such as a collision occured
v. Note: Can check if server is responding to HTTP request by
telnet in via port 80
2. Using IP SLA for Troubleshooting
a. IP service-level agreements
i. User IP SLA ICMP echo to test far-end devices instead of
pinging manually
b. Reasons to use:
i. Edge-to-edge network availability monitoring
1) Packet loss statistics
ii. Network performance monitoring, network performance
visibility
1) Network latency and response time
iii. Troubleshooting basic network operation
c. Configuration
i. R1 (config)# ip sla 1
1) Enable IP SLA operation, choose operation number (1-2.1
billion)
ii. R1 (config-ip-sla)# icmp-echo 172.16.20.254
1) Configure ICMP echo test and destination
iii. R1 (config-ip-sla)# frequency 10
1) Set test frequency in seconds
iv. R1 (config-ip-sla)# exit
v. R1 (config)# ip sla schedule 1 life forever start-time now
1) Configure schedule entry
2) Life in seconds or continuing forever
3) Start time in
a) After certain amount of time
b) Hh:mm
c) Hh:mm:ss
d) Now
e) Pending
d. Verification
i. R1# show ip sla configuration
1) Entry number
2) Target and source address
3) Operation frequency
4) Life
ii. R1# show ip sla statistics
1) Latest operation and return code
2) Number of successes
3. Using SPAN for troubleshooting
a. History:
i. Traffic sniffers attached to hubs were able to intercept all
traffic on a network
1) Hubs repeat incoming signals from all ports except
receiving one
b. Issue:
i. Modern (switched) networks forward traffic out of single port,
sniffer cannot intercept unicast traffic
c. Solution:
i. Implement SPAN
d. What does it do?
 i. SPAN copies traffic from designated incoming/outgoing source
ports
1) Ingress/egress traffic
ii. SPAN sends copied traffic through designated destination
ports for analysis
e. Configuration example (capture traffic flow from PC1 to PC2):
i. S1 (config)# monitor session 1 source interface f0/1
1) Associate SPAN session with source port that will be
monitored
ii. S1 (config)# monitor session 1 dest interface f0/2
1) Associate SPAN session with the destination interface
f. Verification
i. S1 (config) #do sh monitor
1) Type, source/dest ports, encapsulation, ingress
4. Configuring and verifying extended ACLs
i. Remember, ACL placement:
1) Standard: Destination
2) Extended: Source
ii. Extended ACLs filter based on
1) Port number
2) Protocol
3) Source address
4) Destination address
a. Sample ACL configuration
i. R1# telnet 172.16.20.254
1) Test to see if you can telnet to remote host
ii. R1 (config)#ip access-list extended Block_Telnet
1) Note: name is case sensitive when applying to an
interface
iii. R1 (config-ext-nacl)# deny tcp host 10.1.1.1 host
172.16.20.254 eq 23
iv. R1 (config-ext-nacl)# Permit ip any any
v. R1 (config)# int fa0/0
vi. R1 (config-if)# ip access-group Block_Telnet in
vii. Note: this sample configuration is incorrect and fixed in a
later configuration example
b. Verification
i. Show access-list
c. Edit Sequence numbers
i. R1 (config)# ip access-list extended Block_Telnet
ii. R1 (config-ext-nacl)# no 10
iii. R1 (config-ext-nacl)# 10 deny tcp host 10.1.1.10 host
172.16.20.254 eq 80
5. Troubleshooting IPv6 Network Connectivity
a. ICMPv6
i. In IPv4, ICMP is a separate layer 3 protocol
ii. ICMPv6 is an integrated part of IPv6
1) Carried after basic IPv6 header information as an
extension header
2) Used for router solicitation and advertisement
3) Used for neighbor solicitation and advertisement
a) Finding MAC addresses for IPv6 neighbors
4) Used for redirecting host to the best router (default
gateway)
b. NDP (Neighbor Discovery)
i. Replacement for ARP
1) Used to find address of other devices on local link
ii. Replaces IGMP from IPv4
1) Used in IPv4 by host device to tell local router that it
would like to join a multicast group and receive traffic for that group
2) Renamed multicast listener discovery in ICMPv6
iii. Achieved via solicited node address (multicast address)
1) All hosts join solicited node address upon connecting to
a network
iv. NDP enables these functions:
1) Determining MAC address of neighbors
2) RS - FF02::2
3) RA - FF02::1
4) NS
5) NA
6) DAD
v. Issue with IPv4:
1) Only one default gateway can be configured on a host
2) If default gateway goes down host loses connectivity
without intervention
vi. Solution with IPv4: virtual default gateway
vii. Solution with IPv6: Router solicitation and Router
Advertisement
1) IPv6 devices can find their default gateway using
neighbor discovery
2) Send RS onto data link to multicast address FF02::2
3) Routers on same link respond with unicast message to
host
4) If cannot respond with unicast, then send RA using
FF02::1
viii. Hosts can also send solicitation and advertisements to each
other
ix. Note:
1) RS and RA gather or provide information about routers
2) NS and NA gather or provide information about hosts
3) "Neighbor" refers to a host on the same data link or
VLAN
x. General troubleshooting procedure:
1) Check cables and interface for faulty components. Very
interface statistics.
2) Make sure devices are determining correct path from
source to destination. Manipulate routing information if needed.
3) Verify that default gateway is correct
4) Verify DNS settings are correct and DNS server is
reachable via IPv4 and IPv6
5) Verify that there are no ACLs that are blocking traffic
xi. Note:
1) ::1 - IPv6 loopback
2) Default gateway will be link local address of the router
with %# being the connected interface
a) Example: FE80::21a:6dff:fe37:a44e%11
3) Temporary IPv6 addresses
a) 2001:db8:3c5d:3:2f33:44dd:211:1c3d
i) Create global address for host without using
MAC address
ii) Generates random number for interface and
hashes it
iii) Privacy from EUI-64 format
iv) Windows feature
b) To disable:
i) Netsh interface ipv6 set global
randomizeidentifiers=disabled
ii) Netsh interface ipv6 set privacy state-
disabled
xii. Router verification:
1) R1# show ipv6 interface brief
2) R1# show ipv6 neighbors
a) States:
i) ICMP (Incomplete)
One. Address resolution being performed
on entry
Two. NS has been sent but neighbor
message has not yet been received
ii) REACH (reachable)
One. Positive confirmation has been
received
Two. Path to neighbor is functioning
correctly
iii) STALE
One. Interface has not communicated
within the neighbor reachable time frame
Two. Will return to REACH once neighbor
communicates
iv) DELAY
One. Occurs after STALE state
Two. No reachability confirmation
received within DELAY_FIRST_PROBE_TIME time
Three. Path was functioning but has not
communicated within neighbor reachable time frame
v) PROBE
One. Configured interface is resending a
NS and waiting for a reachability confirmation from neighbor
3) R1# show ipv6 route
xiii. Host verification
1) netsh interface ipv6 show neighbor
xiv. R1 (config)# ipv6 route ::/0 fastethernet 0/1
FE80::21A:6DFF:FE64:9B3
1) Static default route out interface fa0/1 to link local
address
xv. R! (config)# ipv6 route ::/0 fa0/1
1) Static default route out fa0/1
2) Can also just use next hop global address
6. Troubleshooting IPv6 Extended Access Lists
a. Note:
i. Named IPv6 ACLs will always be extended
1) Will not see sequence numbers
2) Can delete a line but can also only insert a line at the
end
ii. IPv4 ACLs always have an implicit deny ip any any after the
last command
iii. IPv6 ACLs have 3 implicit statements after the last command:
1) permit icmp any any nd-na
2) permit icmp any any nd-ns
3) deny ipv6 any any
iv. IPv6 will allow ND related traffic before denying any other
IPv6 traffic that has not already been explicitly allowed
b. Sample IPv6 ACL configuration
i. Steps:
1) R1# telnet 2001:db8:3c4d:1:a14c:8c33:2d1:be3d
a) Test that you can telnet into remote host from R1
2) R1 (config)# ipv6 host Server1
2001:db8:3c4d:1:a14c:8c33:2d1:be3d
 a) Create entry in host table for Server1 (do not
need to type address when trying to access)
b) Verify with "do sh host"
3) R1# telnet Server1
a) Test telnet using host name
4) R2 (config)# ipv6 access-list Block_Telnet
5) R2 (config0ipv6-acl)# deny tcp host
2001:dB8:3c4d:2:21a:6dff:fe37:a44f host 2001:db8:3c4d:1:a14c:8c33:2d1:be3d eq
telnet
6) R2 (config-ipv6-acl)# permit ipv6 any any
a) Creating test commands to block telnet access
7) R2 (config-ipv6-acl)# int fa0/1
8) R2 (config-if)# ipv6 traffic-filter Block_Telnet ni
ii. Note: This configuration is designed to block incoming
traffic on R2's Fa0/1 interface, from R1's connected interface which has the
destination address of Server1, directed toward its telnet port
1) Telnet should not be blocked through ACLs applied to an
interface
2) Telnet should be blocked under the VTY config
7. Troubleshooting VLAN Connectivity
a. General steps:
i. Verify VLAN database on all switches
ii. Verify CAM table
iii. Verify that port VLAN assignments are configured correctly
b. Verification commands:
i. show vlan
ii. show mac address-table
iii. show interfaces [interface] switchport
8. Trunk Troubleshooting
a. General steps:
i. Verify that the interface configuration is set to the correct
trunk paramters
ii. Verify that the ports are configured correctly
iii. Verify the native VLAN on each switch
b. Verification commands
i. show interfaces [interface] trunk
ii. show vlan
iii. show dtp interface [interface]
c. General configuration commands:
i. switchport mode
ii. switchport mode dynamic
iii. switchport trunk native vlan [vlan]
iv. switchport access vlan [vlan]
d. Port configurations for DTP
i. Access - Trunking is not allowed on a port set to access mode
ii. Auto - Will trunk only if the remote port is set to on or
desireable
iii. Desirable - Will trunk with all port modes except access
iv. Nonegotiate - No DTP frames generated by interface. Can only
be used if remote interface is manually set as trunk or access
v. Trunk (on) - Will trunk with all modes except access.
Automatically enables trunking regardless of state of neighboring switch and
regardless of any DTP requests
e. Note:
i. For dot1q encapsulation, native VLANs much match
1) ISL will reject native vlan change requests
2) Native VLAN mismatch means untagged frames cant be sent
down link
a) Management frames such as CDP
i) Prevent remote management of switch