Professional Documents
Culture Documents
Troubleshooting Lab
LTRSEC-3021
Translated IP Real IP
logging enable
logging buffered debugging
!
! Optional
logging buffer-size 512000
logging timestamp
Syslogs – ASDM Configuration
• Enable Syslogs in ASDM from the Monitoring Tab Logging
• Specify the ASDM logging Level, and Buffer Limit, then View
Syslogs – Viewing in ASDM
Filtering Output with the CLI
• Filtering output is very useful when issuing ‘show’ commands, or when
searching though syslogs
• ASA supports:
include <regex match>
exclude <regex match>
grep <regex match>
grep -v <regex match>
begin <regex match>
• Example:
! Show the syslogs minus those we aren’t interested in
ASA# show log | exclude 609001|609002|710005
ASDM filter
accepts Regular
Expression
Patterns (regex)
show conn
• All traffic which passes through the ASA will create a connection
• show conn is used to view the ACTIVE ASA connection table
Client IP Server IP
Green means UP
Outside Inside
.2 .1
192.168.1.0/24 10.2.XX.0/24 10.3.XX.0/24
10.4.XX.0/24
Web Server
DMZ
10.3.XX.50
Important Notes
• The ASAs are in production, so only make changes to them that you
would make on your own production ASAs to restore network
connectivity.
• Use the Lab Guide if you get stuck, and work through the Helpful Steps
• After each lab, we will reset the configurations to a default state, and
you will be kicked out. You will lose your SSH/ASDM connection and
will need to reconnect.
LAB 1
• You have been hired as a civilian Networking Expert on a
Fast Attack Submarine. Your job is to ensure the network
for the sub is available at all times. The network controls
both the dive and attack functions of the sub, as well as all
other systems necessary for your survival.
Host NAT
object network obj-WebServer
host 10.3.19.50
nat (inside,outside) static 198.51.100.50
Network NAT
object network Servers
subnet 10.0.54.0 255.255.255.0
nat (inside,outside) static 203.0.113.0
nat (in,out) source static inLocal inGlobal destination static outGlobal outLocal
Static NAT
nat (inside,outside) source static ServerReal ServerTrans
Translate
Static - Twice NAT Source IP
nat (inside,outside) source static ServerReal ServerTrans destination static RemoteSite RemoteTrans
Translate
Destination IP
ASA 8.3+ NAT Troubleshooting
• Prior to 8.3, show xlate was the best command to use for troubleshooting NAT
issues.
• With the NAT changes introduced in 8.3, one should now use the
show nat detail command
• Allows for visibility of IPs/Networks within an object
show xlate vs. show nat detail
Real Translated
Pod19# show xlate
(UnMapped) IP
14 in use, 16 most used (Mapped) IP
Flags: D - DNS, i - dynamic, r - portmap, s - static, I - identity, T - twice
NAT from dmz:10.3.19.98 to outside:209.165.200.252
flags s idle 0:00:07 timeout 0:00:00
NAT from inside:10.3.19.20 to outside:209.165.200.225
flags s idle 0:00:07 timeout 0:00:00
NAT from inside:10.3.19.22 to outside:209.165.200.227
flags s idle 0:00:07 timeout 0:00:00
Pod19# show nat detail Real (UnMapped) Translated (Mapped)
Manual NAT Policies (Section Source
1) IP Source IP
1 (dmz) to (outside) source static obj-10.3.19.98 obj-209.165.200.252 destination
static obj-209.165.201.0 obj-209.165.201.0
translate_hits = 0, untranslate_hits = 0
Source - Origin: 10.3.19.98/32, Translated: 209.165.200.252/32
Destination - Origin: 209.165.201.0/24, Translated: 209.165.201.0/24
198.51.100.50 10.3.19.50
outside inside
Web Server
10.3.19.50
Packet Capture
• ASA packet capture provides built-in sniffer functionality pre- and post-Firewall
processing
• Allows one to see exact changes made on the packet passing through the ASA
• By default, full packet is captured. Optionally, one can specify the packet-
length, or headers-only (L2-L4)
• ACLs or Match criteria should be applied to limit captured traffic to what is
interesting
• ACLs are unidirectional, Match statements are bi-directional
• Multiple captures can be applied to the same interface, but a packet is only
captured once.
Packet Capture
• Captures can be viewed in real-time, but typically better to view in the buffer
• By default, when viewing packets in the buffer the ASA displays L3 and L4
information
• The detailed option adds L2 information
• The dump option displays packet contents (in ascii)
• Captures can be exported in pcap format
• A circular-buffer may be used to allow buffer wrap
• ASDM provides a nice capture wizard
ASDM Packet Capture Wizard
• Enable Packet Captures in ASDM from Wizards Packet Capture Wizard
Packet Capture – Real Benefit
• Allows one to validate if traffic:
1. Is being received by ASA
2. Being passed through, and transmitted out of the ASA
3. If reply traffic is returning to the ASA
4. If the reply traffic transits the ASA and transmitted out egress interface
1. 2.
4. 3.
.2 .1
192.168.1.0/24 10.2.XX.0/24 10.3.XX.0/24
Web Server
10.3.XX.50
Packet Capture – Best Practices
• Use short names for capture names. Preferably, an abbreviation of the
interface name where the capture is applied.
• Create two captures. One for ingress interface, one for egress interface.
(ASDM does this by default :-)
• Don’t apply the same capture to multiple interfaces.
• Stop a capture by removing it from the interface
• Use show capture to see how many bytes have been captured (if any)
ASA# show capture
capture in type raw-data interface inside [Capturing - 4674 bytes]
capture out type raw-data interface outside [Capturing - 74271 bytes]
capture man type raw-data interface management [Buffer Full - 522977 bytes]
Packet Capture – Lab info
• Be sure to filter out your SSH/ASDM connection using either ACLs or the
Match statement
-- OR --
capture out interface outside match tcp any host 10.3.XX.50 eq 80
Frame drop:
No valid adjacency (no-adjacency) 39
Flow is denied by configured rule (acl-drop) 5262
First TCP packet not SYN (tcp-not-syn) 331
TCP failed 3 way handshake (tcp-3whs-failed) 2
TCP RST/FIN out of order (tcp-rstfin-ooo) 2
Slowpath security checks failed (sp-security-failed) 129
Interface is down (interface-down) 8
Non-IP packet received in routed mode (non-ip-pkt-in-routed-mode) 1
Flow drop:
NAT failed (nat-failed) 16
NAT reverse path failed (nat-rpf-failed) 4
Global policy:
Service-policy: global_policy
Class-map: inspection_default
Inspect: dns preset_dns_map, packet 351, drop 0, reset-drop 0
Inspect: ftp, packet 56543., drop 0, reset-drop 0
Inspect: rsh, packet 0, drop 0, reset-drop 0
Inspect: rtsp, packet 0, drop 0, reset-drop 0
tcp-proxy: bytes in buffer 0, bytes dropped 0
Inspect: esmtp _default_esmtp_map, packet 6546418, drop 0, reset-drop 0
Inspect: sqlnet, packet 0, drop 0, reset-drop 0
Inspect: sunrpc, packet 0, drop 0, reset-drop 0
tcp-proxy: bytes in buffer 0, bytes dropped 0
Inspect: netbios, packet 0, drop 0, reset-drop 0
Inspect: tftp, packet 0, drop 0, reset-drop 0
Inspect: ip-options _default_ip_options_map, packet 0, drop 0, reset-drop 0
Class-map: SIP-class
Inspect: sip , packet 5467731, drop 31, reset-drop 0
tcp-proxy: bytes in buffer 35, bytes dropped 5410
Class-map: Skinny-class
Inspect: skinny , packet 0, drop 0, reset-drop 0
tcp-proxy: bytes in buffer 0, bytes dropped 0
Class-map: H323-class
Inspect: h323 h225 _default_h323_map, packet 0, drop 0, reset-drop 0
tcp-proxy: bytes in buffer 0, bytes dropped 0
show service-policy flow …
• show service-policy flow… allows you to define a flow (connection), and the
ASA will tell you what inspection engine is applied to that flow.
• The configuration which is matched is also displayed
Global policy:
Service-policy: global_policy
Class-map: Web-class
Match: access-list WebTraffic
Access rule: permit tcp any any eq www
Action:
Input flow: inspect http
Class-map: class-default
Match: any
Action:
show service-policy flow …
• show service-policy flow is very useful to:
• Understand what policies are applied to any given traffic flow
• Unravel complex configurations, with multiple service-policies (both global and
interface)
• Validate the configuration
• As fate would have it, it’s Friday and the sub pulls into a
nice South-Pacific birth for a weekend of R&R. You decide
to celebrate your recent promotion by drinking some local
concoctions with your friends and some locals. Early
Saturday morning an ensign comes running up saying
access to the Dive system is down again and the Captain
has ordered you to return to the sub. As you stumble back
(with assistance) you are trying to figure out what was in
those coconuts.
Object NAT
object network ServerReal
nat (inside,outside) static ServerTrans
Manual NAT
nat (inside,outside) source static ServerReal ServerTrans
Object NAT vs. Manual NAT
• The difference is where the entries exist in the NAT table (different sections)
NAT Table
Static NAT Manual NAT Policies First Match
Longest Prefix (Section 1) (in config)
. . .
Phase: 3
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
object network obj-ServerNet
nat (inside,outside) static obj-ServerNet
Additional Information:
NAT divert to egress interface inside
Untranslate 10.3.19.50/80 to 10.3.19.50/80
. . .
ASDM Packet Tracer Example
@CiscoTACPodcast
Podcast Episodes
Ep. # Topic Ep. # Topic
41 Troubleshooting ASA Clustering 27 IOS Embedded Event Manager (EEM)
40 Introduction to ASA Clustering 26 Troubleshooting IPSec VPNs
39 Voice Security Concepts and Best Practices 25 Understanding DMVPN and GETVPN
38 Introduction to OnePK 24 The Cisco Identity Services Engine
37 ASA Network Address Translation (NAT) 23 The Cisco ASA Services Module
How Cisco uses the Web Security Appliance to protect its
36 Network Management at Cisco Live! 2013 22
network
35 Identity Services Engine v1.2 21 Cisco Live! Las Vegas 2011
34 Cisco Live! 2013 Orland, FL 20 This Week In TAC!
Virtual Security: The ASA 1000v and Virtual Security
33 19 Troubleshooting the NAC Appliance
Gateway (VSG)
Useful ASA and IPS Commands and Features You Might
32 Investigating Syslogs: Tips and Tricks 18
Not Know About
Answering Questions From The Cisco Support
31 A look into ASA Quality with the Quality Assurance Team 17
Community
30 Introducing FlexVPN 16 Mitigating a SQL attack with ASA, IPS and IOS Firewall
29 Cisco Live! 2012 San Diego 15 Using Certificates on the ASA and IOS platforms
28 The History of the PIX 14 TCP connections through the ASA and FWSM
Podcast Episodes
Ep. # Topic
13 HTTP Filtering on the ASA
12 Securing Cisco Routers
11 ASA Anyconnect VPN
10 ASA Version 8.3 Overview
Multiple Context Mode on the ASA and FWSM
9
Platforms
8 ASA Advanced Application Protocol Inspection
7 Monitoring Firewall Performance
6 Tips for Taking the CCIE Security Exam
5 Troubleshooting Firewall Failover, Part 2
Troubleshooting Firewall Failover Part 1; Guest Omar
4
Santos from PSIRT
3 Transparent Firewall Mode; Lifecycle of a TAC Case
https://supportforums.cisco.com/docs/DOC-5727
Any Final Questions?
Complete Your Online Session Evaluation
• Give us your feedback to be
entered into a Daily Survey
Drawing. A daily winner
will receive a $750 Amazon
gift card.
• Complete your session surveys
though the Cisco Live mobile
app or your computer on
Cisco Live Connect.
Don’t forget: Cisco Live sessions will be available
for viewing on-demand after the event at
CiscoLive.com/Online
Participate in the “My Favorite Speaker” Contest
Promote Your Favorite Speaker and You Could Be a Winner
• Promote your favorite speaker through Twitter and you could win $200 of
Cisco Press products (@CiscoPress)
• Send a tweet and include
• Your favorite speaker’s Twitter handle @ciscotacpodcast
• Two hashtags: #CLUS #MyFavoriteSpeaker
• You can submit an entry for more than one of your “favorite” speakers
• Don’t forget to follow @CiscoLive and @CiscoPress
• View the official rules at http://bit.ly/CLUSwin
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Table Topics
• Meet the Engineer 1:1 meetings
• Related sessions
Thank you
TCP Connection Termination Reasons For your
reference
— Quick Reference
Reason Description
Connection Ended Because It Was Idle Longer Than the
Conn-Timeout
Configured Idle Timeout
Deny Terminate Flow Was Terminated by Application Inspection
The Standby Unit in a Failover Pair Deleted a Connection
Failover Primary Closed
Because of a Message Received from the Active Unit
Force Termination After Ten Minutes Awaiting the Last ACK or
FIN Timeout
After Half-Closed Timeout
Flow Closed by Inspection Flow Was Terminated by Inspection Feature
Flow Terminated by IPS Flow Was Terminated by IPS
Flow Reset by IPS Flow Was Reset by IPS
Flow Terminated by
Flow Was Terminated by TCP Intercept
TCP Intercept
Invalid SYN SYN Packet Not Valid
Connection Timed Out Because It Was Idle Longer than the
Idle Timeout
Timeout Value
IPS Fail-Close Flow Was Terminated Due to IPS Card Down
SYN Control Back Channel Initiation from Wrong Side
TCP Connection Termination Reasons For your
reference
Managing Industrial Networks with This curriculum addresses foundational skills needed to manage and Cisco Industrial
Cisco Networking Technologies (IMINS) administer networked industrial control systems. It provides plant Networking Specialist
administrators, control system engineers and traditional network engineers
with an understanding of the networking technologies needed in today's
connected plants and enterprises
Control Systems Fundamentals For IT and Network Engineers, covers basic concepts in Industrial Control
for Industrial Networking (ICINS) systems including an introduction to automation industry verticals,
automation environment and an overview of industrial control networks
Networking Fundamentals For Industrial Engineers and Control System Technicians, covers basic IP
for Industrial Control Systems (INICS) and networking concepts, and introductory overview of Automation
industry Protocols.
Executing Advanced Cisco Business Value Enables customer transformation through business architecture and Cisco Certified Business
Analysis and Design Techniques solution selling expertise Value Practitioner
Performing Cisco Business-Focused Provides skills and an approach to build a strategic roadmap of IT Cisco Transformative
Transformative Architecture Engagements initiatives, aligned to business priorities Architecture Specialist
Implementing Cisco Secure Mobility Solutions Deploy Cisco’s Identity Services Engine and 802.1X secure network access
(SIMOS)
Protect data traversing a public or shared infrastructure such as the Internet by
implementing and maintaining Cisco VPN solutions
Securing Cisco Networks with Threat Detection and Designed for professional security analysts, the course covers essential areas of Cisco Cybersecurity Specialist
Analysis (SCYBER) competency including event monitoring, security event/alarm/traffic analysis, and
incident response
Network Security Product and Solutions Training For official product training on Cisco’s latest security products, including Adaptive
Security Appliances, NGIPS, Advanced Malware Protection, Identity Services
Engine, Email and Web Security Appliances see
www.cisco.com/go/securitytraining
• Implementing Cisco IP Routing v2.0 Professional level instructor led trainings to prepare candidates for the CCNP® Routing & Switching
• Implementing Cisco IP Switched CCNP R&S exams (ROUTE, SWITCH and TSHOOT). Also available in
Networks V2.0 self study eLearning formats with Cisco Learning Labs.
• Troubleshooting and Maintaining
Cisco IP Networks v2.0
Interconnecting Cisco Networking Devices: Configure, implement and troubleshoot local and wide-area IPv4 and IPv6 CCNA® Routing & Switching
Part 2 (or combined) networks. Also available in self study eLearning format with Cisco Learning
Lab.
Interconnecting Cisco Networking Devices: Installation, configuration, and basic support of a branch network. Also CCENT® Routing & Switching
Part 1 available in self study eLearning format with Cisco Learning Lab.
Implementing Cisco Unified Wireless Network Prepares candidates to design, install, configure, monitor and conduct CCNA® Wireless
Essential basic troubleshooting tasks of a Cisco WLAN in Enterprise installations.
Designing for Cisco Internetwork Solutions Instructor led training focused on fundamental design methodologies used CCDA® (Design Associate)
(DESGN) to determine requirements for network performance, security, voice, and
wireless solutions. Prepares candidates for the CCDA certification exam.
Building Cisco Service Provider Next-Generation The two courses introduce networking technologies and solutions, including OSI CCNA Service Provider®
Networks, Part 1&2 (SPNGN1), (SPNGN2) and TCP/IP models, IPv4/v6, switching, routing, transport types, security, network
management, and Cisco OS (IOS and IOS XR).
Implementing Cisco Service Provider Mobility UMTS The three courses (SPUMTS, SPCDMA, SPLTE) cover knowledge and skills Cisco Service Provider Mobility
Networks (SPUMTS); required to understand products, technologies, and architectures that are found in CDMA to LTE Specialist;
Implementing Cisco Service Provider Mobility CDMA Universal Mobile Telecommunications Systems (UMTS) and Code Division Multiple Cisco Service Provider Mobility UMTS
Networks (SPCDMA); Access (CDMA) packet core networks, plus their migration to Long-Term Evolution to LTE Specialist
Implementing Cisco Service Provider Mobility LTE (LTE) Evolved Packet Systems (EPS), including Evolved Packet Core (EPC) and
Networks (SPLTE) Radio Access Networks (RANs).
Implementing and Maintaining Cisco Technologies Service Provider/Enterprise engineers to implement, verification-test, and optimize Cisco IOS XR Specialist
Using IOS XR (IMTXR) core/edge technologies in a Cisco IOS XR environment.
Implementing Cisco Collaboration Applications Understand how to implement the full suite of Cisco collaboration CCNP® Collaboration
(CAPPS) applications including Jabber, Cisco Unified IM and Presence, and Cisco
Unity Connection.
Implementing Cisco IP Telephony and Video Learn how to implement Cisco Unified Communications Manager, CUBE, CCNP® Collaboration
Part 1 (CIPTV1) and audio and videoconferences in a single-site voice and video network.
Implementing Cisco IP Telephony and Video Obtain the skills to implement Cisco Unified Communications Manager in a
Part 2 (CIPTV2) modern, multisite collaboration environment.
Troubleshooting Cisco IP Telephony and Video Troubleshoot complex integrated voice and video infrastructures
(CTCOLLAB)
Implementing Cisco Collaboration Devices Acquire a basic understanding of collaboration technologies like Cisco Call CCNA® Collaboration
(CICD) Manager and Cisco Unified Communications Manager.
Implementing Cisco Video Network Devices Learn how to evaluate requirements for video deployments, and implement
(CIVND) Cisco Collaboration endpoints in converged Cisco infrastructures.
Implementing Cisco Data Center Unified Fabric Obtain the skills to deploy complex virtualized Data Center Fabric and CCNP® Data Center
(DCUFI); Computing environments with Nexus and Cisco UCS.
Implementing Cisco Data Center Unified
Computing (DCUCI)
Introducing Cisco Data Center Networking Learn basic data center technologies and how to build a data center CCNA® Data Center
(DCICN); Introducing Cisco Data Center infrastructure.
Technologies (DCICT)
Product Training Portfolio: DCAC9k, DCINX9k, Get a deep understanding of the Cisco data center product line including
DCMDS, DCUCS, DCNX1K, DCNX5K, DCNX7K the Cisco Nexus9K in ACI and NexusOS modes
Developing with Cisco Network Programmability Learn how to build applications for network environments and effectively Cisco Network Programmability
(NPDEV); bridge the gap between IT professionals and software developers. Developer Specialist Certification
Developing with Cisco Network Programmability
for Cisco ACI (NPDEVACI)
Designing with Cisco Network Programmability Learn how to expand your skill set from traditional IT infrastructure to Cisco Network Programmability
(NPDES); application integration through programmability. Design Specialist Certification
Designing with Cisco Network Programmability
for Cisco ACI (NPDESACI)
Implementing Cisco Network Programmability Learn how to implement and troubleshoot open IT infrastructure Cisco Network Programmability
(NPENG); technologies. Engineer Specialist Certification
Implementing Cisco Network Programmability
for Cisco ACI (NPENGACI)
UCS Director (UCSDF) Learn how to manage physical and virtual infrastructure using
orchestration and automation functions of UCS Director.
Cisco Prime Service Catalog Learn how to deliver data center, workplace, and application services in an
on-demand, automated, and repeatable method.
Cisco Intercloud Fabric Learn how to implement end-to-end hybrid clouds with Intercloud Fabric
for Business and Intercloud Fabric for Providers.
Cisco Intelligent Automation for Cloud Learn how to implement and manage cloud deployments with Cisco
Intelligent Automation for Cloud