Troubleshooting Wireless LANs with Centralized Controllers

BRKEWN-3011
Wesley Terry

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

1

Troubleshooting Wireless LANs
 Supportability

 Software and Support Model
 Troubleshooting Basics  The Client Debug  WLC Config Analyzer (WLCCA)

 Additional Troubleshooting

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

2

Supportability

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

3

Supportability
 WLC Supportability
Methods of Management Using the GUI Important Show Commands (CLI)
Important Debugs (CLI) Best Practices

 AP Supportability
Methods of Accessing the AP Important Show Commands

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

4

WLC Supportability
Methods of Management

 GUI
HTTPS (E) / HTTP (D)

Default Mode (E)=Enabled (D)=Disabled

 CLI
Console

SSH (E) / Telnet (D)

 SNMP
V1 (D) / V2 (E) – Change me! V3 (E) – Change me Note: Management Via Wireless Clients (D)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

5

WLC Supportability
Using the GUI

 Monitor
AP/Radio Statistics

WLC Statistics Client Details Trap Log

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

6

WLC Supportability
Using the GUI

 Wireless > All APs
AP list shows AP Physical UP Time

APs are sorted by Controller Associated Time Check bottom of AP list for any recent AP disruptions Select AP to see Controller Associated Time (duration)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

7

WLC Supportability
Using the GUI

 Management
SNMP Config

Logs Tech Support

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

8

WLC Supportability
Important Show Commands (CLI)

 Show run-config
Must have! No exceptions!

“show run-config commands” (like IOS show running-config) “show run-config no-ap” (no AP information added)

 Show tech-support  CLI Tip
Log all output Config Paging Disable

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

9

WLC Supportability
Important Debugs (CLI)

 Debug client <client mac address>
Client Involved? Must Have! No Exceptions

 Debug capwap <event/error/detail/info> enable
 CLI Tips
Log all output Debugs are session based, they end when session ends

“Config session timeout 60”, sets 60 minute idle timeout Debug mac addr <mac address> Used to filter debugs on specific Mac Address Debug disable-all (Disables all debugs)
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

10

WLC Supportability
Best Practices

 Change default SNMP Parameters
 Configure Syslog for WLC and AP  Enable Coredump for WLC and AP  Configure NTP Server for Date/Time

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

11

AP Supportability
 Methods of Accessing the AP
Console Telnet (D) / SSH (D) No GUI support
AP Remote Commands
Default Mode (E)=Enabled (D)=Disabled

 Enabling Telnet/SSH
WLC CLI: config ap [telnet/ssh] enable <ap name> WLC GUI: Wireless > All APs > Select AP > Advanced Select [telnet/ssh] > Apply

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

12

AP Supportability
AP Remote Commands (WLC CLI)

 Debug AP enable <AP name>
Enables AP Remote Debug

AP Must be associated to WLC Redirects AP Console output to WLC session

 Debug AP command “<command>” <AP name>
Output is redirected to WLC session AP runs IOS, numerous generic IOS commands available

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

13

AP Supportability
Show Commands (AP CLI or WLC Remote Cmd)

 Show controller Do[0/1] (or Show Tech)
Must have! Before/During/After event

 Show log
 WLC: show ap eventlog <ap name>  Show capwap client <?>

 CLI Tips
Debug capwap console client
Debug capwap client no-reload

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

14

Software and Support Model

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

15

Software and Support Model
 Opening a TAC Service Request

 Cisco Support Model
TAC vs Business Unit

What to expect from TAC How does escalation work?

 WLC Software Trains
CCO (ED/MD/AW) “Engineering Special” vs “Escalation”

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

16

Opening a TAC Service Request
 What should I have ready?
Clear problem description Always: Show run-config If client involved, always: “debug client <mac address>”
Your analysis of any data provided

 Expectations for customer involvement
TAC SR severity level descriptions state that You and Cisco will commit necessary resources according to severity

You must set correct expectation of timeline and severity

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

17

Opening a TAC Service Request
 Potential reasons to slow a TAC SR‟s resolution
Information about the problem is missing The severity level was not set appropriately Data, such as traces or logs, has not been forwarded to the engineer
The scope or time requirements are not well understood by the engineer

The problem cannot be reproduced in the Cisco Technical Assistance Center lab
Access to the affected equipment for debugging purposes is not available

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

18

Cisco Support Model – TAC vs. BU
 TAC
Customer advocate Technology focused with cross technology collaboration Escalation path within TAC exists

 Business Unit - Escalation
Work in conjunction with TAC during specific engagements
Product specific focus Engages development resources when necessary

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

19

Cisco Support Model – Expectations
 What not to expect from TAC
Design and deployment Complete configuration Sales related information

 What to expect from TAC
Configuration assistance
Problem analysis / bug isolation Workarounds or fixes Action plan to resolve SR Hardware replacement

Engage BU when appropriate

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

20

Cisco Support Model - Escalation
 TAC Escalation Process
Multi-Tier support resources within a technology TAC to engage resources (TAC/BU) when appropriate SR ownership might not change hands

 Customer Escalation Process
Raise SR priority (S1/S2)
Engage account team Your satisfaction is important to the Cisco TAC. If you have concerns about the progress of your case, please contact your regional TAC.

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

21

WLC Software Trains
 CCO - Cisco.com release
6.0.202.0, 7.0.116.0, etc… Full test cycle Classified as ED when posted

 AssureWave
AW is no longer tagged on CCO, but AW validation results are available at: http://www.cisco.com/go/assurewave
Results available 4 weeks after CCO

 MD
MD tag represents stable releases for mass adoption MD tag will be considered on CCO after AW release validation, 10 weeks in field and TAC/Escalation signoff
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

22

WLC Software Trains - ES vs. Escalation
 Engineering Special
Development “special” image for fix validation or limited use Sanity tested “As-is”

 Escalation Code
Escalation is a post-CCO maintenance release with specific/minimal customer impacting SW fixes
Fix must be fully committed to the next CCO MR

Sanity + focus tested Fully TAC+BU supported “Running-Master” so each release builds upon the previous

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

23

Troubleshooting Basics

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

24

The 10-Point Capture
chan. 1
EAP IP CAPWAP IP CAPWAP
AP Debugs

Radio

Driver

Supp.
Supplicant Logs

802.11 Data 802.11 Management

WLC EOIP

RADIUS

ACS

IP
DHCP

802.11 Management

WLC
Wired Sniff

Wired Sniff
DHCP Logs

Driver Debugs/ Adapter Capture

Wireless Sniff

Spectrum Analysis

WLC Debugs

ACS Logs

NTP

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

25

Troubleshooting Basics
 Troubleshooting 101
Clearly define the problem Understand any possible triggers Know the expected behavior
Reproducibility
Questions
Problem Definition

 Recommended Tools
Spectrum Analyzer Wireless Sniffer and Wired Captures

Tests

Analysis

Solution(s)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

26

Troubleshooting 101
 Troubleshooting is an art with no right or wrong procedure, but best with a logical methodology.

 Step 1: Define the problem
It is crucial to understand all possible details of a problem Knowing what is and is not working will go a long way With a proper understanding of the problem description you can skip many steps
Bad description: “Client slow to connect”

Good description: “Client associations are rejected with Status17 several times before they associate successfully.”

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

27

Troubleshooting 101
 Step 2: Understand any possible triggers
If something previously worked but no longer works, there should be an identifiable trigger
Understanding any and all configuration or environmental changes could help pinpoint a trigger

 Step 3: Know the expected behavior
If you know the order of expected behavior that is failing, defining where the behavior breaks down (Problem Description) is better than defining the end result. Example: “One way audio between Phone A and B, because Phone A does not get an ARP Response for Phone B”

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

28

Troubleshooting 101
 Step 4: Reproducibility
Any problem that has a known procedure to reproduce (or frequently randomly occurs) should be easy to diagnose
Being able to easily validate or disprove a potential solution saves time by being able to quickly move on to the next theory

If a problem is reproducible in other environments with a known procedure, TAC/BU can facilitate internal testing and proposed fix/workaround verification

 Debugs and Captures of working scenarios can help pin point where exactly the difference is

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

29

Recommended Tools
 Wireless Sniffer
Example: Linksys USB600N with Omnipeek
TAC can publish Omnipeek-RA if you have compatible HW

 Wired Packet Capture
Example: Wireshark
Use for spanned switchports of AP/WLC or client side data

 Spectrum Analyzer
Spectrum Expert with Card or Clean-Air AP

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

30

The Client Debug

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

31

Steps to Building an 802.11 Connection
802.11

1. Listen for Beacons
State 1: Unauthenticated, Unassociated

2. Probe Request 3. Probe Response
AP

4. Authentication Request 5. Authentication Response
State 2: Authenticated, Unassociated

6. Association Request 7. Association Response
WLC

State 3: Authenticated, Associated
BRKEWN-3011

8. (Optional: EAPOL Authentication) 9. (Optional: Encrypt Data) 10. Move User Data
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

32

The Client Debug
debug client <mac address>  A multi-debug macro
(Cisco Controller) >debug client 00:16:EA:B2:04:36 (Cisco Controller) >show debug
MAC address ................................ 00:16:ea:b2:04:36 Debug Flags Enabled:

dhcp packet enabled dot11 mobile enabled
dot11 state enabled dot1x events enabled

dot1x states enabled
pem events enabled pem state enabled

CCKM client debug enabled

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

33

Understanding the Client State
Name
8021X_REQD

Description
802.1x (L2) Authentication Pending

DHCP_REQD WEBAUTH_REQD
RUN

IP Learning State Web (L3) Authentication Pending
Client Traffic Forwarding

(Cisco Controller) >show client detail 00:16:ea:b2:04:36 Client MAC Address............................... 00:16:ea:b2:04:36 ….. Policy Manager State............................. WEBAUTH_REQD 00:16:ea:b2:04:36 10.10.1.103 DHCP_REQD (7) Change state to RUN (20) last state RUN (20)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

34

The Client Debug - Walkthrough
 Association (Start)  L2 Authentication (8021X_REQD)  Client Address Learning (DHCP_REQD)  L3 Authentication (WEBAUTH_REQD)  Client Fully Connected (RUN)
 Deauth/Disassoc  Tips and Tricks

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

35

Client Debug - Association

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

36

Association
(Cisco Controller) >debug client 00:16:EA:B2:04:36 (Cisco Controller) > (Cisco Controller) > Association received from mobile on AP 00:26:cb:94:44:c0 0.0.0.0 START (0) Changing ACL 'none' (ACL ID 0) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1621) Applying site-specific IPv6 override for station 00:16:ea:b2:04:36 - vapId 1, site 'default-group', interface '3' Applying IPv6 Interface Policy for station 00:16:ea:b2:04:36 - vlan 3, interface id 8, interface '3„
STA - rates (12): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0 Processing RSN IE type 48, length 22 for mobile 00:16:ea:b2:04:36

0.0.0.0 START (0) Initializing policy 0.0.0.0 START (0) Change state to AUTHCHECK (2) last state AUTHCHECK (2) 0.0.0.0 AUTHCHECK (2) Change state to 8021X_REQD (3) last state 8021X_REQD (3) 0.0.0.0 8021X_REQD (3) DHCP Not required on AP 00:26:cb:94:44:c0 vapId 1 apVapId 1for this client 0.0.0.0 8021X_REQD (3) Plumbed mobile LWAPP rule on AP 00:26:cb:94:44:c0 vapId 1 apVapId 1 apfMsAssoStateInc apfPemAddUser2 Changing state for mobile 00:16:ea:b2:04:36 on AP 00:26:cb:94:44:c0 from Idle to Associated Scheduling deletion of Mobile Station: (callerId: 49) in 1800 seconds
Sending Assoc Response to station on BSSID 00:26:cb:94:44:c0 (status 0) ApVapId 1 Slot 0

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

37

Association
Association received from mobile on AP 00:26:cb:94:44:c0 0.0.0.0 START (0) Changing ACL 'none' (ACL ID 0) ===> 'none' (ACL ID 255) --- (caller apf_policy.c:1621) Applying site-specific IPv6 override for station 00:16:ea:b2:04:36 - vapId 1, site 'default-group', interface '3' Applying IPv6 Interface Policy for station 00:16:ea:b2:04:36 - vlan 3, interface id 8, interface '3'

 Association received
Association Request, client did not “Roam” (Reassociate)
AP Base Radio = 00:26:cb:94:44:c0

 vapId 1, site 'default-group', interface '3„
vapId = WLAN # site = AP Group (Wlan 1) (default-group)

Interface = Dynamic Interface name (3)

 vlan 3
Vlan = Vlan # of Dynamic Interface
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

38

Association
STA - rates (12): 130 132 139 150 12 18 24 36 48 72 96 108 0 0 0 0 Processing RSN IE type 48, length 22 for mobile 00:16:ea:b2:04:36

 STA - rates
Madatory Rates (>128) = (#-128)/2

Supported Rates (<128) = #/2 1m,2m,5.5m,11m,6s,9s,12s,18s,24s,36s,48s,54s

 Processing RSN IE type 48
WPA2-AES Processing WPA IE type 221 = WPA-TKIP

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

39

Association
0.0.0.0 START (0) Initializing policy 0.0.0.0 START (0) Change state to AUTHCHECK (2) last state AUTHCHECK (2) 0.0.0.0 AUTHCHECK (2) Change state to 8021X_REQD (3) last state 8021X_REQD (3) 0.0.0.0 8021X_REQD (3) DHCP Not required on AP 00:26:cb:94:44:c0 vapId 1 apVapId 1for this client 0.0.0.0 8021X_REQD (3) Plumbed mobile LWAPP rule on AP 00:26:cb:94:44:c0 vapId 1 apVapId 1 apfMsAssoStateInc apfPemAddUser2 Changing state for mobile 00:16:ea:b2:04:36 on AP 00:26:cb:94:44:c0 from Idle to Associated Scheduling deletion of Mobile Station: (callerId: 49) in 1800 seconds

 0.0.0.0 START
0.0.0.0 = IP we know for client (In this case nothing)

 Change state to 8021X_REQD
Passed association, moving client to next state: 8021X_REQD

 Scheduling deletion
Session Time on WLAN (1800 seconds in this case)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

40

Association
Sending Assoc Response to station on BSSID 00:26:cb:94:44:c0 (status 0) ApVapId 1 Slot 0

 Slot 0 = B/G(2.4) Radio
Slot 1 = A(5) Radio

 Sending Assoc Response Status 0 = Success
Anything other than Status 0 is Failure
Common Assoc Response Failures: 1 – Unknown Reason – Anything not matching defined reason codes 12 – Unknown or Disabled SSID 17 – AP cannot handle any more associations 18 – Client is using a datarate that is not allowed 35 – WLAN requires the use of WMM and client does not support it 201 – Voice client attempting to connect to a non-platinum WLAN 202 – Not enough available bandwidth to handle a new voice call (CAC Rejection)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

41

Association - FSR
Processing WPA IE type 221, length 22 for mobile 00:16:ea:b2:04:36 CCKM: Mobile is using CCKM CCKM: Processing REASSOC REQ IE Including CCKM Response IE (length 62) in Assoc Resp to mobile Sending Assoc Response to station on BSSID 00:26:cb:94:44:c0 (status 0) Vap Id 6 Slot 1
OR

Processing RSN IE type 48, length 22 for mobile 00:16:ea:b2:04:36 Received RSN IE with 1 PMKIDs from mobile 00:16:ea:b2:04:36 Received PMKID: (16) [0000] cb bc 27 82 88 14 92 fd 3b 88 de 6a eb 49 be c8 Found an entry in the global PMK cache for station Computed a valid PMKID from global PMK cache for mobile FSR aIOS CUWN

CCKM - WPA CCKM - WPA2 WPA2 PKC
WPA2 "Sticky"
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

yes yes no
yes
Cisco Public

yes yes yes
no*
42

Association - Takeaway
 Association vs. Reassociation  Debug shows
AP, Slot, AP-Group, WLAN ID, Interface, Data Rates, Encryption type

 Association Response
Confirms if Client is associated Defines reason if denied

 Further troubleshooting
May require Wireless Sniffer or capture at AP Switchport If not sending Assoc Request, must know why from Client

Trying disabling WLAN features to “dumb it down”

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

43

Client Debug – L2 Authentication

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

44

802.1X Authentication
Supplicant Authenticator

Server

EAPOL-START EAP-ID-Request EAP-ID-Response RADIUS (EAP-ID_Response)

Rest of the EAP Conversation EAP-Success
The Supplicant Derives the Session Key from User Password or Certificate and Authentication Exchange
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Radius-Access-Accept
(Key) Session Key
45

Cisco Public

WPA2-AES-802.1X
Sending Assoc Response to station on BSSID 00:26:cb:94:44:c0 (status 0) ApVapId 1 Slot 0 Station 00:16:ea:b2:04:36 setting dot1x reauth timeout = 1800 dot1x - moving mobile 00:16:ea:b2:04:36 into Connecting state

Sending EAP-Request/Identity to mobile 00:16:ea:b2:04:36 (EAP Id 1)
Received EAPOL EAPPKT from mobile 00:16:ea:b2:04:36 Username entry (cisco) created for mobile

Received Identity Response (count=1) from mobile 00:16:ea:b2:04:36
EAP State update from Connecting to Authenticating for mobile 00:16:ea:b2:04:36 dot1x - moving mobile 00:16:ea:b2:04:36 into Authenticating state ………………….. Entering Backend Auth Req state (id=3) for mobile 00:16:ea:b2:04:36 Sending EAP Request from AAA to mobile 00:16:ea:b2:04:36 (EAP Id 3) Received EAPOL EAPPKT from mobile 00:16:ea:b2:04:36 Received EAP Response from mobile 00:16:ea:b2:04:36 (EAP Id 3, EAP Type 25) ........................... Received EAP Response from mobile 00:16:ea:b2:04:36 (EAP Id 10, EAP Type 25) Entering Backend Auth Response state for mobile 00:16:ea:b2:04:36 Processing Access-Challenge for mobile 00:16:ea:b2:04:36 Entering Backend Auth Req state (id=11) for mobile 00:16:ea:b2:04:36 Sending EAP Request from AAA to mobile 00:16:ea:b2:04:36 (EAP Id 11) Received EAPOL EAPPKT from mobile 00:16:ea:b2:04:36 Received EAP Response from mobile 00:16:ea:b2:04:36 (EAP Id 11, EAP Type 25) Entering Backend Auth Response state for mobile 00:16:ea:b2:04:36

Processing Access-Accept for mobile 00:16:ea:b2:04:36 ***OR*** Processing Access-Reject for mobile 00:16:ea:b2:04:36
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

46

Common EAP Types
 1 – Identity

 2 – Notification
 3 – NAK  4 – MD5  5 – OTP  6 – Generic Token  13 – EAP TLS  17 – LEAP
Sending EAP Request from AAA to mobile 00:16:ea:b2:04:36 (EAP Id 3) Received EAPOL EAPPKT from mobile 00:16:ea:b2:04:36 Received EAP Response from mobile 00:16:ea:b2:04:36 (EAP Id 3, EAP Type 25)

 18 – EAP SIM  21 – EAP TTLS  25 – PEAP  43 – EAP-FAST

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

47

802.1X (Cont.) (WPA2-AES-PSK)
Sending Assoc Response to station on BSSID 00:26:cb:94:44:c0 (status 0) ApVapId 1 Slot 0 Creating a PKC PMKID Cache entry for station 00:16:ea:b2:04:36 (RSN 2) Adding BSSID 00:26:cb:94:44:c0 to PMKID cache for station 00:16:ea:b2:04:36 New PMKID: (16) [0000] 31 d5 5b 0b 64 28 2b be c5 8d d5 4c 03 30 c7 cd Initiating RSN PSK to mobile 00:16:ea:b2:04:36 dot1x - moving mobile 00:16:ea:b2:04:36 into Force Auth state Skipping EAP-Success to mobile 00:16:ea:b2:04:36 Including PMKID in M1 (16) [0000] 31 d5 5b 0b 64 28 2b be c5 8d d5 4c 03 30 c7 cd Starting key exchange to mobile 00:16:ea:b2:04:36, data packets will be dropped
Sending EAPOL-Key Message to mobile 00:16:ea:b2:04:36 state INITPMK (message 1), replay counter 00.00.00.00.00.00.00.00 Received EAPOL-Key from mobile 00:16:ea:b2:04:36 Ignoring invalid EAPOL version (1) in EAPOL-key message from mobile 00:16:ea:b2:04:36 Received EAPOL-key in PTK_START state (message 2) from mobile 00:16:ea:b2:04:36 Stopping retransmission timer for mobile 00:16:ea:b2:04:36 Sending EAPOL-Key Message to mobile 00:16:ea:b2:04:36 state PTKINITNEGOTIATING (message 3), replay counter 00.00.00.00.00.00.00.01 Received EAPOL-Key from mobile 00:16:ea:b2:04:36 Ignoring invalid EAPOL version (1) in EAPOL-key message from mobile 00:16:ea:b2:04:36 Received EAPOL-key in PTKINITNEGOTIATING state (message 4) from mobile 00:16:ea:b2:04:36 apfMs1xStateInc 0.0.0.0 8021X_REQD (3) Change state to L2AUTHCOMPLETE (4) last state L2AUTHCOMPLETE (4)
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

48

WPA2-AES-PSK - Failed
Starting key exchange to mobile 00:1e:8c:0f:a4:57, data packets will be dropped Sending EAPOL-Key Message to mobile 00:1e:8c:0f:a4:57 state INITPMK (message 1), replay counter 00.00.00.00.00.00.00.00 Received EAPOL-Key from mobile 00:1e:8c:0f:a4:57 Ignoring invalid EAPOL version (1) in EAPOL-key message from mobile 00:1e:8c:0f:a4:57 Received EAPOL-key in PTK_START state (message 2) from mobile 00:1e:8c:0f:a4:57 Received EAPOL-key M2 with invalid MIC from mobile 00:1e:8c:0f:a4:57 802.1x 'timeoutEvt' Timer expired for station 00:1e:8c:0f:a4:57 Retransmit 1 of EAPOL-Key M1 (length 121) for mobile 00:1e:8c:0f:a4:57 Received EAPOL-Key from mobile 00:1e:8c:0f:a4:57 Ignoring invalid EAPOL version (1) in EAPOL-key message from mobile 00:1e:8c:0f:a4:57 Received EAPOL-key in PTK_START state (message 2) from mobile 00:1e:8c:0f:a4:57 Received EAPOL-key M2 with invalid MIC from mobile 00:1e:8c:0f:a4:57 802.1x 'timeoutEvt' Timer expired for station 00:1e:8c:0f:a4:57 Retransmit 2 of EAPOL-Key M1 (length 121) for mobile 00:1e:8c:0f:a4:57 ………………… 802.1x 'timeoutEvt' Timer expired for station 00:1e:8c:0f:a4:57 Retransmit failure for EAPOL-Key M1 to mobile 00:1e:8c:0f:a4:57, retransmit count 3, mscb deauth count 3 Blacklisting (if enabled) mobile 00:1e:8c:0f:a4:57 apfBlacklistMobileStationEntry2 (apf_ms.c:4192) Changing state for mobile 00:1e:8c:0f:a4:57 on AP 00:16:9c:4b:c4:c0 from Associated to Exclusion-list (1)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

49

L2 Authentication - Takeaway
 8021X_REQD means L2 Authentication pending
Authentication/Encryption has not be established

 PSK is 802.1X, key is derived from PSK not AAA  If “Processing Access-Reject”
AAA/RADIUS Rejected the user (not the WLC)

 If “Processing Access-Accept”
AAA/Radius Accepted the user M1-M4 should follow

 Further Troubleshooting
Debug aaa [all/event/detail/packet] enable Debug dot1x [aaa/packet] enable
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

50

Client Debug – IP Learning State

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

51

Client DHCP
00:16:ea:b2:04:36 Received EAPOL-key in PTKINITNEGOTIATING state 00:16:ea:b2:04:36 apfMs1xStateInc

00:16:ea:b2:04:36 0.0.0.0 8021X_REQD (3) Change state to L2AUTHCOMPLETE (4)
00:16:ea:b2:04:36 0.0.0.0 L2AUTHCOMPLETE (4) DHCP Not required on AP 00:26:cb:94:44:c0 vapId 3 apVapId 3for this client 00:16:ea:b2:04:36 0.0.0.0 L2AUTHCOMPLETE (4) Plumbed mobile LWAPP rule on AP 00:26:cb:94:44:c0 vapId 3 apVapId 3

00:16:ea:b2:04:36 0.0.0.0 L2AUTHCOMPLETE (4) Change state to DHCP_REQD (7)
00:16:ea:b2:04:36 0.0.0.0 DHCP_REQD (7) pemAdvanceState2 4755, Adding TMP rule 00:16:ea:b2:04:36 0.0.0.0 DHCP_REQD (7) Successfully plumbed mobile rule (ACL ID 255) 00:16:ea:b2:04:36 Stopping retransmission timer for mobile 00:16:ea:b2:04:36

*pemReceiveTask: 00:16:ea:b2:04:36 0.0.0.0 Added NPU entry of type 9, dtlFlags 0x0 ................... 00:16:ea:b2:04:36 DHCP received op BOOTREQUEST (1) (len 308,vlan 0, port 29, encap 0xec03) ................... 00:16:ea:b2:04:36 DHCP received op BOOTREPLY (2) (len 308,vlan 0, port 29, encap 0xec00) ................... 00:16:ea:b2:04:36 10.10.1.103 DHCP_REQD (7) Change state to RUN (20) last state RUN (20) 00:16:ea:b2:04:36 10.10.1.103 Added NPU entry of type 1, dtlFlags 0x0

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

52

Client DHCP
 Client is in DHCP_REQD state
Client State = “DHCP_REQD“

 Proxy Enabled:
DHCP Relay/Proxy

DHCP Proxy Enabled
Client DHCP Discover Unicast to DHCP Servers

DHCP Proxy Disabled

Between WLC and Server Required for Internal DHCP

Client DHCP Discover Is Bridged to DS

 Proxy Disabled:
Between Client and Server DHCP is broadcast out VLAN IP helper or other means required

DHCP Offer from Server

Client DHCP Request

DHCP ACK from Server

IP Address Learned
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

53

DHCP Proxy Enabled – DHCP Discover
*pemReceiveTask: 00:16:ea:b2:04:36 0.0.0.0 Added NPU entry of type 9, dtlFlags 0x0 32.151: 00:16:ea:b2:04:36 DHCP received op BOOTREQUEST (1) (len 312,vlan 0, port 29, encap 0xec03) 32.151: 00:16:ea:b2:04:36 DHCP selecting relay 1 - control block settings: dhcpServer: 0.0.0.0, dhcpNetmask: 0.0.0.0, dhcpGateway: 0.0.0.0, dhcpRelay: 0.0.0.0 VLAN: 0 32.151: 00:16:ea:b2:04:36 DHCP selected relay 1 - 10.10.1.1 (local address 10.10.1.4, gateway 10.10.1.1, VLAN 0, port 29) 32.151: 00:16:ea:b2:04:36 DHCP transmitting DHCP DISCOVER (1) 32.151: 00:16:ea:b2:04:36 DHCP op: BOOTREQUEST, htype: Ethernet, hlen: 6, hops: 1 32.151: 00:16:ea:b2:04:36 DHCP xid: 0x91014db0 (2432781744), secs: 0, flags: 0 32.152: 00:16:ea:b2:04:36 DHCP chaddr: 00:16:ea:b2:04:36 32.152: 00:16:ea:b2:04:36 DHCP ciaddr: 0.0.0.0, yiaddr: 0.0.0.0 32.152: 00:16:ea:b2:04:36 DHCP siaddr: 0.0.0.0, giaddr: 10.10.1.4 32.152: 00:16:ea:b2:04:36 DHCP requested ip: 10.99.76.147 32.152: 00:16:ea:b2:04:36 DHCP sending REQUEST to 10.10.1.1 (len 346, port 29, vlan 0) 32.152: 00:16:ea:b2:04:36 DHCP selecting relay 2 - control block settings: dhcpServer: 0.0.0.0, dhcpNetmask: 0.0.0.0, dhcpGateway: 0.0.0.0, dhcpRelay: 10.10.1.4 VLAN: 0 32.152: 00:16:ea:b2:04:36 DHCP selected relay 2 - NONE

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

54

DHCP Proxy Enabled – DHCP Offer
34.166: 00:16:ea:b2:04:36 DHCP received op BOOTREPLY (2) (len 308,vlan 0, port 29, encap 0xec00) 34.166: 00:16:ea:b2:04:36 DHCP setting server from OFFER (server 10.10.1.3, yiaddr 10.10.1.103) 34.167: 00:16:ea:b2:04:36 DHCP sending REPLY to STA (len 414, port 29, vlan 0) 34.167: 00:16:ea:b2:04:36 DHCP transmitting DHCP OFFER (2) 34.167: 00:16:ea:b2:04:36 DHCP op: BOOTREPLY, htype: Ethernet, hlen: 6, hops: 0 34.167: 00:16:ea:b2:04:36 DHCP xid: 0x91014db0 (2432781744), secs: 0, flags: 0 34.167: 00:16:ea:b2:04:36 DHCP chaddr: 00:16:ea:b2:04:36 34.167: 00:16:ea:b2:04:36 DHCP ciaddr: 0.0.0.0, yiaddr: 10.10.1.103 34.167: 00:16:ea:b2:04:36 DHCP siaddr: 0.0.0.0, giaddr: 0.0.0.0 34.168: 00:16:ea:b2:04:36 DHCP server id: 1.1.1.1 rcvd server id: 10.10.1.3

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

55

DHCP Proxy Enabled – DHCP Request
38.169: 00:16:ea:b2:04:36 DHCP received op BOOTREQUEST (1) (len 316,vlan 0, port 29, encap 0xec03) 38.169: 00:16:ea:b2:04:36 DHCP selecting relay 1 - control block settings: dhcpServer: 10.10.1.3, dhcpNetmask: 0.0.0.0, dhcpGateway: 0.0.0.0, dhcpRelay: 10.10.1.4 VLAN: 0 38.169: 00:16:ea:b2:04:36 DHCP selected relay 1 - 10.10.1.3 (local address 10.10.1.4, gateway 10.10.1.3, VLAN 0, port 29) 38.169: 00:16:ea:b2:04:36 DHCP transmitting DHCP REQUEST (3) 38.169: 00:16:ea:b2:04:36 DHCP op: BOOTREQUEST, htype: Ethernet, hlen: 6, hops: 1 38.170: 00:16:ea:b2:04:36 DHCP xid: 0x91014db0 (2432781744), secs: 0, flags: 0 38.170: 00:16:ea:b2:04:36 DHCP chaddr: 00:16:ea:b2:04:36 38.170: 00:16:ea:b2:04:36 DHCP ciaddr: 0.0.0.0, yiaddr: 0.0.0.0 38.170: 00:16:ea:b2:04:36 DHCP siaddr: 0.0.0.0, giaddr: 10.10.1.4 38.170: 00:16:ea:b2:04:36 DHCP requested ip: 10.10.1.103 38.170: 00:16:ea:b2:04:36 DHCP server id: 10.10.1.3 rcvd server id: 1.1.1.1 38.170: 00:16:ea:b2:04:36 DHCP sending REQUEST to 10.10.1.3 (len 354, port 29, vlan 0) 38.170: 00:16:ea:b2:04:36 DHCP selecting relay 2 - control block settings: dhcpServer: 10.10.1.3, dhcpNetmask: 0.0.0.0, dhcpGateway: 0.0.0.0, dhcpRelay: 10.10.1.4 VLAN: 0 38.171: 00:16:ea:b2:04:36 DHCP selected relay 2 - NONE

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

56

DHCP Proxy Enabled – DHCP Ack

38.172: 00:16:ea:b2:04:36 DHCP received op BOOTREPLY (2) (len 308,vlan 0, port 29, encap 0xec00) 38.173: 00:16:ea:b2:04:36 10.10.1.103 DHCP_REQD (7) Change state to RUN (20) last state RUN (20) 38.173: 00:16:ea:b2:04:36 10.10.1.103 RUN (20) Reached PLUMBFASTPATH: from line 5273 38.173: 00:16:ea:b2:04:36 10.10.1.103 RUN (20) Replacing Fast Path rule 38.173: 00:16:ea:b2:04:36 Assigning Address 10.10.1.103 to mobile 38.173: 00:16:ea:b2:04:36 DHCP success event for client. Clearing dhcp failure count for interface management 38.174: 00:16:ea:b2:04:36 DHCP sending REPLY to STA (len 414, port 29, vlan 0) 38.174: 00:16:ea:b2:04:36 DHCP transmitting DHCP ACK (5) 38.174: 00:16:ea:b2:04:36 DHCP op: BOOTREPLY, htype: Ethernet, hlen: 6, hops: 0 38.174: 00:16:ea:b2:04:36 DHCP xid: 0x91014db0 (2432781744), secs: 0, flags: 0 38.174: 00:16:ea:b2:04:36 DHCP chaddr: 00:16:ea:b2:04:36 38.174: 00:16:ea:b2:04:36 DHCP ciaddr: 0.0.0.0, yiaddr: 10.10.1.103 38.174: 00:16:ea:b2:04:36 DHCP siaddr: 10.10.1.30, giaddr: 0.0.0.0 38.174: 00:16:ea:b2:04:36 DHCP server id: 1.1.1.1 rcvd server id: 10.10.1.3 38.179: 00:16:ea:b2:04:36 10.10.1.103 Added NPU entry of type 1, dtlFlags 0x0

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

57

DHCP Proxy Disabled – Discover/Offer
*pemReceiveTask: 00:16:ea:b2:04:36 0.0.0.0 Added NPU entry of type 9, dtlFlags 0x0 *00:16:ea:b2:04:36 DHCP received op BOOTREQUEST (1) (len 312,vlan 0, port 29, encap 0xec03) *00:16:ea:b2:04:36 DHCP processing DHCP DISCOVER (1) *00:16:ea:b2:04:36 DHCP op: BOOTREQUEST, htype: Ethernet, hlen: 6, hops: 0 *00:16:ea:b2:04:36 DHCP xid: 0x18a596d9 (413505241), secs: 1024, flags: 0 *00:16:ea:b2:04:36 DHCP chaddr: 00:16:ea:b2:04:36 *00:16:ea:b2:04:36 DHCP ciaddr: 0.0.0.0, yiaddr: 0.0.0.0 *00:16:ea:b2:04:36 DHCP siaddr: 0.0.0.0, giaddr: 0.0.0.0 *00:16:ea:b2:04:36 DHCP requested ip: 10.10.3.86 *00:16:ea:b2:04:36 DHCP successfully bridged packet to DS

*00:16:ea:b2:04:36 DHCP received op BOOTREPLY (2) (len 308,vlan 3, port 29, encap 0xec00) *00:16:ea:b2:04:36 DHCP processing DHCP OFFER (2) *00:16:ea:b2:04:36 DHCP op: BOOTREPLY, htype: Ethernet, hlen: 6, hops: 0 *00:16:ea:b2:04:36 DHCP xid: 0x18a596d9 (413505241), secs: 0, flags: 0 *00:16:ea:b2:04:36 DHCP chaddr: 00:16:ea:b2:04:36 *00:16:ea:b2:04:36 DHCP ciaddr: 0.0.0.0, yiaddr: 10.10.3.86 *00:16:ea:b2:04:36 DHCP siaddr: 0.0.0.0, giaddr: 0.0.0.0 *00:16:ea:b2:04:36 DHCP server id: 10.10.3.3 rcvd server id: 10.10.3.3 *00:16:ea:b2:04:36 DHCP successfully bridged packet to STA
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

58

DHCP Proxy Disabled – Request/Ack
*00:16:ea:b2:04:36 DHCP received op BOOTREQUEST (1) (len 316,vlan 0, port 29, encap 0xec03) *00:16:ea:b2:04:36 DHCP processing DHCP REQUEST (3) *00:16:ea:b2:04:36 DHCP op: BOOTREQUEST, htype: Ethernet, hlen: 6, hops: 0 *00:16:ea:b2:04:36 DHCP xid: 0x18a596d9 (413505241), secs: 1024, flags: 0 *00:16:ea:b2:04:36 DHCP chaddr: 00:16:ea:b2:04:36 *00:16:ea:b2:04:36 DHCP ciaddr: 0.0.0.0, yiaddr: 0.0.0.0 *00:16:ea:b2:04:36 DHCP siaddr: 0.0.0.0, giaddr: 0.0.0.0 *00:16:ea:b2:04:36 DHCP requested ip: 10.10.3.86 *00:16:ea:b2:04:36 DHCP server id: 10.10.3.3 rcvd server id: 10.10.3.3 *00:16:ea:b2:04:36 DHCP successfully bridged packet to DS *00:16:ea:b2:04:36 DHCP received op BOOTREPLY (2) (len 308,vlan 3, port 29, encap 0xec00) *00:16:ea:b2:04:36 DHCP processing DHCP ACK (5) *00:16:ea:b2:04:36 DHCP op: BOOTREPLY, htype: Ethernet, hlen: 6, hops: 0 *00:16:ea:b2:04:36 DHCP xid: 0x18a596d9 (413505241), secs: 0, flags: 0 *00:16:ea:b2:04:36 DHCP chaddr: 00:16:ea:b2:04:36 *00:16:ea:b2:04:36 DHCP ciaddr: 0.0.0.0, yiaddr: 10.10.3.86 *00:16:ea:b2:04:36 DHCP siaddr: 0.0.0.0, giaddr: 0.0.0.0 *00:16:ea:b2:04:36 DHCP server id: 10.10.3.3 rcvd server id: 10.10.3.3 *00:16:ea:b2:04:36 10.10.3.86 DHCP_REQD (7) Change state to RUN (20) last state RUN (20) *00:16:ea:b2:04:36 Assigning Address 10.10.3.86 to mobile *00:16:ea:b2:04:36 DHCP successfully bridged packet to STA *00:16:ea:b2:04:36 10.10.3.86 Added NPU entry of type 1, dtlFlags 0x0

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

59

Learning IP without DHCP
*Orphan Packet from 10.99.76.147 on mobile *0.0.0.0 DHCP_REQD (7) Successfully plumbed mobile rule (ACL ID 255) *Installing Orphan Pkt IP address 10.99.76.147 for station *10.99.76.147 DHCP_REQD (7) Change state to RUN (20) last state RUN (20)

 Client IP can be learned by ways other than DHCP
Client sends gratuitous ARP or ARP Request (Static Client)

Client sends IP packet (Orphan Packet), we learn IP DS sends packet to client, we learn IP from DS

 Seen with mobile devices that talk before validating DHCP  Up to client to realize their address is not valid for the subnet  DHCP Required on WLAN for prevent this

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

60

Client DHCP - Takeway
 DHCP_REQD means Learning IP State
Only “Required” if enabled on WLC

 If Proxy is enabled
Confirm DHCP Server on Interface (or Wlan) is correct
DHCP Server may not respond to WLC Proxy (Firewalls?)

 If Proxy is disabled, DHCP is similar to wired client

 Further Troubleshooting
Check DHCP Server for what it believes is happening
If WLC does not show a BOOTREQUEST, confirm the client request arrives to the WLC and leaves in the configured way

If still believed to be on WLC: debug dhcp message enable

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

61

Client Debug – L3 Authentication

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

62

Webauth
*apfReceiveTask: 00:16:ea:b2:04:36 0.0.0.0 DHCP_REQD (7) Successfully plumbed mobile rule (ACL ID 255)

*pemReceiveTask: 00:16:ea:b2:04:36 0.0.0.0 Added NPU entry of type 9, dtlFlags 0x0
*DHCP Proxy DTL Recv Task: 00:16:ea:b2:04:36 DHCP received op BOOTREQUEST (1) (len 312,vlan 0, port 29, encap 0xec03) ……………………………...

*DHCP Proxy DTL Recv Task: 00:16:ea:b2:04:36 10.10.3.86 DHCP_REQD (7) Change state to WEBAUTH_REQD (8) last state WEBAUTH_REQD (8)
*DHCP Proxy DTL Recv Task: 00:16:ea:b2:04:36 10.10.3.86 WEBAUTH_REQD (8) pemAdvanceState2 5170, Adding TMP rule *DHCP Proxy DTL Recv Task: 00:16:ea:b2:04:36 10.10.3.86 WEBAUTH_REQD (8) Successfully plumbed mobile rule (ACL ID 255) *DHCP Proxy DTL Recv Task: 00:16:ea:b2:04:36 Assigning Address 10.10.3.86 to mobile

*pemReceiveTask: 00:16:ea:b2:04:36 10.10.3.86 Added NPU entry of type 2, dtlFlags 0x0
*pemReceiveTask: 00:16:ea:b2:04:36 Sent an XID frame *apfReceiveTask: 00:16:ea:b2:04:36 Orphan Packet from 10.10.3.86 on mobile *apfReceiveTask: 00:16:ea:b2:04:36 Orphan Packet from 10.10.3.86 on mobile

*apfReceiveTask: 00:16:ea:b2:04:36 Orphan Packet from 10.10.3.86 on mobile
………………………………

*emWeb: 00:16:ea:b2:04:36 Username entry (cisco) created for mobile
*emWeb: 00:16:ea:b2:04:36 10.10.3.86 WEBAUTH_REQD (8) Change state to WEBAUTH_NOL3SEC (14) last state WEBAUTH_NOL3SEC (14)

*emWeb: 00:16:ea:b2:04:36 10.10.3.86 WEBAUTH_NOL3SEC (14) Change state to RUN (20) last state RUN (20)
*emWeb: 00:16:ea:b2:04:36 Session Timeout is 1800 - starting session timer for the mobile *emWeb: 00:16:ea:b2:04:36 10.10.3.86 RUN (20) Reached PLUMBFASTPATH: from line 5063 *emWeb: May 17 22:25:16.564: 00:16:ea:b2:04:36 10.10.3.86 RUN (20) Fast Path rule (contd...) 802.1P = 0, DSCP = 0, TokenID = 5006 IPv6 Vlan = 3, IPv6 intf id = 8 *emWeb: May 17 22:25:16.564: 00:16:ea:b2:04:36 10.10.3.86 RUN (20) Successfully plumbed mobile rule (ACL ID 255)

*pemReceiveTask: May 17 22:25:16.578: 00:16:ea:b2:04:36 10.10.3.86 Added NPU entry of type 1, dtlFlags 0x0

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

63

Webauth Redirect
 Client in WEBAUTH_REQD state
 ARP and DNS must be functional  Client attempts to browse internet  WLC “Hijacks” the handshake

Webauth
Client State = “WEBAUTH_REQD“

ARP and DNS Function
3-Way Handshake HTTP HTTP GET 200 Response 3-Way Handshake HTTP(S) GET

 Client redirects to Virtual Interface  Certificate negotiation if applicable  Webauth page is displayed  Client authenticates

Webauth Page Displayed

Successful Authentication

Client State = “RUN“
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

64

ARP and DNS Function

Confirm ARP and DNS Function

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

65

Capture from Wireless Adapter
Webauth Redirect

3-Way Handshake HTTP GET 200 Response 3-Way Handshake HTTP(S) GET Webauth Page Displayed

WLC Responding with SYN, ACK Redirect to Virtual Interface Comes from Here
WLC Responding with SYN, ACK

Client Is Talking to Webauth….

Address for Client to Redirect to (Virtual IP/Name)
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

66

Webauth - Takeaway
 If WEBAUTH_REQD, then not authenticated
Only traffic allowed is DHCP, ARP, DNS, Pre-Auth ACL, IPv6*

 If not redirected, can client browse to virtual IP?  Cert issue? Consider disabling HTTPS for HTTP webauth  Most common scenario involves ARP/DNS failure
Must confirm that client actually sends TCP SYN (http) to IP

 If proven that TCP SYN is sent and WLC does not SYN ACK, then there may be a WLC side problem
Debug webauth enable <client ip address> debug client <MAC Address>
debug pm ssh-appgw enable debug pm ssh-tcp enable
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

67

Client Debug - Run

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

68

Run State
10.10.3.82 DHCP_REQD (7) Change state to RUN (20) last state RUN (20) 10.10.3.82 RUN (20) Reached PLUMBFASTPATH: from line 5273 10.10.3.82 Added NPU entry of type 1, dtlFlags 0x0 OR 10.10.3.86 WEBAUTH_REQD (8) Change state to WEBAUTH_NOL3SEC (14) 10.10.3.86 WEBAUTH_NOL3SEC (14) Change state to RUN (20) last state RUN (20) Session Timeout is 1800 - starting session timer for the mobile 10.10.3.86 RUN (20) Reached PLUMBFASTPATH: from line 5063 10.10.3.86 Added NPU entry of type 1, dtlFlags 0x0

 RUN State is the Client Traffic Forwarding State  Client is Connected and should be functional

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

69

Client Debug – Deauth/Disassoc

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

70

Deauthenticated Client
 Idle Timeout
Occurs after no traffic received from Client

Default Duration is 300 seconds
Received Idle-Timeout from AP 00:26:cb:94:44:c0, slot 0 for STA 00:1e:8c:0f:a4:57 apfMsDeleteByMscb Scheduling mobile for deletion with deleteReason 4, reasonCode 4 Scheduling deletion of Mobile Station: (callerId: 30) in 1 seconds apfMsExpireCallback (apf_ms.c:608) Expiring Mobile! Sent Deauthenticate to mobile on BSSID 00:26:cb:94:44:c0 slot 0(caller apf_ms.c:5094)

 Session Timeout
Occurs at scheduled duration (default 1800 seconds)
Will force WEBAUTH user to WEBAUTH again
apfMsExpireCallback (apf_ms.c:608) Expiring Mobile! apfMsExpireMobileStation (apf_ms.c:5009) Changing state for mobile 00:1e:8c:0f:a4:57 on AP 00:26:cb:94:44:c0 from Associated to Disassociated Scheduling deletion of Mobile Station: (callerId: 45) in 10 seconds apfMsExpireCallback (apf_ms.c:608) Expiring Mobile! Sent Deauthenticate to mobile on BSSID 00:26:cb:94:44:c0 slot 0(caller apf_ms.c:5094)
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

71

Deauthenticated Client
 WLAN Change
Modifying a WLAN in anyway Disables and Renables WLAN
apfSendDisAssocMsgDebug (apf_80211.c:1855) Changing state for mobile 00:1e:8c:0f:a4:57 on AP 00:26:cb:94:44:c0 from Associated to Disassociated Sent Disassociate to mobile on AP 00:26:cb:94:44:c0-0 (reason 1, caller apf_ms.c:4983) Sent Deauthenticate to mobile on BSSID 00:26:cb:94:44:c0 slot 0(caller apf_ms.c:5094)

 Manual Deauth
From GUI: Remove Client From CLI: config client deauthenticate <mac address>
apfMsDeleteByMscb Scheduling mobile for deletion with deleteReason 6, reasonCode 1 Scheduling deletion of Mobile Station: (callerId: 30) in 1 seconds apfMsExpireCallback (apf_ms.c:608) Expiring Mobile! apfMsExpireMobileStation (apf_ms.c:5009) Changing state for mobile 00:1e:8c:0f:a4:57 on AP 00:26:cb:94:44:c0 from Associated to Disassociated Sent Deauthenticate to mobile on BSSID 00:26:cb:94:44:c0 slot 0(caller apf_ms.c:5094)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

72

Deauthenticated Client
 Authentication Timeout
Auth or Key Exchange max-retransmissions reached
Retransmit failure for EAPOL-Key M3 to mobile 00:1e:8c:0f:a4:57, retransmit count 3, mscb deauth count 0 Sent Deauthenticate to mobile on BSSID 00:26:cb:94:44:c0 slot 0(caller 1x_ptsm.c:534)

 AP Radio Reset (Power/Channel)
AP disasassociates clients but WLC does not delete entry
Cleaning up state for STA 00:1e:8c:0f:a4:57 due to event for AP 00:26:cb:94:44:c0(0) apfSendDisAssocMsgDebug (apf_80211.c:1855) Changing state for mobile 00:1e:8c:0f:a4:57 on AP 00:26:cb:94:44:c0 from Associated to Disassociated Sent Disassociate to mobile on AP 00:26:cb:94:44:c0-0 (reason 1, caller apf_ms.c:4983)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

73

Deauthentication - Takeaway
 Client can be removed for numerous reasons
WLAN change, AP change, configured interval

 Start with Client Debug to see if there is a reason for a client‟s deauthentication  Further Troubleshooting
Client debug should give some indication of what kind of deauth is happening
Packet capture or client logs may be require to see exact reason

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

74

Client Debug – Tips and Tricks

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

75

Tips and Tricks
 Collect a client debug for an extended duration
Several roams, deauths, failures, etc…

 Use an enhanced text editor with filter or “find all”
I use Notepad++

 Find All
“Association Received” (will also pull reassociations) “Assoc Resp” “Access-Reject” “timeoutEvt”

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

76

Tips and Tricks

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

77

Tips and Tricks

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

78

Client Debug – Summary

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

79

Client Connectivity
 Unified Wireless Network: Troubleshoot Client Issues Document ID: 107585

 Configuration Issues
SSID Mismatch Security Mismatch Disabled WLAN
Unsupported Data-Rates Disabled Clients Radio Preambles

 Cisco Features - Issues with Third Party Clients
Aironet IE

MFP
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

80

802.11n Speeds
 Troubleshoot 802.11n Speeds Document ID: 112055

 Configuration Issues
11n Support Enabled

WMM is Allowed or Required Open or WPA2-AES 5Ghz Channel Width
2.4Ghz does not support 40-Mhz Channels

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

81

802.11n A-MPDU/A-MSDU
 Aggregation methods used could impact interop or performance

 WLC Default 11n Config:

802.11n Status: A-MPDU Tx: Priority 0............................... Enabled Priority 1............................... Disabled Priority 2............................... Disabled Priority 3............................... Disabled Priority 4............................... Enabled Priority 5............................... Enabled Priority 6............................... Disabled Priority 7............................... Disabled A-MSDU Tx: Priority 0............................... Enabled Priority 1............................... Enabled Priority 2............................... Enabled Priority 3............................... Enabled Priority 4............................... Enabled Priority 5............................... Enabled Priority 6............................... Disabled Priority 7............................... Disabled
Cisco Public

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

82

WLC Config Analyzer (WLCCA)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

83

What Is the WLCCA?
 It is a Post Sales tool
 Main objective: Save time while analyzing configuration files from WLCs

 Secondary objective: Carry out RF analysis  It is NOT a management or monitoring tool  Focused to work off-line to the WLC
 Not TAC supported  Development: wlc-conf-app-dev@cisco.com  General internal alias:wlc-conf-app@cisco.com  “Pet project”: no official Cisco product.

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

84

Where?
 Support Forums DOC-1373

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

85

Input Needed
 Complete config output from WLC
Show run-config

 It does not work with old “show running-config” or with TFTP backup, or with show tech

 The show run-config acts as “snapshot” of current config + RF state  Likely best to obtain config from SSH with
config paging disable

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

86

Functionality Overview - Checks
 Audit Checks
More than 100 config detail verifications Based on TAC/Escalation cases experience Some obvious, some hard to catch

No “change this” messages, some need “contextualization”

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

87

Functionality Overview
 Audit Checks

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

88

Functionality Overview
 Config View

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

89

WLCCA – High RF Index APs

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

90

Reducing CCI
 Turn off excess 2.4 radios. May want to do this gradually, e.g. turn off 20% of radios per attempt
 After turning off excess radios, could set DCA sensitivity to high  Let DCA/power settings settle down overnight.  See how things look in the morning  Repeat till you see the desired coverage in 2.4GHz

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

91

2.4GHz – Target Coverage
 Most all 2.4GHz radios are at power 2 - 5 (don't want 7 or 8)
 In all locations, you have coverage that looks like this (take these as guidelines, not gospel):
Hottest channel's AP is at least -67dBm
Next hottest AP on that channel is at least 19 dB below the hottest

Next hottest channel's AP is at least -67dBm
OK if next hottest AP on that channel is less than 19 dB below the hottest

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

92

5 GHz – Target Coverage
 Most all 5GHz radios are at power 1 – 3 (at least 14dBm)
Consider the RRM min power setting in 6.0 Consider a radically high tx-power-threshold, like -55 dBm

 8 – 12 channels in use (20 seem to be too many for the 792x to scan)  In all locations, seek this:
Hottest channel's AP is at least -67dBm
Next hottest AP on that channel is at least 19 dB below the hottest

Next hottest channel's AP is at least -67dBm
OK if next hottest AP on that channel is less than 19 dB below the hottest

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

93

Additional Troubleshooting

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

94

Additional Troubleshooting
 Wireshark Tutorial

 Clean Air SE-Connect / AP Sniffer Mode
 AP Join  RRM  Multicast/Broadcast

 Mobility  VoWiFi

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

95

Wireshark Tutorial

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

96

Wireshark Tutorial
 Default Wireshark view might look like this:

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

97

Wireshark Tutorial
 Newer versions of Wireshark have a feature for “Apply as Column”
This will take any decodable parameter and make a column

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

98

Wireshark Tutorial
 Within seconds your wireshark can also have:

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

99

Wireshark Tutorial
 Filtering data is just as easy

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

100

Wireshark Tutorial - CAPWAP
 User data is encapsulated in CAPWAP

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

101

Wireshark Tutorial
 Wireshark can also de-encapsulate CAPWAP DATA
Edit > Preference > Protocols > CAPWAP

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

102

Wireshark Tutorial
 With CAPWAP de-encapsulated you can see all the packets to/from client (between AP and WLC)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

103

SE-Connect – Clean Air AP Sniffer Mode

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

104

SE-Connect and Sniffer Mode
 Clean Air APs can be used in lieu of Spectrum Card for Spectrum Analysis
AP can be placed in SE-Connect mode for full functionality AP in local mode can be used now for Spectrum Analysis of current channel

 AP Sniffer Mode can be used in lieu of Wireless Sniffer
Packets can be sent from either radio upstream to a packet capture software (Wireshark or Omnipeek for example)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

105

Spectrum Expert with Clean Air

 Obtain Spectrum Key

 Connect to Remote Sensor

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

106

Spectrum Expert with Clean Air

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

107

Sniffer Mode AP
 Select channel to Sniff  Select destination for traffic

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

108

Sniffer Mode AP
 Omnipeek has a Remote Adapter to capture this data  Wireshark, just capture network adapter
NOTE: Wireshark does not open the port UDP 5000 PC will send ICMP Unreachables

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

109

Sniffer Mode AP
 With wireshark, filter !icmp.type == 3  Data (UDP 5000) still not intelligible yet
Decode as Airopeek

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

110

Sniffer Mode AP

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

111

AP Discover/Join

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

112

AP Discover/Join
AP Runs Hunting Algorithm to Find Candidate Controllers to Join

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

113

AP - Discover Process
 AP Discovery Req to known and learned WLCs  Broadcast
Reaches WLCs with MGMT Interface in local subnet of AP Use “ip helper-address <ip>” with “ip forward-protocol udp”

 Dynamic
DNS: cisco-capwap-controller

DHCP: Option 43

 Configured (nvram)
High Availability WLCs – Pri/Sec/Ter/Backup Last WLC All WLCs in same mobility group as last WLC Manual from AP - “capwap ap controller ip address <ip>”
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

114

AP - Discover Process

X
 Discover Request sent to all methods the AP knows  Discover Response sent from all WLCs that received the Discovery Request
broadcast
115

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

AP – WLC Selection/Join
 WLCs send Discovery Response back to AP
Name, Capacity, AP Count, Master?, AP-MGR, Load per APMGR

 AP selects the single best WLC candidate from
High Availability Config: Primary/Secondary/Tertiary/Backup

Master Controller Greatest available capacity Ratio of total capacity to available capacity

 AP sends single Join Request to best candidate
WLC responds with Join Response AP joins and receives config (or downloads image if not correct)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

116

Troubleshooting AP Discovery/Join
 “Lightweight AP (LAP) Registration to a Wireless LAN Controller (WLC)”, Document ID 70333  Make sure time on WLC is accurate!  From AP:
Debug ip udp Debug capwap client events

 From WLC
Debug mac addr <AP ethernet mac> Debug capwap [event/error/packet] enable
Debug pm pki enable

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

117

RRM

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

118

RRM
 There are usually only two common scenarios or issues involving RRM

 APs not changing channel
Check if other APs are in each others neighbor list

 APs not changing power
Nearby APs list meets the general rule of RSSI from 3rd closest AP is better than TPC Threshold

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

119

RRM Debugs
 WLC – debug airewave-director <?>

 AP
debug capwap rm mesurements debug capwap rm rogue

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

120

RRM Show AP Auto-RF (In Run-Config)
 show ap auto-rf [802.11a/b] <AP Name>

 Load Information
Receive Utilization.. 0 % Rx load to Radio

Transmit Utilization.. 2 % Channel Utilization.. 12 %

Tx load from Radio % Busy

 Nearby APs
AP 00:16:9c:4b:c4:c0 slot 0.. -28 dBm on 11 (10.10.1.5) AP 00:26:cb:94:44:c0 slot 0.. -32 dBm on 11 (10.10.1.4)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

121

Broadcast/Multicast

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

122

Broadcast/Multicast

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

123

Broadcast/Multicast
 AP Multicast Mode – Multicast
Address must be unique among WLCs

 Broadcast Traffic is delivered via the Multicast Mode  AP/WLC/Client Subnets must be Multicast enabled
For Multicast Mode - Multicast

 Quick check for Multicast is to confirm that MulticastUnicast mode works

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

124

Broadcast/Multicast
 AP Show Commands
Show capwap mcast Show capwap mcast mgid all

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

125

Client Mobility

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

126

Mobility—Intra-Controller
 Client roams between two APs on the same controller

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

127

Mobility—Inter-Controller (Layer 2)

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

128

Mobility—Layer 3
 Layer 3 roaming (a.k.a. anchor/foreign)
New WLC does not have an interface on the subnet the client is on New WLC will tell the old WLC to forward all client traffic to the new WLC

 Asymmetric traffic path established (deprecated)  Symmetric traffic path

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

129

Mobility—Messaging Flow
 When a client connects to a WLC for the first time, the following happens:
New WLC sends MOBILE_ANNOUNCE to all controllers in the mobility group when client connects
Old WLC sends HANDOFF_REQUEST New WLC sends HANDOFF_REPLY

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

130

Debug Client <Mac Address>

Mobility— L2 Inter WLC

Debug Mobility Handoff Enable

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

131

Debug Client <Mac Address>

Mobility— L3 Inter WLC

Debug Mobility Handoff Enable

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

132

Debug Client <Mac Address>

Mobility— L3 Inter WLC

Debug Mobility Handoff Enable

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

133

Mobility— L3 Handoff Ignored
*mmListen: Mobility packet received from: *mmListen: 10.4.22.55, port 16666 *mmListen: type: 3(MobileAnnounce) subtype: 0 version: 1 xid: 783 seq: 1453 len 116 flags 0 *mmListen: group id: e42cb3a9 87f62b45 57c0f8a3 92747b23 *mmListen: mobile MAC: 00:23:33:41:71:10, IP: 0.0.0.0, instance: 0 *mmListen: VLAN IP: 10.4.23.97, netmask: 255.255.255.0 *mmListen: Switch IP: 10.4.22.55 *mmListen: Handoff Virtual IP Mismatch, Local = 1010101, Request = 1020304 **** Handoff Request Ignored *apfReceiveTask: 10.4.122.127 RUN (20) State Update from Mobility-Complete to Mobility-Incomplete *apfReceiveTask: Mobile 00:23:33:41:71:10 associated with another AP elsewhere, delete mobile *apfReceiveTask: 10.4.122.127 RUN (20) mobility role update request from Local to Handoff Peer = 0.0.0.0, Old Anchor = 10.4.130.70, New Anchor = 0.0.0.0 *apfReceiveTask: Clearing Address 10.4.122.127 on mobile *apfReceiveTask: apfMsRunStateDec *apfReceiveTask: 10.4.122.127 RUN (20) Change state to DHCP_REQD (7) last state RUN (20) *apfReceiveTask: apfMmProcessDeleteMobile (apf_mm.c:548) Expiring Mobile! *apfReceiveTask: Mobility Response: IP 0.0.0.0 code Handoff Indication (2), reason Client handoff successful anchor retained (0), PEM State DHCP_REQD, Role Handoff(6) *apfReceiveTask: apfMsExpireMobileStation (apf_ms.c:5009) Changing state for mobile 00:23:33:41:71:10 on AP 10:8c:cf:eb:69:80 from Associated to Disassociated *apfReceiveTask: Deleting mobile on AP 10:8c:cf:eb:69:80(1) *pemReceiveTask: 0.0.0.0 Removed NPU entry.
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

134

Mobility Group vs. Mobility Domain
 Mobility Group - WLCs with the same group name
L2/L3 Handoff

Auto Anchoring Fast Secure Roaming APs get all of these as a Discover candidate

 Mobility Domain - WLCs in the mobility list
L2/L3 Handoff

Auto Anchoring

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

135

Mobility Data/Control Path
 Sent between all WLCs, by member with lowest MAC
Control Path = UDP 16666 (30 Seconds) Data Path = EoIP Protocol 97 (10 Seconds) debug mobility keep-alive enable <IP Address>

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

136

Voice over WiFi

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

137

VoWiFi
 Wireless IP Phone Deployment Guide
http://www.cisco.com/en/US/docs/voice_ip_comm/cuipph/79 25g/7_0/english/deployment/guide/7925dply.pdf

 Best Practices
-67 dBm signal with 20-30% cell overlap
802.11A CCKM for Fastest Roaming Avoid designs where AP is seen at superb signal, but drops off instantly

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

138

VoWiFi - Troubleshooting
 Must know if problem occurs during roaming events or when no association change takes place

 If no change in connection
Interference Coverage loss with no other candidate End to End QOS missing/problem

 If during roaming event
How long did the roam take?
Does the client associate to another AP again within seconds?

Does the client associate to the same AP again? Is the phone roaming to the designed next candidate?
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

139

VoWiFi - Troubleshooting
 Define a reproducible area where you believe you have perfect voice coverage but have problems
 Place phone in Neighbor List Mode (On a call)
Real Time current AP RSSI and candidate list Confirm AP as next best candidate is realistically a good candidate

Confirm devices roams to correct candidate where the intended design specifies

 Watch out for sudden drops in coverage

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

140

VoWiFi - Debugs
 Phone can Trace (debug) to file or syslog
Recommend USB Connection and SYSLOG
Configured via GUI Enable Debug level for Kernel, WLAN MGR, WLAN Driver

 WLC Debugs
Debug client <mac> Debug cac all enable

 Wireless Packet Captures

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

141

Summary

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

142

Summary
Client
WLC - show run-config, debug client <mac>, debug dhcp message enable, debug dot1x <?> enable, debug aaa <?> enable, AP - Show tech, show controller D<0/1> Data - Driver/Supplicant Logs, Wireless Capture, AAA Logs, DHCP Logs

Webauth
WLC - (Client debugs), debug webauth enable <IP>, debug pm ssh-appgw enable, debug pm ssh-tcp enable Client - local capture

Mobility
WLC - debug mobility handoff enable, debug mobility keepalive enable <IP> Data - Wired capture

AP Join
WLC - debug capwap [events/error/packet] enable AP - debug capwap client events, debug ip udp Data - Wired capture

RRM
WLC - show run-config, debug airewave-director <?> AP - debug capwap rm measurements, debug capwap rm rogue

Multicast/Broadcast
AP - show capwap mcast, show capwap mcast mgid all Data - Infrastructure Configuration

Voice
WLC - (Client debugs), debug cac all enable Data – Wireless capture, Phone traces
BRKEWN-3011
© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

143

Summary
 Links:
Understanding Debug Client on Wireless LAN Controllers (WLCs) Document ID: 100260

Unified Wireless Network: Troubleshoot Client Issues Document ID: 107585 Troubleshoot 802.11n Speeds Document ID: 112055 Troubleshoot a Lightweight Access Point Not Joining a Wireless LAN Controller Document ID: 99948

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

144

Complete Your Online Session Evaluation
 Receive 25 Cisco Preferred Access points for each session evaluation you complete.  Give us your feedback and you could win fabulous prizes. Points are calculated on a daily basis. Winners will be notified by email after July 22nd.

 Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center.
 Don‟t forget to activate your Cisco Live and Networkers Virtual account for access to all session materials, communities, and on-demand and live activities throughout the year. Activate your account at any internet station or visit www.ciscolivevirtual.com.

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

145

Visit the Cisco Store for Related Titles http://theciscostores.com

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

146

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

147

Thank you.

BRKEWN-3011

© 2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

148