Using Editcap to clean up a Packet Capture

This is just a quick one. I recently needed to filter out a couple packet captures of unneeded frames/packets for some training material. I had unfortunately captured a crap ton of data though and really didn’t want to post the whole 20M pcap file. Thankfully google is great and so I ran across the wireshark function called “editcap”. There is a lot that you can do with editcap as you can see in the following link:

Editing trace files with Editcap

However all I needed to do was remove the frames before and after the area of interest. To do this on a macbook:

editcap -r {source file} {destination file} {packets/frames you want to keep} 

e.g.

editcap -r wirelesseapol.pcap wirelesseapolfiltered.pcap 790-900

Screen Shot 2017-07-08 at 12.26.22 PM

This will take the wirelesseapol.pcap, remove frames 1-789 and 901 – end, leaving me 790-900 in a new packet captures called wirelesseapolfiltered.pcap.

EXTREMELY HANDY.

Following the 802.1X AAA process with Packet Captures

EDIT: After chatting with David Westcott (@davidwestcott) I have made a few additions to this post. He has graciously asked that I add a little more details including the packet captures so everyone can follow along. This was a great idea, so please enjoy!

802.1X is typically the first step in one of the more advanced security implementations you will have to dip your toes into when moving your network to a secure state. That being said a lot of times we as engineers get stuck in a state of understanding enough to be dangerous and not enough to be highly successful and more importantly, capable. I personally learned a lot about dot1x via trial and error through implementations in the past as well as lab time at home and during my CCIE studies.

So the question is, how do we become masters of a protocol that is literally quite capable of being the success or demise of a network’s security and operation? 

…By taking advantage of the resources that are out there as well as practical labs. One thing to keep in mind is that while every vendor of a dot1x solution (Cisco, Aruba, FreeRADIUS, Microsoft) has a certain way of going about authentication, they all fall back to the same protocol: 802.1X/RADIUS. As long as you can understand that protocol to a fairly deep level you can troubleshoot any RADIUS environment.

A few great resources I have used and/or reviewed are:

Lab Minutes

Cisco ISE Communities Aggregate Post

Meraki Common RADIUS Error Codes with Microsoft NPS

Packet6 FreeRADIUS Install Guide

Katherine McNamara’s Network Node Blog

Now that being said I also recommend learning about not only how to debug the RADIUS servers via logs and troubleshooting tools, but being able to understand what it looks like from a packet capture perspective both wired and wireless and all the information you can glean from a pcap.

Overview of RADIUS/Dot1x Process

EAP & Radius Process

The above diagram is utilizing EAP-PEAP, this may look a little different depending on the Auth method.

Getting Started with Captures:

There are technically 2 points that you can grab packet captures, both of which will provide relevant information for troubleshooting. If you capture at Point 1 in the diagram below, the frame exchanges will be using EAPoL as a protocol. If you capture at Point 2 we will be seeing the traffic as RADIUS messages. That being said lets dive in!

dot1x packet Captures

Example Environment Details:

In this write up I used a Nexus 6P, Cisco Meraki MR33, and a Cisco ISE server. Included below are links to download the .pcap files.

Wireless EAPoL and Wired RADIUS packet capture files

Please download the files in the link above if you would like to follow along and review.

Wireless EAPoL capture clients:

STA (Supplicant) = 00:9a:cd:b7:c9:f0

AP (Authenticator) = e2:55:2d:f2:d1:54

Wired Radius Capture Clients:

AP (Authenticator) = 10.10.0.102

RADIUS Server (Authentication Server) = 10.10.1.37

Point 1: Supplicant to Authenticator (EAPoL)

You can capture between the client device (supplicant) and the access device (authenticator). At this point in the network you are looking at EAPoL messages. This is the communication method utilized that provides the Authenticator and the Client a line of communication prior to network access.

This is what the capture will look like:

Screen Shot 2017-07-08 at 11.22.32 AM

To perform this capture from a wireless perspective you will need either an access point capable of dumping/monitoring traffic, or a client that is capable of turning its adapter into monitor mode. Most of the time for wireless captures I use a Macbook Pro and Adrian’s Airtool App. This way is fairly flexible and quick.

If you are capturing for a wired authentication, you will want to perform a port SPAN or mirror to receive the needed frames.

Once you have performed the frame capture, you will want to open it in Wireshark and apply a few filter/s to get only the most useful info.

Example:

Screen Shot 2017-07-08 at 11.24.08 AM

In this case we combined 3 filters by the use of the &&. First we want to make sure we only see the eapol messages. We then want to filter on the Access point or switch as well as the client. This can be done in the case of wireless with wlan.addr == {mac address}. Quite often you will see wlan.ta/ra/dst etc. however in this case we want to simply isolate the mac addresses, not which position they are in the frame.

For wired we would still use the filter “eapol” and instead of wlan.addr we would use eth.addr.

The EAPoL portion of communication will vary depending on the authentication type. In my examples, we are using EAP-PEAP w/EAP-MsCHAPv2. This is a fairly standard form of authentication, and from a .pcap would resemble closely to EAP-TTLS w/PAP.

The useful portions that can usually be derived from a pcap are:

EAP-Identity Response:

Screen Shot 2017-07-08 at 11.18.59 AM

In this frame (frame #29 in the .pcap) you can see the Client’s (Supplicant) Identity being used of “employeealex“. This can be extremely useful when trying to determine if the supplicant is going to authenticate as the user or the machine account as well as what the user could be typing into the username prompt.

EAP Auth Method Negotiation and Credential Exchange:

Screen Shot 2017-07-08 at 11.52.33 AM.png

The first message in this clip is the server’s proposal of EAP-TLS (frame #32 in the .pcap), then the client’s response with, “Hey what about EAP-PEAP?” In some situations, depending on the RADIUS server configuration, the client may try to propose a method that is not permitted or supported by the server. This is where you would see that negotiation fail, and ultimately an Access-Reject/EAP-Failure.

However in this capture you can see the client and server negotiate EAP-PEAP. Once that is completed, the Server will present the client with its certificate. If the client does not trust the certificate from the server, and the user does not accept the certificate, the exchange will fail after the first frame or two of the handshake.

In this situation however the client trusts the server certificate and the two endpoints secure the medium with a TLS tunnel. Once secured you should notice that the protocol becomes purely TLS and since the traffic is encrypted, we can only see that the frames are “Application Data”. This is the point at which the client and server are exchanging inner authentication data such as EAP-MsCHAPv2 or EAP-TLS.

EAP Success(wired and wireless) and 4 Way Handshake (when the client is wireless):

Screen Shot 2017-07-08 at 11.20.25 AM

Once the client has been successfully authenticated and authorized, there is an EAP Success message sent back (frame #78) to signify the end of the process. If this is a wired client the process is over and the client is able to start transmitting and receiving data frames. If this is a wireless client, the station will utilize a few EAP attributes and the AP will utilize two MPPE key attributes in the Radius Access-Accept response to perform the 4 way handshake and create the encryption keys for secure communication (Frames 80, 82, 85, and 87).

Point 2: Authenticator to Authentication Server (RADIUS)

In the previous section we were only able to see the authentication process and not the authorization or accounting as that communication is only between the authenticator and authentication server. Luckily you are also able to capture traffic from the Access/Authenticator Device (i.e. switch/WLC/AP) directly to the Authentication Server (RADIUS server). I HIGHLY recommend capturing this traffic at the same time as your client EAPoL capture so you can reference/correlate the same data. At this point in the process you will be looking at RADIUS messages. These messages are typically not encrypted unless using AES-KeyWrap or RADSEC which is not common today unless in a highly secured environment or public environment where traffic is inherently insecure.

This is what the capture will look like:

Screen Shot 2017-07-08 at 11.32.10 AM

To perform this capture, we would want to span the traffic from the Authenticator or from the RADIUS server. The choice is yours. Once we have spanned the port we would perform a capture as usual and open in wireshark.

There are a few very useful filters in this capture as well that we will want to use to weed out the undesirable packets.

Example:

Screen Shot 2017-07-08 at 11.33.12 AM

As you can see we are not getting too complicated with our filters. We just want to combine filters once again with the && delimiter. Then we also want to filter on radius as the protocol, and the ip addresses of the Authenticator and Authentication server with ip.addr == {ip address}.

As you can see from the capture example above, the communication between the access device and RADIUS server are comprised of a ton of Access-Requests with Access-Challenges as the response from the RADIUS server. One thing to note is that all of the communication between the client and server during authentication is encapsulated in a RADIUS request. This can be extremely helpful in following how the Authenticator translates the information to the RADIUS server, as well as all the extra information that is appended. Now on that note….

The useful portions of the RADIUS exchanges are:

The initial Access-Request packet:

Screen Shot 2017-07-08 at 11.36.56 AM

In this packet (in the capture #31) we can see a ton of useful information in the form of Attribute Value Pairs. Here we can see for instance the username that is being sent as well as the connectivity information. When troubleshooting RADIUS environments it is a good idea to verify the NAS-Port-TypeService-Type, and Called-Station-ID. If you have watched any of my videos that I have created on how to configure Cisco ISE you will know we use these three attributes quite frequently for granular authorization policies.

One other piece of information that is extremely useful for troubleshooting is the NAS-IP-Address. This information is typically what is used to differentiate RADIUS clients in the RADIUS server. In my video on Network Devices and Groups you can see where in Cisco ISE we configure the Authenticator devices via their IP address. This information does need to match the NAS-IP-Address. If it does not the radius server will most likely not respond as the RADIUS client cannot be found.

The Final Access-Accept Packet:

Screen Shot 2017-07-08 at 11.38.05 AM

This packet (#55) could also be the Access-Reject packet if the authentication failed. In this instance however we passed authentication. As part of the authorization policies I even returned a Filter-ID that specifies a Group Policy in the Meraki Dashboard only giving me BYOD-Access. There can be many AVPs included depending on the platform including Airespace ACLs, AVC Rulesets, QoS parameters, VLAN assignment, User Roles, and much much more.

For those that are not familiar, in a Cisco Meraki wireless environment we can use Group Policies to specify different access through L3-L7 firewall rules as well as perform traffic shaping, policing, and QoS remarking on specific application traffic. 

RADIUS Accounting Messages:

Screen Shot 2017-07-08 at 11.39.20 AM

These messages (#56 & 57) provide the RADIUS server with connectivity information for authenticated and authorized clients. Notice the UDP ports change from 1812 to 1813 when the communication changes to Accounting. These are not mandatory for 802.1X however are useful in providing current connectivity stats to the RADIUS server. These will also show when a client disconnects with an Accounting-Stop message. As you can see in the above screenshot the message also include Event-Timestamps, and Session-IDs on top of the other connectivity information. You can also find the client IP address information under the Framed-IP-Address field.

In closing…

I hope this breakdown of the process was informative. It took me a while to honestly get to the point of being able to grab captures and decipher what they mean. If all of this is above your head just remember practice makes perfect. I recommend if you would like to get deeper into RADIUS to install something like FreeRADIUS and play around. Hands on experience beats any other form of study.

Thank you for reading!

 

Single SSID BYOD Onboarding

**This video builds on top of the previous video of BYOD with Device Registration and Native Supplicant Provisioning. So please be sure to watch it for configuring the certificate templates and some of the SSID configuration. **

In this video we configure ISE and wireless with a single SSID for WPA2-Enterprise to perform device registration and EAP-TLS provisioning.

BYOD with Device Registration and Native Supplicant Provisioning

Aside from standard radius authentication and guest access, ISE is also useful for secure BYOD access. In this video I walk through building an onboarding SSID and Secure SSID in dashboard. Then in ISE we configure the guest portal, certificate template, native supplicant provisioning profile, and rule sets to put it all in play. Once that is done we test and verify access.

iPerf for Performance Testing

Over the years of working with customers and other fellow engineers I have run into a number of situations where wired and wireless performance testing was requested. This has ranged from “look how much bandwidth I can push through this AP” to just level setting expectations and verification. Typically I see people running various internet speed tests to verify performance. This is in my opinion a good way to disappoint yourself and all involved. Internet speed tests are never a guarantee and always rely on not only your internet bandwidth and availability (which can vary depending on usage and load from others), firewall capability, but most importantly – speed test server availability.

Believe it or not, anyone with a gigabit pipe to the internet, static IP, and the ability to map DNS can be a speed test server node. This does count out most homelabbers, but I have witnessed a few organizations that ran speed test nodes that, in my opinion, should never have been qualified. So that leaves me with this: Learn to use tools that can run locally on your network. This allows you to properly test your environment without relying on someone else’s server to verify that you have adequately designed, deployed, and tuned your environment to support the necessary bandwidth requirements for your users.


The two most common scenarios where iPerf fits the bill:

Throughput testing – This format can be used for performance testing mesh links and the unfortunate events where a customer has been sold on marketing “numbers” instead of real world performance.

Client load testing – This is often misconstrued as throughput testing. However, this should be based on number of clients + expected SLA. This is often seen with 30+ clients connecting to an AP and running some sort of performance test.


To get started using iPerf you have to remember that it operates in a client / server configuration. You never want your server to be the bottleneck on the network unless you have no other choice. That being said I highly recommend a device with at least a gigabit adapter which can be small and compact like the ODroidC2 (Thanks WLPC!).

1. Download iperf for your devices.

For Mac and Windows – You can download the application from: https://iperf.fr/iperf-download.php . If you are savvy with Windows 10 you can also install the linux subsystem for windows and then “apt-get install iperf3“. With Mac you could also use homebrew and install via “brew install iperf3“.

For linux – Just use your friendly package manager: yum/apt/pacman and install your iperf3 package.

For Android – There are many apps available via the Play Store as well as Access Agility’s WiFiPerf endpoint app: (https://www.accessagility.com/wifiperf)

For iOS – There are a few apps available via the app store or once again using WiFiPerf from Access Agility: (https://www.accessagility.com/wifiperf)


2. Choose which side you would like to have run as the server and the client.

I highly recommend running the remote devices as servers. This way you can easily initiate multiple iperf sessions without having to get up and touch each device. The key is to utilize the option -R so that the process is run in reverse as if the wired machine is the server.

Screen Shot 2017-04-19 at 11.05.55


3. Choose the options that you may want to implement to tailor the test to your scenario. Here are some common ones I use:

-w {TCP Window in KB or MB} #### This sets the amount of data that can be sent before there needs to be an acknowledgement. This can be tricky as loss will affect throughput as the packets need to be retransmitted. That being said, fewer ACKs can mean more throughput as there is less time spent sending and receiving acknowledgements on the network. It is always good to run a few tests to get a median number. I usually stick with 1M – 2M.

-P {number of concurrent client streams} #### This creates multiple streams. I usually do anywhere from 5-10 when I do want to run multiple streams. This will allow iperf to spread the streams across multiple processors and could be handy if you are on a low clock CPU with multiple cores. YMMV

-S {IP type of service value} #### Check this handy page out for the values to use: https://www.tucny.com/Home/dscp-tos. This is an option for changing the traffic markings to, for example: Expedited Forwarding with “0xB8”. I’ve used this for testing DSCP marking preservation quite often.

-t {time in seconds to run the test} #### This is great to set for a longer period of time when you are trying to initiate multiple concurrent tests. I usually run for 120-180 seconds when I am running with multiple devices.

-u #### runs as UDP. This can allow you to stress test a link to find out what speed you are able to attain without loss as well as displays jitter and packet loss. Highly recommend pairing with the next option.

-b {kbps or mbps} #### sets the kbps or mbps of the transmission. If running with -P for multiple streams it will cap each stream at the set value. example: “-b 10M“. This is a great use for SLA testing where you are expecting to provide X number of clients with a certain amount of bandwidth.

-i {time in seconds} #### update interval on screen; default is 1 second. Can be useful when running large numbers of concurrent tests to increase the interval, or disable with “-i 0”

–logfile {filename} #### outputs what would normally be in your terminal to a log file of your choosing.


4. Run your tests and review the output. 


The one fun thing about iPerf is that you get to do a bit of trial and error testing typically to find what settings work for the configurations you’ve implemented. It can be helpful to have a few examples which I will include below. As one of my friends said recently, “step 1. copy/paste code. step 2. troubleshoot error message. step 3. read initial post where code was copied from.” That being said don’t be afraid to look at the iPerf3 manpage at: https://www.mankier.com/1/iperf3 to really get a deep dive and create your own tests.


Examples of Client configurations:

Throughput testing:

“iperf 3 -c {server ip} -w 1M -P 5 -t 60 -i 5” #### 1M TCP windows, 5 parallel streams, for 60 seconds with 5 second update intervals.

“iperf 3 -c {server ip} -w 1M -P 5 -t 60 -i 5 -S 0x08” #### the same as above with traffic marked as Background CS1 / DSCP 8.

“iperf 3 -c {server ip} -u -t 60 -i 5 -b {expected throughput of link}” #### The nice thing here is you will be able to see if there was loss or jitter on the link.

For example if you were testing throughput on an expected 500mbps line to a server at 10.10.1.50 your command could be:

iperf3 -c 10.10.1.50 -u -t 30 -b 500M

and the neat output:

– – – – – – – – – – – – – – – – – – – – – – – – –

[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams

[  4]   0.00-30.00  sec  1.74 GBytes   499 Mbits/sec  0.015 ms  2/1292334 (0.00015%) 

[  4] Sent 1292334 datagrams

SLA Testing with a client connecting to multiple servers:

“iperf3 -c {server ip} -u -b 10M -t 60 -i 5 -R” #### this is a fairly basic udp test attempting at 10mbps for 60 seconds with updates every 5 seconds.

“iperf3 -c {server ip} -u -b 10M -t 60 -i 60 -R –logfile test1.txt” #### runs iPerf with UDP at an attempted 10Mbps for 60 seconds in reverse. The -i matched with the command runtime and logfile command will only output the results to a text file called test1.txt

The above tests can also be run as TCP with the -b option. The one nice thing about running as UDP is you can see how much loss occurred in transmission.


 

Good Luck!! And as always please leave a comment below if you find any mistakes or have anything to add!