How to perform packet captures to troubleshoot a VeloCloud Virtual Edge on ESXi.

Products

• VMware SD-WAN by VeloCloud

Issue/Introduction

This KB will help to identify inbound/outbound traffic of a Virtual Edge on ESXi, useful for activations or any other network issue.

Environment

VMware SD-WAN by VeloCloud

Resolution

1- First, you have to get the World ID of the virtual machine:

# esxcli network vm list | grep “vmName”

You can always filter the output by using grep and the vm name.

2- get the Port ID (Switchport) of the VM using the world ID, worldID was collected on last output “2115346”.

# esxcli network vm port list -w WorldID

 

Have in mind that on a virtual edge, you will have more than one interface, therefore more than one Port ID, you can match the proper Port ID to capture using the MAC address by comparing this to your VCE ifconfig info. For example, I would like to capture on GE3, which is the WAN interface for the edge who’s MAC address is F0:8E:DB:E8:7A:85.
 

Basic syntaxes, capturing inbound/outbound traffic on the VM. Access to ESXi CLI is required to run the command

Command:

pktcap-uw --switchport PortID --dir 2 -o /tmp/capture.pcap
 

dir 2= Means that packet capture will run bi-directional

PortID was collected on previous step, this is the logical port that the VM is using.

 

On above example, we captured 21 packets and saved them to file /tmp/capture.pcap.

 

We can also capture on NIC that is in use by the virtual machine, inbound/outbound.

 

 

On this case, uplink is vmnic6.

Command:

pktcap-uw --uplink TeamUplink --dir 2 -o /tmp/capture.pcap

Example:

pktcap-uw --uplink vmnic6 --dir 2 -o /tmp/capture.pcap

You can also run the same capture on the physical NIC's used by the ESXi.
 

There are several ways to determine the correct NIC, an easy one is:

 

 Login into the ESXi > Networking > vSwitch in question > vSwitch topology.

 

 

 

Vmnic0 and vmnic1 are physically in use by the ESXi.

It is a good practice to add filters when capturing on ESXi NIC's, normally there is a considerable amount of traffic here.
Capturing and Tracing Network Packets by Using the pktcap-uw Utility
 

Example with filters:

 pktcap-uw --uplink TeamUplink --dir 2  --srcmac mac_address --dstmac mac_address -o /tmp/capture.pcap

 

 Have in mind that there are many ways to retrieve the information of vmnic IDs.

 

 All above examples will save the captures to /tmp/capture.pcap or the location that would like to setup, you can also run a live capture without saving it to a file.

 Example:

pktcap-uw --uplink TeamUplink --dir 2  --srcmac mac_address --dstmac mac_address -o - | tcpdump-uw -enr –

 

How to retrieve your packet capture from the ESXi:

Few different ways to do this, easiest one is to use the SCP commands, or WinSCP client for Windows.

 

WinSCP:

Connect to the ESXi using WinSCP client, go the file directory and download the file to the local PC.

 

SCP:
scp /tmp/capture.pcap [email protected]:/remote/directory/capture.pcap

 

 This option will be useful from SCP client, Linux works perfectly.

 

ESXi to customer datastore:

There are scenarios where customer is unable to install WinSCP or run SCP, another option is to copy the file to a share Datastore which the ESXi has access to, then the customer can get the file from there. 

 

 You can list the datastore by running the command ls /vmfs/volumes/

 

 

 I will copy from /tmp/ to /vmfs/volumes/NSX1SiteA/ which is a datastore that the ESXi has access to.