Identifying virtual machines attempting to use promiscuous network mode on ESXi

Products

• VMware vSphere ESXi

Issue/Introduction

Messages from the ESXi VMkernel indicate that a virtual machine is trying to promiscuously capture all network traffic on a vSwitch portgroup, but the vSwitch portgroup policy is configured to deny promiscuous mode. No warning is emitted for virtual machines attached using promiscuous mode when it is permitted by the effective vSwitch portgroup policy.

By default, a guest operating system's virtual network adapter only receives frames that are meant for it. Placing the guest's network adapter in promiscuous mode causes it to receive all frames passed on the virtual switch that are allowed under the VLAN policy for the associated portgroup. This can be useful for intrusion detection monitoring or if a sniffer needs to to analyze all traffic on the network segment. 

If software within a virtual machine is attempting to put the guest network adapter in promiscuous mode, contrary to the defined vSwitch portgroup policy, it may be necessary to investigate if the virtual machine is running undesired software.

This article provides steps for identifying a virtual machine that attempts to promiscuously capture all network traffic on a vSwitch portgroup, based upon the VMkernel log messages. Investigation within a virtual machine is outside the scope of this article.

Symptoms:

• The VMkernel logs at /var/log/vmkernel or /var/log/messages contain entries similar to:
  
  cpuN:nnnn)etherswitch: L2Sec_EnforcePortCompliance: 0xnnnnnnnn: peer not allowed promiscuous, revoking setting

Environment

VMware vSphere
VMware ESXi

Resolution

The log message identifies a specific virtual network interface on an ESX or ESXi host by its PortID. This identifier is not displayed in the vSphere Client, but can be associated with a virtual machine by searching the performance metrics for all virtual machine network interfaces on the host.

To identify which virtual machine is attempting to enter promiscuous mode, convert the logged hexadecimal port identifier to a base 10 number and locate the virtual machine using that port number.

1. From the VMkernel log message, identify the virtual machine world number and port identifier. For example:

cpuN:12345)etherswitch: L2Sec_EnforcePortCompliance: 0x5000003: peer not allowed promiscuous, revoking setting
   
   In this example, the world number is 12345 and the port identifier is 0x5000003.


2. Convert the hexadecimal port identifier to base-10 decimal. For example, base-16 port number 0x5000003 is 83886083 in base-10.
3. Map the base-10 decimal port identifier to a virtual machine name using one of the following methods:

Identifying a virtual machine by port identifier using the PowerCLI

To identify a virtual machine by its base-10 port identifier using the PowerCLI:

1. Open the PowerCLI command prompt.
2. Connect to the ESXi host which reported the warning using the command:
   
   Connect-VIServer -Server <ESXHostnameOrIPAddress>
   
   Note: Authenticate using an administrative user, such as root.
   
   
3. Fetch the esxtop performance counter name matching the port identifier using a command similar to:
   
   Get-ESXTOP -CounterName NetPort | select PortID,WorldLeader,ClientName | Where { $_.PortID -eq "83886082" } | ft -AutoSize

   The output appears similar to:
   
   PortID WorldLeader ClientName
------ ----------- ----------
83886082 12344 VirtualMachineName
   
   
4. The virtual machine world number and name are available. Locate the virtual machine in the vSphere Client and optionally begin investigation.

 

Identifying a virtual machine by port identifier using esxtop

The esxtop command-line performance monitoring utility is included the local ESX/ESXi console. For more information on use of esxtop.

To identify a virtual machine by its base-10 port identifier using esxtop:

1. Open a console to the ESXi host.
2. Define a variable containing the port identifier using a command similar to:
   
   PORTID="83886082"
   
   
3. Fetch the performance counter name matching the port identifier using the command:
   
   esxtop -n 1 -b | tr ',' '\n' | grep -o 'Network Port(.*:$PORTID:.*) ' | sort -u
   
   Note: Authenticate using an administrative user, such as root.
   
   The output appears similar to:
   
   Network Port(vSwitchName:83886082:12344:VirtualMachineName)
   
   
4. The virtual machine world number and name are available. Locate the virtual machine in the vSphere Client and optionally begin investigation.

Identifying a virtual machine by port identifier using esxcfg-info

The local esxcfg-info command-line utility is included the local ESX/ESXi console. This command provides a view of the internal state of various components.

To identify a virtual machine by its base-10 port identifier using the output of esxcfg-info:

1. Open a console to the ESX or ESXi host. 
   
   
2. Parse the section of esxcfg-info output matching the port identifier using a command similar to:
   
   esxcfg-info --network | grep -A 3 "Port Id.*83886082"
   
   The output appears similar to:
   
   |----Port Id..........83886082
|----World Leader.....12344
|----Client Name......VirtualMachineName
   
   
3. The virtual machine world number and name are available. Locate the virtual machine in the vSphere Client and optionally begin investigation.