Monitoring network traffic from within a virtual machine on a VMware vSphere ESX/ESXi server

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Software within a virtual machine can be used to monitor or capture network traffic passing through a vSwitch Portgroup on the same ESX/ESXi host. Traffic may be limited to the traffic intended for other virtual machines on the same host, or come from a mirrored port on an upstream physical switch.

Configuration and the usage of the third-party monitoring software within a virtual machine, or upstream switch configuration, is outside the scope of this article.

Environment

VMware ESXi 4.0.x Embedded VMware vCenter Server 4.0.x VMware vCenter Server 4.1.x VMware ESXi 4.1.x Installable VMware ESXi 3.5.x Embedded VMware VirtualCenter 2.5.x VMware ESX 4.0.x

VMware ESXi 3.5.x Installable VMware ESXi 4.1.x Embedded VMware ESX Server 3.5.x VMware ESXi 4.0.x Installable VMware ESX 4.1.x VMware ESX Server 3.0.x

Resolution

Capturing and monitoring of network traffic using third-party monitoring software within a virtual machine is only possible if the network traffic is made available to that virtual machine.

The default security policy for VMware vSphere virtual machines denies the usage of promiscuous mode to capture traffic on a vSwitch portgroup. Traffic that is not addressed to the monitoring virtual machine's network interface's MAC address is not be received or captured by the virtual machine or the third-party monitoring software. For more information, see How promiscuous mode works at the virtual switch and portgroup levels.

To capture or monitor network traffic exposed to the ESX/ESXi host on a specific portgroup:

1. Configure the vSwitch and the portgroup to allow virtual machines to use promiscuous mode. For more information, see Configuring promiscuous mode on a virtual switch or portgroup.
2. Deploy a virtual machine with the third-party monitoring or packet-capturing software.
3. Attach the virtual machine's virtual network interface to the monitoring portgroup. 
4. Configure the third-party software to capture traffic on the corresponding interface.

To capture or monitor network traffic external to the ESX/ESXi host, additionally:

1. Consider creating a dedicated vSwitch and portgroup for monitoring.
2. Optionally configure the portrgroup to limit traffic to a specific VLAN to be monitored. For more information, see Sample configuration of virtual switch VLAN tagging (VST Mode) and Virtual Switch VLAN Tagging (VST) mode on a vNetwork Distributed Switch .
3. Configure an upstream physical switch port to span or mirror desired traffic, such as a specific VLAN, or traffic matching given MAC address range, to a physical uplink NIC on the ESX/ESXi host.
4. Attach the upstream NIC to a vSwitch and portgroup to be used for monitoring.

Additional Information

• When a virtual machine attempts to utilize promiscuous mode in violation of the defined vSwitch and Portgroup security policy, the attempt is denied and logged by the ESX/ESXi host. For more information, see Identifying virtual machines attempting to use promiscuous network mode on ESX/ESXi.
• How promiscuous mode works at the virtual switch and portgroup levels
• Sample configuration of virtual switch VLAN tagging (VST Mode)
• Configuring promiscuous mode on a virtual switch or portgroup