Missing Cryptographic Parameters in vDS Port Mirroring Deep Packet Inspection

Products

• VMware vCenter Server

Issue/Introduction

When configuring vSphere Distributed Switch (vDS) Encapsulated Remote Mirroring (L3) with GRE encapsulation from a VMware Aria Operations for Networks (vRNI) collector to a destination Analyzer VM, the captured packets lack Layer 7 cryptographic payload parameters. Deep Packet Inspection (DPI) targets, including ciphersuites, TLS protocol versions, digital signature algorithms, and key exchange algorithms, are not present. The packet capture contains only IPFIX flow records and vRNI management traffic.

Environment

VMware vSphere ESXi 8.x

Cause

The vRNI Collector VM receives IPFIX (NetFlow) metadata streams from ESXi hosts, not the raw packet payloads of the monitored workloads. IPFIX exported by the vDS does not perform Deep Packet Inspection (DPI) and inherently cannot extract or export Layer 7 cryptographic payload parameters.

Resolution

To successfully capture TLS negotiation parameters utilizing the existing Encapsulated Remote Mirroring (L3) GRE architecture, target the workload interfaces directly to capture the initial plaintext TLS handshake packets (ClientHello and ServerHello):

1. Edit the vDS Port Mirroring session and remove the vRNI Collector VM port from the Source field.

2. Add the specific Distributed Ports or Distributed Port Groups associated with the actual workload VMs (the servers and clients generating the TLS traffic) as the Source.

3. Set the traffic direction to Ingress/Egress.

4. Ensure the Mirrored packet length parameter is set to Disabled so the vDS forwards the entire packet payload rather than truncating it to the headers.

5. Configure the Analyzer VM packet capture tool (e.g., Wireshark, Zeek, Suricata) to decapsulate the GRE tunnel traffic to expose the original workload packets.

Additional Information

When configuring vDS Port Mirroring, configuring the TCP/IP Stack option to Mirror is required to physically separate encapsulated mirror traffic from the ESXi default management stack.

Architectural Warning: Mirroring an entire datacenter’s traffic via vDS L3 Encapsulated Mirroring is unsuited for broad system-level monitoring. Processing packet duplication and GRE encapsulation inside the ESXi software CPU scheduler introduces severe host CPU degradation, network uplink saturation, and Analyzer VM receive buffer exhaustion. For holistic, system-level Deep Packet Inspection across multiple vCenter environments, utilize VMware NSX Network Introspection (Distributed IDS/IPS) or physical switch-level SPAN/TAPs natively intercepting packets at the vNIC hypervisor level.