Does enabling In-band Network Telemetry (INT) for NSX Traceflow for VLAN-backed segments incur any overhead?

Products

• VMware NSX

Issue/Introduction

Before NSX 3.2, Traceflow could only inject packets on logical ports of overlay logical switches/segments. For overlay segments, Traceflow uses the Geneve header to carry a special Traceflow packet flag when the tracing packet is transmitted on the wire.

Cause

For VLAN-backed logical switches/segments where we do not have Geneve encapsulation, we use a non-default and non-standard DSCP value (excluding the 21 Per-Hop Behavior values of DSCP, to avoid conflict with QoS, in case it is configured). Additionally, we utilize INT-over-TCP/UDP/ICMP techniques to carry Traceflow information in the format of In-band Network Telemetry. Only IPv4/IPv6 packets are supported for Traceflow on VLAN logical networks.

Resolution

Once you have enabled In-band Network Telemetry (INT) by specifying a DSCP value (excluding the DSCP Per-Hop Behaviours) as per KB - https://knowledge.broadcom.com/external/article/317776/traceflow-within-nsx-for-vlan-backed-net.html, INT data will only be inserted to the packets with the specified DSCP value set i.e. Traceflow packets. Regular packets (Non-Traceflow injected) will not have this DSCP value set and thus will not receive the INT data. Telemetry data is inserted as TCP/UDP/ICMP payload.

 

Additional Information

The capture on the left shows a regular ICMP echo request and the one on the right shows a Traceflow-injected ICMP echo request. Traceflow injected ICMP echo request has a higher payload length as it includes INT data. Also, a DSCP value of 2 (configured for INT in this environment) can be seen on the Traceflow injected packet's IP header whereas the regular packet has the default value of 0.