Does enabling In-band Network Telemetry (INT) for NSX Traceflow for VLAN-backed segments incur any overhead?

Products

• VMware NSX

Issue/Introduction

Before NSX 3.2, Traceflow could only inject packets on logical ports of overlay logical switches/segments. For overlay segments, Traceflow uses the Geneve header to carry a special Traceflow packet flag when the tracing packet is transmitted on the wire.

Cause

For VLAN-backed logical switches/segments where we do not have Geneve encapsulation, we use a non-default and non-standard DSCP value (excluding the 21 Per-Hop Behavior values of DSCP, to avoid conflict with QoS, in case it is configured). Additionally, we utilize INT-over-TCP/UDP/ICMP techniques to carry Traceflow information in the format of In-band Network Telemetry. Only IPv4/IPv6 packets are supported for Traceflow on VLAN logical networks.

Resolution

Once you have enabled In-band Network Telemetry (INT) by specifying a DSCP value (excluding the DSCP Per-Hop Behaviours) as per KB - https://knowledge.broadcom.com/external/article/317776/traceflow-within-nsx-for-vlan-backed-net.html, INT data will only be inserted to the packets with the specified DSCP value set i.e. Traceflow packets. Regular packets (Non-Traceflow injected) will not have this DSCP value set and thus will not receive the INT data. Telemetry data is inserted as TCP/UDP/ICMP payload.

 

Additional Information

The capture on the left shows a regular ICMP echo request and the one on the right shows a Traceflow-injected ICMP echo request. Traceflow injected ICMP echo request has a higher payload length as it includes INT data. Also, a DSCP value of 2 (configured for INT in this environment) can be seen on the Traceflow injected packet's IP header whereas the regular packet has the default value of 0.

Click here for additional information for Troubleshooting NSX Traceflow.