IDPS Oversubscription Metrics Report Artificially Low Values on SCRX Hosts

Products

• VMware vDefend Firewall with Advanced Threat Prevention
• VMware vDefend Firewall
• SSP

Issue/Introduction

On SCRX host transport nodes, the IDPS oversubscription metrics may report incorrect or zero values despite packet drops occurring due to rate limiting. When network traffic hits rate-limiting thresholds, dropped packets resulting from Bytes Per Second (BPS) limits fail to increment the Packets Per Second (PPS) oversubscription metrics. Additionally, the BPS oversubscription counter incorrectly increments by only 1 byte per dropped packet rather than the actual packet payload size.

As a result, NSX and SSP monitoring dashboards report artificially low bandwidth drops, which can mask network performance issues and cause unexpected application degradation while metrics appear healthy.

Environment

VMware SSP

VMware vDefend Firewall

Cause

In the SCRX rate limiter (vsip_scx_limit.c), two underlying accounting issues exist in the dataplane perfdata reporting path:

1. Dropped packets triggered by BPS limit enforcement do not execute the call to update the PPS oversubscription metric counter.

2. Bandwidth drops on the BPS path pass a fixed increment value of 1 instead of the actual packet length (nbytes) to the performance counter update function (VSIPHostPerfOversubBPSAdd).

Resolution

This issue is resolved in NSX versions 4.2.4, Upcoming 9.x and later releases. To permanently resolve this issue, upgrade the affected host transport nodes to a fixed release version.

Verification / Workaround via CLI:

To verify if packet drops are occurring due to rate-limiting on an ESXi transport node despite low or zero metrics in the dashboard, run the following commands in the host shell:

1. 1. Check actual rate-limiter drops:

      vsipioctl getscxratelimit
      

      Look at the Total: section for non-zero Denied / Dropped counters.

   2. Compare with performance data reporting:

      vsipioctl getperfdata
      

      If oversub_pps_peak or oversub_bps_peak remains 0 while vsipioctl getscxratelimit shows active drops, the host is encountering this issue.

   3. Temporary Mitigation: If rate-limiting drops are causing application impact, temporarily increase or adjust the SCRX rate limit parameters using:

      vsipioctl setscxratelimit -b <bps_limit> -p <pps_limit>