Packet drops occur due to the bandwidth limit of the Gateway QoS profile

Products

• VMware NSX

Issue/Introduction

• Poor network performance is observed through a Tier-1 Gateway that has a Gateway QoS Profile applied.
  
  
• During the time the issue occurs, many logs similar to the following are recorded in /var/log/syslog on the affected NSX Edge.
  

  <Timestamp> <Hostname> NSX <Pid>> ROUTING [nsx@6876 comp="nsx-edge" subcomp="datapathd" s2comp="lrouter" level="INFO"] Trace the drop packet @735: <Unreadable_Packet_Data>

   

• When logging into the NSX Edge on the communication path via CLI as admin, switching to the VRF of the target Gateway, and checking the stats as shown below, the rx_drop_rate_limit and tx_drop_rate_limit counters are increasing.
  

  edge> get gateways
  Gateway
  UUID                                   VRF    Gateway-ID      Name                              Type                        Ports   Neighbors
  <Gateway_#0_UUID>                      0      0                                                 TUNNEL                      3       12/5000
  <Gateway_#1_UUID>                      1      <Gateway_#1_ID> DR-<TIER0_NAME>                   DISTRIBUTED_ROUTER_TIER0    5       3/50000
  <Gateway_#2_UUID>                      2      <Gateway_#2_ID> DR-<TIER1_NAME>                   DISTRIBUTED_ROUTER_TIER1    24      35/50000
  <Gateway_#3_UUID>                      3      <Gateway_#3_ID> SR-<TIER0_NAME>                   SERVICE_ROUTER_TIER0        5       0/50000
  <Gateway_#5_UUID>                      5      <Gateway_#4_ID> SR-<TIER1_NAME>                   SERVICE_ROUTER_TIER1        5       2/50000
  
  edge> vrf 5
  
  edge(tier1_sr[5])> get interfaces | json
        {
          "access_vlan": "untagged",
          "admin": "up",
  ...
          "ifuid": <ifuid>>,
          "ifuuid": "<ifuuid>",
          "internal_operation": "up",
          "ipns": [
            "<ipv4>",
            "<ipv6>",
            "<ipv6>"
          ],
          "ipv6 ra route info": {},
          "l3_forwarding_mode": "IPV4_ONLY",
          "lrouter": "<Gateway_#5_UUID>",
          "mac": "<mac>",
          "mtu": 1500,
          "multicast_oper_up": true,
          "name": "<name>",
  ...
          "overlay_vni": <vni>,
          "peer": "<peer_UUID>",
  ...
          "ptype": "uplink",
  ...
          "stats": {
  ...
            "rx_drop_rate_limit": 1266543563,
  ...
            "tx_drop_rate_limit": 1052713,
  ...
          },
          "t0_router_link": false,
          "ttl": 1,
          "type": "lif",
          "urpf-mode": "NONE",
          "v6_linklocal_valid": true
        },

Environment

VMware NSX

Cause

Packets are being dropped due to the bandwidth limit configured in the applied Gateway QoS Profile, which is impacting network performance.

Resolution

Review and adjust the settings of the Gateway QoS Profile.