Distributed Firewall DPU kernel memory usage is very high

Products

• VMware NSX

Issue/Introduction

• Title: "Distributed Firewall DPU kernel memory usage is very high"
  
  
• Event ID: distributed_firewall.dfw_memory_usage_very_high_on_dpu
  
  
• Added in release: 4.0.0
  
  
• Alarm Description 
  
  
  • Purpose: This alarm indicates that kernel heap memory usage for the specified heap on DPU is very high.
  • Impact: We can expect failures in new/updated configuration from being applied. Additionally, it could also impact new firewall connections.
    
    vsip-rules - Will affect new rule config. Rules may not get realized
    vsip-fprules - Will affect new rule config. May cause vMotion failures
    vsip-fqdn - May affect L7 rule enforcement
    vsip-attr - May affect L7 rule enforcement

Environment

VMware NSX

Resolution

• Resolution:
  
  
  • Maintenance window required for remediation: NO
    
    
  • Steps to resolve:

    • View the current Distributed Firewall kernel memory usage by invoking the NSX CLI command 'get firewall thresholds' on the host. Check the heap that has high memory usage

      • • vsip-rules - Use Applied to field in the rules so the rules are applied to fewer/specific VMs effectively reducing the rule count. Groups containing IP addresses should use CIDR blocks as much as possible, rather than specifying each individual IP address. Also try to enable global_macset_optimization_mode_enabled flag to reduce heap usages in kernel via Policy API path /policy/api/v1/infra/settings/firewall/security.
        • vsip-fprules - Re-balance the workloads on this host to other hosts. Groups containing IP addresses should use CIDR blocks as much as possible, rather than specifying each individual IP address.
        • vsip-fqdn/vsip-attr - Consider refining L7 rules to targeted traffic using from/to addresses for these rules

      Reduce the number of firewall rules.

      • • Use groups for source/destination addresses instead of individual IPs.
        • When a rule has multiple ports, protocols, or services, those should be defined as a group as opposed to listed individually in the rule.
        • Add Applied To field in rules to define the VM scope. Without the use of this field, every rule will get applied to every entity in the DFW expanse.