Distributed Firewall Rule continues to block or allow traffic after removing IP from group membership.

Products

• VMware vDefend Firewall

Issue/Introduction

• NSX 4.1.2.*
• Using nested groups, and the groups within the nested group tree have the same IP address.
• After the removal of a static IP in a group, the Distributed Firewall rule continues to allow or deny traffic.
• The DFW rule is not matched as expected after the removal of the IP address.

Environment

NSX 4.1.2.*

Cause

NSX CCP (Central Control Plane) fails to remove IP addresses from host nestdb when the same IP address is configured both as IP without prefix and as IP with prefix (IPv4/32 or IPv6/128), resulting in stale IP entries.

Firewall rules that rely on the IP group membership may not function correctly, as stale IP addresses remain in the group. This can result in incorrect firewall rules being applied, potentially allowing or blocking traffic to/from IP addresses that should no longer be associated with the group. 

When the same IP address is added to a group from multiple sources (Nested Groups) with different prefix notations (e.g., 1.1.1.1 configured as a static IP without a prefix, and 1.1.1.1/32 added as a dynamic IP from IP discovery), removing these IP addresses leads to inconsistent behavior. 
NSX controller leaves stale IP entries that cannot be properly removed because the prefix notation from one source conflicts with the non-prefixed notation from the other source, causing IP removal operations to fail.

 

Resolution

 This is fixed in 4.2.0 and later

Option 1
To resolve the issue restart the affected NSX Controller service. 

1. On the ESXi host experiencing the problem, identify the Controller it connects to.
   
   
   1. From root shell run the command
   nsxcli -c get controller 
   
   For example
   > nsxcli -c get controllers 
    Controller IP    Port     SSL         Status       Is Physical Master   Session State  Controller FQDN 
     X.X.X.X    1235   enabled     connected             true               up               NA
     Y.Y.Y.Y    1235   enabled      not used            false              null              NA       
     Z.Z.Z.Z    1235   enabled      not used            false              null              NA
   
   
   
2. ssh to the Controller identified in step 1 as admin and restart the controller service using the cli command
   
    > restart service controller
   
   
3. Run get cluster status and confirm CONTROLLER is reported as STABLE with all 3 nodes UP

Option 2
Restart the Controller service on all 3 Managers as admin user

     > restart service controller

 

Option 3

Avoid using static IP addresses IPv4/32 and IPv6/128 in the static group definition. Due to dynamic IPs being read as /32 in the NSX controller, this issue is not 100% avoidable. 

 

Additional Information

An IPv4/32 or IPv6/128 address can be added to NSX either statically or dynamically. Dynamic IPs are any IPs that are discovered by VM Name, VMware Tools, etc.