VM's connected to NSX segment were unable to get an IP address from External DHCP server after ESXI host Upgrade

Products

• VMware NSX

Issue/Introduction

• No Recent upgrade performed in NSX environment, Only ESXI hosts was upgraded from vCenter
• VM's connected to the NSX segment were not getting the DHCP IP address from External DHCP server and were getting APIPA IP address
• Same VM's when migrated to other ESXI host's which were not upgraded will receive ip address as expected.

DFW Rules of VM when present in Non working HOST : 

[root@esxihost:/var/log] vsipioctl getrules -f nic-#######-eth0-vmware-sfw.2
ruleset mainrs {
  # generation number: 0
  # realization time : 2025-##-##T07:37:28
  # PRE_FILTER rules
  rule 5 at 1 inout protocol any from malicious to any drop with log tag 'MALICIOUS IP AT SOURCE RULE';
  rule 6 at 2 inout protocol any from any to malicious drop with log tag 'MALICIOUS IP AT DESTINATION RULE';
ERROR: failed to get 5 rule in ruleset mainrs: ioctl failed
}

ruleset mainrs_L2 {
  # generation number: 0
  # realization time : 2025-##-##T07:37:28
  # FILTER rules
  rule 1 at 1 inout ethertype any stateless from any to any accept;
}

DFW Rules of VM when present in working HOST : 
[root@esxihost:~] vsipioctl getrules -f nic-########-eth0-vmware-sfw.2
ruleset mainrs {
  # generation number: 0
  # realization time : 2025-##-##T07:21:18
  # PRE_FILTER rules
  rule 5 at 1 inout protocol any from malicious to any drop with log tag 'MALICIOUS IP AT SOURCE RULE';
  rule 6 at 2 inout protocol any from any to malicious drop with log tag 'MALICIOUS IP AT DESTINATION RULE';
  rule 2059 at 3 in inet protocol any from addrset ########-403c-47dd-b816-############ to addrset ########-9f1c-436c-936a-############ accept with log;
  rule 1016 at 4 in inet protocol udp from addrset ########-31c3-4288-a3e4-############ to any port {10-65##, ##, ##, ##, ##, ###, ###, ###, ###, ###, ###, ###, ####} accept with log;
  rule 1016 at 5 in inet protocol tcp strict from addrset ########-31c3-4288-a3e4-############ to any port {1###-5###, 10##-6####, ##, ##, ##, ##, ###, ###, ###, ###, 1###, 3###, 3###} accept with log;
  rule 1016 at 6 in inet protocol tcp strict from addrset ########-31c3-4288-a3e4-############ to any port {49###-65###, 5###, 9###} accept with log;
  rule 1016 at 7 in inet protocol tcp strict from addrset ########-31c3-4288-a3e4-############ to any port ### accept with log as dcerpc;
  # internal # rule 1016 at 8 in inet protocol tcp strict from addrset ########-31c3-4288-a3e4-############ to any port ### accept with log;
  .
  .
  .
  .
  ruleset mainrs_L2 {
  # generation number: 0
  # realization time : 2025-##-##T07:37:28
  # FILTER rules
  rule 1 at 1 inout ethertype any stateless from any to any accept;
}

 

• On the ESXI host (/var/run/log/nsx-syslog) we can see Port disconnected logs as below.

104509:2025-03-06T20:50:00.653Z Er(179) cfgAgent[2099972]: NSX 2099972 - [nsx@6876 comp="nsx-controller" subcomp="cfgAgent" s2comp="nsx-monitoring" entId="########-####-####-####-########" tid="97DD7700" level="fatal" eventState="On" eventFeatureName="distributed_firewall" eventSev="critical" eventType="dfw_vmotion_failure"] The DFW vMotion for DFW filter nic-######-eth0-vmware-sfw.2 on destination host <hostname> has failed and the port for the entity has been disconnected

 

Environment

VMware NSX-T Data Center

VMware NSX

Cause

DFW rules for the Upgraded ESXI hosts were missing after the Host Upgrade.

Resolution

If we see the Alarms on the NSX Manager Web GUI as per the DFW vMotion Failure Alarm, follow the resolution as mentioned in the KB 
If we still see the issue, perform a rolling reboot of the NSX Managers as per the KB NSX is Impacted by JDK-8330017