HCX-IX Migration failing - Error syncing firmware configuration

Products

• VMware HCX

Issue/Introduction

• HCX-IX appliances are unable to perform any migration.
  
  
• The following error is observed from the HCX-IX appliance /var/log/vmware/vpxd/vpxa.log:
  Agent unable to save configuration to disk: Error syncing firmware configuration: Fault cause: vmodl.fault.NotSupported '
  
  
• The following error is observed from the HCX-IX appliance hbr log where NFC connection are getting destroyed and hearbeat failed with detected dead connection.
  /var/log/vmware/hbrsrv.log
  

  <timestamp> info hbrsrv[13138] [Originator@6876 sub=StorageManager groupID=VRID-####.### opID=hsl-1fd3913c] Running destructor for NFC connection to host-####.
  <timestamp> info hbrsrv[13138] [Originator@6876 sub=StorageManager groupID=VRID-####.### opID=hsl-1fd3913c] Destroying NFC connection to host-####.
  [...]
  <timestamp> info hbrsrv[13138] [Originator@6876 sub=Host groupID=VRID-####.### opID=hsl-1fd3913c] NFC creating session to target: #.#.#.# port: 902
  [...]
  <timestamp> info hbrsrv[01460] [Originator@6876 sub=StorageManager opID=####.###] Enable host host-#### (name=<unset>, addr=<IP>)
  <timestamp> info hbrsrv[01460] [Originator@6876 sub=HostCreds opID=####.###] Address for host-####: "<IP>"
  <timestamp> info hbrsrv[01460] [Originator@6876 sub=Host opID=####.###] Heartbeat handler detected dead connection for agent: host-####/hostd
  <timestamp> info hbrsrv[01460] [Originator@6876 sub=Main opID=####.###] HbrError stack:
  <timestamp> info hbrsrv[01460] [Originator@6876 sub=Main opID=####.###] [0] Exception Vmacore::InvalidStateException: No connection (host-####/hostd)
  <timestamp> info hbrsrv[01460] [Originator@6876 sub=Main opID=####.###] [1] Heartbeat failed (host-####/hostd)

  
  
  
  

Environment

HCX

Cause

This can happen if the IX appliance is unable to communicate to ESXi host on port TCP/UDP 902.

Resolution

Please verify if the IX appliance have access to vCenter/ESXi on port TCP/UDP 902. This is required by the VPXA service to perform Network File Copy(NFC) , heartbeat services and other.
https://ports.broadcom.com/home/VMware-HCX