vCenter is not accessible due to vmafdd service being down

Products

• VMware vCenter Server

Issue/Introduction

• vCenter Server becomes inaccessible after a storage or NAS outage. The vCenter Server Management Interface (VAMI) health status displays an error regarding the database storage.

  Symptoms

  • vCenter UI is inaccessible with a "connection refused" or "site not accessible" error.
  • VAMI health status shows: File system /storage/db is not writeable.
  • The vmafdd service fails to start.
  • Log /var/log/vmware/vmafdd/vmafdd.log contains: VmAfdRpcServerUseProtSeqEp Unable to bind to rpc server.
  • Manual start command service-control --start vmafdd returns: Stderr: Job for vmafdd.service failed because the control process exited with error code See "systemctl status vmafdd.service" and "journalctl -exe" for details.

Environment

• vCenter Server 7.x
• vCenter Server 8.x

Cause

The vmafdd service cannot start because a stale process remains from a previous instance following an unexpected storage disconnect, preventing the service from binding to the RPC runtime.

Resolution

To resolve this issue, clear the orphaned processes and re-establish the RPC runtime:

1. Verify the underlying storage (NAS/SAN) is online from the storage side and writeable for the vCenter appliance.
2. Reboot the vCenter Server Appliance.
3. If the vCenter is in Enhanced Linked Mode (ELM), reboot all partner vCenter Servers in the linked mode to re-establish the RPC runtime.
4. Verify all services are running using the command: service-control --status --all.