vCenter Server UI displays "no healthy upstream" due to vpxd service failing to start

Products

VMware vCenter Server

Issue/Introduction

Accessing VCSA server UI returns "no healthy upstream" error.
Attempting to start all the services manually using the following command fails to start vpxd service: service-control --start --all
vCenter services status:

root@vcenter [/]# service-control --status
Running:
applmgmt lookupsvc lwsmd observability observability-vapi pschealth vc-ws1a-broker vlcm vmafdd vmcad vmdird vmware-analytics vmware-certificateauthority vmware-certificatemanagement vmware-cis-license vmware-eam vmware-envoy vmware-envoy-hgw vmware-envoy-sidecar vmware-hvc vmware-infraprofile vmware-perfcharts vmware-pod vmware-postgres-archiver vmware-rhttpproxy vmware-sca vmware-stsd vmware-topologysvc vmware-trustmanagement vmware-updatemgr vmware-vapi-endpoint vmware-vdtc vmware-vmon vmware-vpostgres vmware-vpxd-svcs vmware-vsm vsphere-ui vtsdb
StartPending:
wcp
Stopped:
vmcam vmonapi vmware-content-library vmware-imagebuilder vmware-netdumper vmware-rbd-watchdog vmware-sps vmware-vcha vmware-vpxd vmware-vsan-health vstats

Below error is observed in /var/log/vmware/vmon/vmon.log

YYYY-DD-HHTHH:MM:SSZ In(05) host-2687 Received start request for vpxd
YYYY-DD-HHTHH:MM:SSZ In(05) host-2687 <vpxd-prestart> Constructed command: /usr/bin/python /usr/lib/vmware-vpx/py/vpxd-prestart.py
YYYY-DD-HHTHH:MM:SSZ In(05) host-2687 <vpxd> Service pre-start command completed successfully.
YYYY-DD-HHTHH:MM:SSZ In(05) host-2687 <vpxd> Constructed command: /usr/sbin/vpxd
YYYY-DD-HHTHH:MM:SSZ In(05) host-2687 <vpxd> Running the API Health command as user vpxd
YYYY-DD-HHTHH:MM:SSZ In(05) host-2687 <vpxd-healthcmd> Constructed command: /usr/bin/python /usr/lib/vmware-vmon/vmonApiHealthCmd.py -n vpxd -f /var/run/vpxd/vmware-vpxd-health-status.xml
YYYY-DD-HHTHH:MM:SSZ In(05) host-2687 Client info Uid=0,Gid=0,Pid=221072,Comm=(vmon-coredumper),PPid=2,Comm=(kthreadd),PPid=0
YYYY-DD-HHTHH:MM:SSZ In(05) host-2687 <vpxd> Service is dumping core. Coredump count 0. CurrReq: 1

/sbin/vpxd returns "Aborted (core dumped)"

Failed reading /etc/vmware-vpx/vpxd.cfg; N7Vmacore19FileFormatExceptionE(Duplicate child: useSSL)
[VpxdMain] Failed to init vmacore
Aborted (core dumped)

Environment

VMware vCenter Server 7.0.x

VMware vCenter Server 8.0.x

Cause

This problem arises from invalid or duplicate entries found in the vpxd configuration file located at /etc/vmware-vpx/vpxd.cfg, specifically related to SSL for NFC data traffic.
Incorrect configuration when enabling or disabling vSphere SSL for the provisioning of NFC data traffic through the vCenter UI may result in a duplicate entry in vpxd.cfg.

Resolution

NOTE: Before applying the steps below, please take a backup or an offline-snapshot (in powered-off state) of the vCenter Server Appliance . If the vCenter is part of a Linked Mode replication setup, please backup/snapshot all replicating nodes as well.
Please refer: VMware vCenter in Enhanced Linked Mode pre-changes snapshot (online or offline) best practice

Log in to the VCSA using ssh
Verify if duplicate entry for "useSSL" or "maxConcurrentSyncItems" is present /etc/vmware-vpx/vpxd.cfg using vi editor
```
<nfc>
    <useSSL>false</useSSL>

    <useSSL>false</useSSL>

</nfc>
```
If a duplicate entry is present, Delete the duplicate entry using VI editor.
Start the services on the vCenter Server: service-control --start --all

Additional Information

Disabling SSL for NFC data traffic in vCenter Server: Disabling SSL for NFC data traffic in vCenter Server