vCenter Server service (vpxd) fails to start with the error, Failed to update memStopInMB in vpxd.cfg :xmlReadFd() failed

Products

• VMware vCenter Server

Issue/Introduction

vCenter Server services fails to start with the error, 

Service-control failed, Failed to start services in profile ALL. RC=1, stderr=Failed to start sps, vstats, vpxd, vsan-health, wcp. Services.Error: Operation timed out.

VMon logs report the below error.

File: /var/log/vmware/vmon/vmon.log

YYYY-MM-DDTHH:MM:SS.741Z Wa(03) host-#### <vpxd> Service pre-start command's stderr: Failed to update memStopInMB in vpxd.cfg :xmlReadFd() failed
YYYY-MM-DDTHH:MM:SS.905Z Er(02) host-#### <vpxd> Service pre-start command failed with exit code 1.

Manually stopping and starting services on the vCenter Server appliance is stuck when starting the vmware-sps service.
The vmware-sps logs report the below error indicating that the vmware-sps service is unable to communicate with the vpxd service.

File: /var/log/vmware/vmware-sps/sps.log

YYYY-MM-DDTHH:MM:SS.205-07:00 [main] ERROR opId=sps-Main-####-953 com.vmware.vim.storage.common.task.retry.CallableRetryDecorator - Caught exception - com.vmware.vim.storage.common.serviceclient.vpxd.VpxdException: Error occurred while retrieving service content

Environment

VMware vCenter Server

Cause

The below error occurs when the vpxd service is unable to read entries in the config file: /etc/vmware-vpx/vpxd.cfg .

Failed to update memStopInMB in vpxd.cfg :xmlReadFd() failed

This error can occur due to an unexpected parameter or corruption in the vpxd.cfg file.

To investigate the integrity of the vpxd.cfg file, we can manually read the memStopInMB parameter using the below command:

$> /usr/lib/vmware-vpx/py/xmlcfg.py -f /etc/vmware-vpx/vpxd.cfg get /config/vpxd/memChecker/memStopInMB

 

Resolution

Validate the integrity of the vCenter server config file and all the parameters within, /etc/vmware-vpx/vpxd.cfg using the command,

/usr/lib/vmware-vpx/py/xmlcfg.py -f /etc/vmware-vpx/vpxd.cfg get /config/vpxd/memChecker/memStopInMB

In one instance, the vpxd.cfg file had extra parenthesis like below:

  <vcls>
    <clusters>
      <domain-c(####)>
        <enabled>True</enabled>
      </domain-c(####)>

 vCenter Server services started successfully after removing the parenthesis as below:

  <vcls>
    <clusters>
      <domain-c####>
        <enabled>True</enabled>
      </domain-c####>

Also validate that the vpxd.cfg file does not have duplicate entries.