The VPXD service crashes when we add a VM

Products

• VMware

Issue/Introduction

Symptoms

• After adding a new VM, the vpxd service crashes after a few minutes of running.
• You may see messages in the vCenter vpxd log similar to: 

error vpxd[****] [Originator@6876 sub=MoHost opID=FdmMonitor-domain-c*****-***] Could not read 'config' from hostd for vim.HostSystem:host-******: N5Vmomi5Fault17HostCommunication9ExceptionE(Fault cause: vmodl.fault.HostCommunication)
 --> )

error vpxd[****} [Originator@6876 sub=IO.Http] User agent failed to send request; (null), N7Vmacore17CanceledExceptionE(Operation was canceled)

 

To confirm that this is the issue, ensure you have gone through the article  VPXD Service Fails to Start  and check that all of the dependent services are running while the vpxd service is stopped. These should be running: 

• rhttpproxy
• vmware-vpostgres
• vpxd-svcs
• vmware-vpostgres.
• lookupsvc

The vmware-vpxd service shows as stopped. You can check that with the command: 

service-control --status --all 

Then check if vmware-vpxd is not running but the  rhttpproxy, vmware-vpostgres, lookupsvc, and vpxd-svcs services are started. 

 

Environment

• vCenter 7.0.x
• vCenter 8.0.x

Cause

This issue is caused by a malformed object in the vCenter inventory. If there is a malformed VM, vpxd will crash when trying to read the vCenter Database. 

Resolution

The issue will occur as soon as they problem VM is added to the vCenter. So we need to:

1. Log in to vCenter Server
2. Remove the most recently created VMs a
3. Then test if vpxd will keep running by starting the services:
   • service-control --start--all 

 

If you do not know which VM or host is causing the issue then the workaround is: 

1. Disconnect a host from vCenter
2. Try to turn on vpxd and wait to see if it crashes again 
3. Reconnect the Host if vpxd fails to start
4. Repeat steps 1-3 for each host until vpxd stays on 

This will tell us which host has the malformed VM. Then, reconnect the bad host and then you can go through all the VMs on that host to find the malformed VM. To do this: 

1. Removed a VM from inventory 
2. Try to turn on vpxd and wait to see if it crashes again 
3. Re-add the VM to the cluster if vpxd crashes
4. Repeat steps 1-3 for each host until vpxd stays on. 

Then we can remove the malformed VM

Additional Information

For instructions on removing the VM see:  Remove and Reregister VMs and VM Templates