vpxd service crashes when interacting with VM templates.

Products

• VMware vCenter Server

Issue/Introduction

Symptoms: 

• vpxd service crashes when performing any of the following tasks when interacting with VM templates from UI:
  • New VM from This Template.
  • Convert to Virtual Machine.
  • Clone to Template/Library

• Log file /var/log/vmware/vmon/vmon.log on VC will show similar to below snippets :

yyyy-mm-ddThh:mm:ss.864Z Wa(03) host-#### <vpxd> Service exited. Exit code 1
yyyy-mm-ddThh:mm:ss.864Z Wa(03) host-#### <vpxd> Service exited unexpectedly. Crash count 5. Taking configured recovery action.
yyyy-mm-ddThh:mm:ss.333Z Wa(03) host-#### Failed to publish health status change.

• Log file /var/log/vmware/vmon/vpxd.log on VC will show similar to below snippets :

yyyy-mm-ddThh:mm:ss.072+08:00 verbose vpxd[######] [Originator@6876 sub=Default opID=q-1954:h5ui-getProperties:urn:vmomi:VirtualMachine:vm-######:B55893C8-3F36-4EB9-BBAF-######:######:01-a0] [VpxVmomi] Invoking; <<5299959b-b5ab-ef57-eb42-######, <TCP '127.0.0.1 : 8085'>, <TCP '127.0.0.1 : 56732'>>, ResourceModel, vim.dp.ResourceModel.queryBatch>
yyyy-mm-ddThh:mm:ss.072+08:00 info vpxd[######] [Originator@6876 sub=vpxLro opID=q-1954:h5ui-getProperties:urn:vmomi:VirtualMachine:vm-######:B55893C8-3F36-4EB9-BBAF-######:######:01-a0] [VpxLRO] -- BEGIN lro-3619 -- ResourceModel -- vim.dp.ResourceModel.queryBatch -- 5299959b-b5ab-ef57-eb42-######(52d6a6a5-eadc-013d-XXXX-XXXXXXXX)

• Log file /var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log will show similar to below snippets:

  [yyyy-mm-ddThh:mm:ss.859+08:00] [ERROR] http-nio-5090-exec-9         70001531 100089 200009 com.vmware.vise.mvc.exception.GlobalExceptionHandler              Exception handled while processing request for /ui/data/properties/urn:vmomi:VirtualMachine:vm-######:B55893C8-3F36-4EB9-BBAF-######?properties=name,dc,serverGuid,config.guestId,config.vAppConfig.property,firstFolderWithCreateFromExistingPrivilege,firstComputeWithCloneVmPrivilege:  com.vmware.vim.vmomi.client.common.UnexpectedStatusCodeException: Unexpected status code: 503
          at com.vmware.vim.vmomi.client.common.Response$Status.getStatus(Response.java:56)
          at com.vmware.vim.vmomi.client.http.impl.HttpExchangeBase.parseResponse(HttpExchangeBase.java:271)

 

Environment

VMware vCenter Server 7.0.x
VMware vCenter Server 8.0.x

Cause

This issue typically occurs when a host is removed from the vCenter Server database, leaving behind a zombie ComputeResource that is no longer associated with any host. This orphaned ComputeResource can lead to the crash of the vpxd service when interacting with templates.

Resolution

To fix the issue, the orphaned entity needs to be cleaned up from the vCenter database.

Please open a support ticket with Broadcom technical support team.