Testing VCDA failivers failing with "Operation canceled due to an unexpected error."

Products

• VMware vCenter Server

Issue/Introduction

Testing VCDA failover failing with "Operation canceled due to an unexpected error."

In /opt/vmware/h4/replicator/log/replicator.log on replicator you see similar entries 

 

XXXX-XX-XX XX:XX:33.147 ERROR - [UI-XXXXXX-XXX-XXXX-XXX-XXXXXX-XXXX-xx] [task-poller-4] com.vmware.h4.jobengine.JobExecution     : Task XXXXXX-XXX-XXXX-XXX-XXXXXX-XXXX (WorkflowInfo{type='failoverTest', resourceT
ype='vmReplication', resourceId='C4-XXXXXX-XXX-XXXX-XXX-XXXXXX-XXXX', isPrivate=false, resourceName='XXXXXX'}) has failed

com.vmware.h4.api.error.exceptions.GenericServerException: Operation aborted due to an unexpected error.

 

Following task in vCenter logs you see similar entries

 

XXXX-XX-XXTXX:XX:XX.XXXZ error vpxd[06569] [Originator@6876 sub=Default opID=XXXXXXX] [VpxLRO] -- ERROR lro-XXXXXX -- XXXXX-XXXX-XXXX-XXXX-XXXXXX(XXXXX-XXXX-XXXX-XXXX-XXXXXXXX) -- nfcService -- vim.NfcService.fileManagement: :vmodl.fault.SystemError
--> Result:
--> (vmodl.fault.SystemError) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = <unset>,
-->    reason = "Datastore XXXXX.XXX.XXXXX.XXXX:/XXXXXXXXX/XXXX(XXXXXXXXXX) has zero length URL on host XXXXXX.XXXXX.XXXXX"
-->    msg = "Received SOAP response fault from [<<io_obj p:0xXXXXXXXX, h:XX, <UNIX ''>, <UNIX '/var/run/envoy-hgw/hgw-pipe'>>, /hgw/host-XXXX/sdk>]: fileManagement
--> A general system error occurred: "Datastore XXXXX.XXX.XXXXX.XXXX:/XXXXXXXXX/XXXX(XXXXXXXXXX) has zero length URL on host XXXXXX.XXXXX.XXXXX"
--> }
--> Args:
-->
--> Arg ds:
--> 'vim.Datastore:XXXXXX-XXXX-XXXX-XXXX-XXXXXX:datastore-XXXXX'
--> Arg hostForAccess:
--> 'vim.HostSystem:XXXXXX-XXXX-XXXX-XXXX-XXXXXX:datastore-XXXXX''
--> Arg files:
--> (string) [
-->    "rtvm-C4-XXXXXXX-XXXX-XXXX-XXXX-XXXXX-XXXX/rtvm-C4-XXXXXXX-XXXX-XXXX-XXXX-XXXXX-XXXX.vmx"

 

RTVM-C4 ID from vCenter logs is matching to failed task in replicator.

 

Similar messages form VPXD logs can be seen in ESXI hostd logs.

Environment

vCenter 8.x

ESXI 8.x

Cause

Temporary communication issue between host and NFS datastore

Resolution

Resync datastore and refresh capacity information for datastore on hosts