"No valid hosts could be found due to datastore accessibility" error when creating an anti-affinity rule in Cloud DIrector

Products

• VMware Cloud Director

Issue/Introduction

Symptoms:

• "No valid hosts could be found due to datastore accessibility" error is seen when trying to create an anti-affinity rule.
• Within vCenter Server it's observed that a drsShellVm creation task has been initiated but failed to run successfully.
• Within the log /opt/vmware/vcloud-director/logs/vcloud-container-debug.log you observe the drsShellVm being created just prior to the error. "Datastore01" is the datastore name in this example.

2022-10-24 13:15:24,605 | INFO   | vim-proxy-activity-pool-82 | CreateVmActivity        | [Activity Execution] Creating VM drsShellVm-<DRSSHELLVM_UUID> in resource pool [vcId=<VC_UUID>, moref=resgroup-123], folder [vcId=<VC_UUID>, moref=group-v123], datastore "[vcId=<VC_UUID>, moref=datastore-123]" Datastore01, storage profile * - Handle: urn:uuid:<HANDLE_UUID> | requestId=<REQUEST_UUID>,request=POST https://cloud.example.com/api/vdc/<VDC_UUID>/vmAffinityRules/,requestTime=1666617319306,remoteAddress=<IP>:29108,userAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ...,accept=application/*+xml;version 36.3 vcd=<VCD_UUID>,task=<TASK_UUID activity=(com.vmware.vcloud.backendbase.management.system.TaskActivity,urn:uuid:<TASK_UUID>) activity=(com.vmware.ssdc.backend.services.impl.DistributedAffinityRuleCreationActivity,urn:uuid:<ACTIVITY_UUID>) activity=(com.vmware.ssdc.backend.services.impl.CreateDrsShellVMActivity,urn:uuid:<ACTIVITY_UUID>) activity=(com.vmware.vcloud.val.internal.impl.CreateVmActivity,urn:uuid:<ACTIVITY_UUID>)

2022-10-24 13:15:25,208 | ERROR  | vim-proxy-activity-pool-82 | TaskWaiterInvRecord      | Task dump: [valref = [vcId=<VC_UUID>, moref=task-123], state = ERROR, taskName = CreateVM_Task, progress = null, entityName = drsShellVm-<DRSSHELLVM_UUID>:domain-c123, errorMessage = A component of the virtual machine is not accessible on the host.] | requestId=<REQUEST_UUID>,request=POST https://cloud.example.com/api/vdc/<VDC_UUID>/vmAffinityRules/,requestTime=1666617319306,remoteAddress=<IP>:29108,userAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ...,accept=application/*+xml;version 36.3 vcd=<VCD_UUID>,task=<TASK_UUID> activity=(com.vmware.vcloud.backendbase.management.system.TaskActivity,urn:uuid:<TASK_UUID>) activity=(com.vmware.ssdc.backend.services.impl.DistributedAffinityRuleCreationActivity,urn:uuid:<ACTIVITY_UUID>) activity=(com.vmware.ssdc.backend.services.impl.CreateDrsShellVMActivity,urn:uuid:<ACTIVITY_UUID>) activity=(com.vmware.vcloud.val.internal.impl.CreateVmActivity,urn:uuid:<ACTIVITY_UUID>)

Environment

VMware Cloud Director for Service Provider 10.x

Cause

This issue occurs when Cloud Director tries to create the drshellvm on a datastore which is mounted in ReadOnly state.

Resolution

This is a Known Issue resolved in VMware Cloud Director 10.4.1 which is available from Download Broadcom products and software.

Workaround:

To workaround the issue one of the following options could be performed where possible:

1. Put the impacted datastore into maintenance mode before creating the Affinity rule.
2. Enable Read/Write on the datastore before creating the Affinity rule.
3. Create the Anti-Affinity rule directly in vCenter Server UI on the cluster level. For this workaround to be applicable the VMs need to be located within the same ESXi Cluster. 

Additional Information

This issue can be commonly encountered in VCF environments which utilize the LCM-Bundle-Repo datastore.