VM Not Able to Power-On After Replication Disabled in vCloud Director Availability

Products

• VMware vCenter Server
• VMware Cloud Director Availability - Disaster Recovery 4.x
• VMware Cloud Director

Issue/Introduction

The VM is replicated through the Vcloud Director Availability (VCDA) appliance. It is not possible to change a VM's encryption state while a replica is syncing. The replication was deleted before changing the VM's Storage Policy from unencrypted to Encrypted. No replication should currently exist for the VM.

While powering on the VM:

vpxd
2024-10-23T11:03:57.593Z info vpxd[23636] [Originator@6876 sub=Default opID=vcd-########-####-####-####-############;activity=urn:uuid:########-####-####-####-###############-##-##-##-##] [VpxLRO] -- ERROR task-####### -- VM-Name -- Drm.ExecuteVmPowerOnLRO: vim.fault.GenericVmConfigFault:
--> Result:
--> (vim.fault.GenericVmConfigFault) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = (vmodl.LocalizableMessage) [
-->       (vmodl.LocalizableMessage) {
-->          key = "msg.msg.scsi.esx.filter.hbr.invalidEncrypted",
-->          arg = <unset>,
-->          message = "The virtual machine is encrypted."
-->       },
-->       (vmodl.LocalizableMessage) {
-->          key = "msg.scsi.esx.filterAttachmentFailed",
-->          arg = (vmodl.KeyAnyValue) [
-->             (vmodl.KeyAnyValue) {
-->                key = "1",
-->                value = "hbr_filter"
-->             },
-->             (vmodl.KeyAnyValue) {
-->                key = "2",
-->                value = "scsi0:0"
-->             },
-->             (vmodl.KeyAnyValue) {
-->                key = "3",
-->                value = "msg.vmk.status.VMK_NOT_SUPPORTED"
-->             },
-->             (vmodl.KeyAnyValue) {
-->                key = "4",
-->                value = "195887136"
-->             }
-->          ],
-->          message = "Failed to attach filter 'hbr_filter' to 'scsi0:0': Not supported (195887136). "
-->       },
-->       (vmodl.LocalizableMessage) {
-->          key = "msg.scsi.esx.virtualDevFailed",
-->          arg = (vmodl.KeyAnyValue) [
-->             (vmodl.KeyAnyValue) {
-->                key = "1",
-->                value = "scsi0:0"
-->             },
-->             (vmodl.KeyAnyValue) {
-->                key = "2",
-->                value = "/vmfs/volumes/########-########-####-#########/VM-Name/VM_Name.vmdk"
-->             }
-->          ],
-->          message = "Unable to create virtual SCSI device for 'scsi0:0' (/vmfs/volumes/########-########-####-#########/VM-Name/VM-Name.vmdk). "
-->       },
-->       (vmodl.LocalizableMessage) {
-->          key = "msg.moduletable.powerOnFailed",
-->          arg = (vmodl.KeyAnyValue) [
-->             (vmodl.KeyAnyValue) {
-->                key = "1",
-->                value = "DevicePowerOn"
-->             }
-->          ],
-->          message = "Module 'DevicePowerOn' power on failed. "
-->       },
-->       (vmodl.LocalizableMessage) {
-->          key = "msg.vmx.poweron.failed",
-->          arg = <unset>,
-->          message = "Failed to start the virtual machine."
-->       }
-->    ],
-->    reason = "The virtual machine is encrypted."
-->    msg = "The virtual machine is encrypted."
--> }
--> Args:
-->

While trying to reconfigure the VM through VCD:

[ ########-####-####-####-########### ] Underlying system error: Failed to change encryption state for the virtual machine with vSphere Replication enabled. vCenter Server task (moref: task-#######) failed in vCenter Server 'VC-Name' (########-####-####-####-#############). - Failed to change encryption state for the virtual machine with vSphere Replication enabled.

Environment

• vSphere 7.0
• vSphere 8.0
• vCloud Director 10.5.1
• vCloud Director 10.6.1
• vCloud Director Availability 4.7.2

Cause

This issue occurs due to stale vSphere Replication entries in the virtual machine configuration file (.vmx).

Resolution

To work around the issue, remove the vSphere Replication Host-Based Replication (HBR) hbr_filter entries from the .vmx file:

1. Connect to the ESXi host the virtual machine resides on using an SSH session. For more information, see Using ESXi Shell in ESXi.
2. Navigate to the location of the virtual machine's .vmx file:
   /vmfs/volumes/virtual_machine_datastore/virtual_machine_folder/
3. Edit the virtual machine using a text editor:
   vi virtual_machine.vmx
4. Remove all the entries containing hbr_filter. For example:

   hbr_filter.configGen = ""
   scsi0:0.filters = ""
   hbr_filter.rpo = ""
   hbr_filter.destination = ""
   hbr_filter.port = ""
   hbr_filter.gid = ""
   hbr_filter.protocol = ""
   hbr_filter.quiesce = ""
   hbr_filter.opp = ""
   hbr_filter.pause = ""
   scsi0:0.hbr_filter.rdid = ""
   scsi0:0.hbr_filter.persistent = ""

   Note: You may also comment out each line by preceding each line with a #.
5. Save the changes and exit the text editor.
6. To find the vmid, run this command:
   vim-cmd vmsvc/getallvms | grep -i VM_Name
7. To reload the virtual machine into inventory, run this command:
   vim-cmd vmsvc/reload vmid
   For more information, see Reloading a vmx file without removing the virtual machine from inventory (1026043).
8. Power on the virtual machine.