Embedded vCLS VMs do not Support vMotion or Reconfigure
Article ID: 370222
Updated On:
Products
Issue/Introduction
Embedded vCLS (introduced in 8.0 U3) may disrupt scripts or 3rd party solutions that expect to use vMotion or Reconfigure on vCLS VM's. A few example use cases that may be impacted are:
-
Mass Reconfigure: Some scripts or 3rd-party solutions may seek to enforce or otherwise apply a particular configuration on individual or groups of VMs without regard to whether or not they are vCLS. With the right permissions configured, this would have worked with External vCLS. However, Embedded vCLS blocks all reconfigure operations, so such a script would get an error upon attempting to perform this reconfiguration.
-
Console: Some administrators use the VM console to inspect the guest environment. Embedded vCLS VMs do not provide console access, and do not have a shell running. Attempting to open the console for such a VM will not succeed.
-
Mass vMotion: Some scripts or 3rd-party solutions may seek to migrate individual or groups of VMs without regard to whether or not they are vCLS. With the right permissions configured, this would have worked with External vCLS. However, Embedded vCLS is incapable of being migrated, so such a script would get an error upon attempting to perform this migration.
-
vCLS Placement: Some scripts or administrators may wish to place vCLS VMs in particular ways so that they run on certain hosts and not others. In External vCLS, this was possible with both vMotion and Anti-Affinity. vMotion is no longer possible with Embedded vCLS, so such scripts or administrators will be unable to perform the desired operations.
Environment
vCenter Server 8.0 U3
ESXi 8.0 U3
Cause
The container runtime used for Embedded vCLS confers several management benefits, including a desired-state configuration approach and not having any need to deploy OVF images to inventory datastores due to being packaged with ESX. However, this comes at the expense of some additional restrictions on actions that an administrator can perform. Two notable operations affected are Reconfigure, where a VM's settings are changed in some way, and vMotion, where a VM is migrated to another host. Attempting such operations will be grayed out in the UI and will throw exceptions from APIs.
Resolution
Follow the section below related to the specific use case impacted for workarounds.
Mass Reconfigure, vMotion, or Console
Embedded vCLS VMs are system-managed. Their configurations and host assignments should generally be left alone by scripts. This includes excluding them from any configuration assertions or mass-updates, and from any manual evacuation or balancing acts intended to move workload VMs around. They don't have Console support or any shell that would warrant needing console access, so administrators should not attempt to interact with their guest OS. These VMs do not need to be evacuated in advance of Maintenance Mode or other host unavailability operations, as they are automatically destroyed when such operations are pending. If they need to be moved around for other reasons, refer to the next section.
vCLS Placement
If an administrator needs to move vCLS VMs onto some hosts and away from others, vMotion should no be leveraged. Use one of the below methods:
-
- Maintenance Mode and Disconnect: If an Embedded vCLS VM needs to be moved off of a host temporarily to make way for another operation, the host can be put in Maintenance Mode or Disconnected in order to destroy its Embedded vCLS VM and place it elsewhere. Depending on the number of other available hosts in the cluster the vCLS VM may remain placed elsewhere, or re-deploy on the target host afterwards.
- Retreat Mode: To destroy all vCLS VMs across a whole cluster, Retreat Mode can be used. It behaves the same for External and Embedded types of vCLS. It is useful for completely disabling vCLS during sensitive operations at the expense of making DRS unavailable. For more information, refer to How to Disable vCLS on a Cluster via Retreat Mode.
- Anti-Affinity: For finer-grained control of VM placement, Anti-Affinity can be used to prevent vCLS VMs from being placed on particular hosts or alongside certain other VMs. There is no positive affinity mechanism to place VMs specifically on selected hosts, but a similar affect can be achieved by applying Anti-Affinity to all hosts except the ones desired for running vCLS. However, if all available hosts in a cluster have Anti-Affinity, it will be overridden to ensure that at least one Embedded vCLS VM is available. For more information about vCLS Anti-Affinity, refer to vCLS VM Anti-Affinity Policies.
- Maintenance Mode and Disconnect: If an Embedded vCLS VM needs to be moved off of a host temporarily to make way for another operation, the host can be put in Maintenance Mode or Disconnected in order to destroy its Embedded vCLS VM and place it elsewhere. Depending on the number of other available hosts in the cluster the vCLS VM may remain placed elsewhere, or re-deploy on the target host afterwards.
Additional Information
vSphere Cluster Services (vCLS) VM datastore location is chosen by a default datastore selection logic. To override the default vCLS VM datastore placement for a cluster, you can specify a set of allowed datastores by browsing to the cluster and clicking ADD under Configure > vSphere Cluster Service > Datastores.
Doing so will deploy a new vCLS me to the specified datastore that the cluster has been configured to use.
Feedback
