Error: VLCM_REMEDIATE_PERSONALITY_FAILED when remediating ESXi hosts during SDDC cluster creation
search cancel

Error: VLCM_REMEDIATE_PERSONALITY_FAILED when remediating ESXi hosts during SDDC cluster creation

book

Article ID: 422030

calendar_today

Updated On:

Products

VMware Cloud Foundation VMware SDDC Manager / VCF Installer VMware vCenter Server

Issue/Introduction

  • In VMware Cloud Foundation (VCF), the task to remediate ESXi hosts to comply with the cluster image fails during cluster creation, expansion, or host addition via SDDC Manager. This failure occurs when the target vSphere cluster's vSphere Lifecycle Manager (vLCM) image definition is inconsistent with the software versions present on the physical ESXi hosts.

Symptoms:

  • The SDDC Manager task history reports that one or more ESXi hosts are in an 'Incompatible' compliance state with the error VLCM_REMEDIATE_PERSONALITY_FAILED.

  • The /var/log/vmware/vcf/lcm/lcm-debug.log file on the SDDC Manager records the incompatible hosts error:|

    ERROR [vcf_dm,6############################d,5186] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-7]  [V####8] VLCM_REMEDIATE_PERSONALITY_FAILED Applying personality to cluster domain-c#### failed with error: com.vmware.vapi.std.errors.Error: Error (com.vmware.vapi.std.errors.error) => {
        messages = [LocalizableMessage (com.vmware.vapi.std.localizable_message) => {
        id = com.vmware.vcIntegrity.lifecycle.RemediateClusterTask.IncompatibleHosts,
        defaultMessage = Cluster 'sample-cluster-vsan' remediation failed because hosts 'esxi01.example.com, esxi02.example.com, esxi03.example.com, esxi04.example.com' are in 'incompatible' compliance state.,
        args = [sample-cluster-vsan, esxi01.example.com, esxi02.example.com, esxi03.example.com, esxi04.example.com],
        params = <null>,
        localized = Cluster 'sample-cluster-vsan' remediation failed because hosts 'esxi01.example.com, esxi02.example.com, esxi03.example.com, esxi04.example.com' are in 'incompatible' compliance state.
    }],
        data = <null>,
        errorType = ERROR
    }
    com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Applying personality to cluster domain-c#### failed with error: com.vmware.vapi.std.errors.Error: Error (com.vmware.vapi.std.errors.error) => {
        messages = [LocalizableMessage (com.vmware.vapi.std.localizable_message) => {
        id = com.vmware.vcIntegrity.lifecycle.RemediateClusterTask.IncompatibleHosts,
        defaultMessage = Cluster 'sample-cluster-vsan' remediation failed because hosts esxi01.example.com, esxi02.example.com, esxi03.example.com, esxi04.example.com' are in 'incompatible' compliance state.,
        args = [sample-cluster-vsan, esxi01.example.com, esxi02.example.com, esxi03.example.com, esxi04.example.com],
        params = <null>,
        localized = Cluster 'sample-cluster-vsan' remediation failed because hosts 'esxi01.example.com, esxi02.example.com, esxi03.example.com, esxi04.example.com' are in 'incompatible' compliance state.
    }],
        data = <null>,
        errorType = ERROR
    }

  • The /var/log/vmware/vlcm/vlcm.log file on the relevant vCenter Server records the corresponding vLCM task failure:

    info vmware-vum-server[09383] [Originator@6876 sub=PM.AsyncTask.RemediateClusterTask{125}] [vciTaskBase 1493] SerializeToVimFault fault:
    --> (vmodl.fault.SystemError) {
    -->    faultCause = (vmodl.MethodFault) null,
    -->    faultMessage = (vmodl.LocalizableMessage) [
    -->       (vmodl.LocalizableMessage) {
    -->          key = "com.vmware.vcIntegrity.lifecycle.RemediateClusterTask.IncompatibleHosts",
    -->          arg = (vmodl.KeyAnyValue) [
    -->             (vmodl.KeyAnyValue) {
    -->                key = "1",
    -->                value = "sample-cluster-vsan"
    -->             },
    -->             (vmodl.KeyAnyValue) {
    -->                key = "2",
    -->                value = "esxi01.example.com, esxi02.example.com, esxi03.example.com, esxi04.example.com"
    -->             }
    -->          ],
    -->          message = <unset>
    -->       }
    -->    ],
    -->    reason = "vLCM Task failed, see Error Stack for details."
    -->    msg = "{
    -->     "data": null,
    -->     "error_type": "ERROR",
    -->     "messages": [
    -->         {
    -->             "args": [
    -->                 "sample-cluster-vsan",
    -->                 "esxi01.example.com, esxi02.example.com, esxi03.example.com, esxi04.example.com"
    -->             ],
    -->             "default_message": "Cluster 'sample-cluster-vsan' remediation failed because hosts 'esxi01.example.com, esxi02.example.com, esxi03.example.com, esxi04.example.com' are in 'incompatible' compliance state.",
    -->             "id": "com.vmware.vcIntegrity.lifecycle.RemediateClusterTask.IncompatibleHosts"

Environment

  • VMware Cloud Foundation 9.x

  • VMware Cloud Foundation 

  • SDDC Manager 9.x

  • SDDC Manager

  • vCenter Server 9.x 

  • vCenter Server

Cause

  • The ESXi hosts are incompatible with the target cluster's vSphere Lifecycle Manager (vLCM) image. The vLCM compliance check reports an error because one or more Add-on components present on the ESXi hosts are missing from the cluster's target image definition. To maintain stability, vLCM prevents component downgrades during remediation.

Resolution

The resolution involves manually updating the image for the cluster in vCenter Server to include the necessary component versions that are present on the ESXi hosts. This is done to achieve a compliant state before VCF remediation can proceed.

Follow these steps:

  1. Identify the Incompatibility:

    1. Navigate to the cluster in the vSphere Client.

    2. Go to Updates > Image.

    3. Click Run Compliance Check (if not already run).

    4. Review the Compliance section and note the specific component(s) that are causing the downgrade error.

  2. Edit the Desired Image:

    1. Click on Edit Image for the cluster.

    2. In the Add-ons section, search for and add the component(s) identified in the compliance check.

    3. Ensure the selected Add-on version is equal to or newer than the version currently installed on the ESXi hosts to prevent the downgrade error.

  3. Validate Compliance:

    1. After saving the modified vLCM image, run another Compliance Check.

    2. The ESXi hosts should now transition to a Compliant state.

  4. Retry the VCF Task:

    1. Once VCSA reports the hosts as compliant with the cluster image, return to the SDDC Manager UI.

    2. Retry the failed VCF task, 'Remediate ESXi Host(s) to be compliant with Cluster's Image'

This process aligns with the procedures documented in KB Article 320562. The successful compliance check in vCenter Server resolves the underlying blocking condition, allowing the VCF SDDC Manager remediation task to complete.

Powered by Wolken Software