vLCM Image Compliance Check Reports Host Unknown

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

Symptom:

• The image compliance status for the cluster is reported as Host Unknown on the Updates tab in vSphere Client.
• An ESXi host update is being attempted to enable the distributed IDPS turbo mode on NSX.
• The following error is logged in the vCenter Server log file /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log
  

  "com.vmware.esx.settings.notification": {
    "id": "com.vmware.vcIntegrity.lifecycle.EsxImage.UnknownError",
    "message": {
        "STRUCTURE": {
            "com.vmware.vapi.std.localizable_message": {
                "args": [],
                "default_message": "An unknown error occurred while performing the operation.",
                "id": "com.vmware.vcIntegrity.lifecycle.EsxImage.UnknownError"

• The following message is logged in the ESXi host log file /var/run/log/lifecycle.log:
  
  YYYY-MM-DDTHH:MM:SSZ Er(##) lifecycle[##########]: Scanner:906 Failed to validate/extract the softwareSpec: 'VMware_bootbank_nsx-scx_4.2.3.2.0-8.0.25077146'

Cause

The scrx_vlcm_enabled flag on the NSX Manager was set to true before the host compliance check using the vLCM Image (which installs the scrx VIB for IDPS Turbo Mode) was completed.
The scrx_vlcm_enabled should be set after the compliance check is completed.

Because this flag is set to true, a mismatch occurs when vLCM performs the compliance check, resulting in the "Host is in an unknown state" error.

Resolution

To resolve this issue, clear the current configuration and re-run the compliance check by following the steps below:

Step 1: Take a snapshot for vCenter Server

1. Stop the vCenter Server Appliance VM.
   
   
2. Take a snapshot of the vCenter Server

Step 2: Verify and reset the scrx_vlcm_enabled flag

1. Power On the vCenter Server Appliance VM and ensure you can access the vSphere Client UI.
   
   
2. Log in to the vCenter Server Appliance via SSH as the root user and switch to the BASH shell.
   
   
3. Verify the current scrx_vlcm_enabled flag.
   
   curl -k -u 'admin:<NSX_Manager_Admin_Password>' -H 'Content-Type: application/json' -X GET https://<NSX_Manager_IP>/policy/api/v1/system-config?key=scrx_enabled

4. Change the setting to false:
   
   curl -k -u 'admin:<NSX_Manager_Admin_Password>' -H 'Content-Type: application/json' -X PATCH --data '{"keyValuePairs": [{"key": "scrx_vlcm_enabled","value": "false"}]}' https://<NSX_Manager_IP>/policy/api/v1/system-config

Step 3: Reset the Update Manager database

1. Log in to the vCenter Server Appliance via SSH as the root user and switch to the BASH shell.
   
   
2. Run the following commands in sequence:
   
   service-control --stop vmware-updatemgr
python /usr/lib/vmware-updatemgr/bin/updatemgr-utility.py reset-db
rm -rf /storage/updatemgr/patch-store/*
service-control --start vmware-updatemgr

Step 4: Create a temporary cluster to force-load image Information

1. Log in to vSphere Client.
   
   
2. Create a new cluster (any cluster name is fine).
   
   
3. Check the box for "Manage all hosts in the cluster with a single image." and Select "Import image from an existing host in the vCenter Server inventory" in the pop-up wizard.
   
   
4. Select any host, and complete the cluster creation.
   

   Note: Since the image cannot be imported directly into the original cluster, this step indirectly forces the vCenter to load the required information.
   Once the temporary cluster is created, you may safely delete it.

Step 5: Verify the image management status

1. Log in to the vCenter Server via the vSphere Client.
   
   
2. Click the "vSphere Client" logo at the top of the page to open the shortcut menu, and select "Hosts and Clusters" from the menu.
   
   
3. Select the affected cluster from the inventory tree on the left.
   
   
4. Navigate to the Updates tab on the right, then select Hosts -> Image.
   
   
5. Confirm that the following message (or equivalent in your language) is displayed, indicating that image management is active:
   

   Image
   Hosts in this cluster are managed collectively. The image below will be applied to all hosts in this cluster.
   
   Note: You can safely ignore any UI errors that may temporarily appear during this step.

Step 6: Download the NSX LCP file to vCenter Server

1. Log in to the vCenter Server via SSH as root and switch to the BASH shell.
   
   
2. Retrieve your transport-node-collection-id by running:
   
   curl -k -u 'admin:<NSX_Manager_Admin_Password>' -X GET https://<NSX_Manager_IP>/api/v1/transport-node-collections
   
   
3. Run the following command to download the NSX LCP file to the vCenter Server side:
   
   curl -k -u 'admin:<NSX_Manager_Admin_Password>' -X POST https://<NSX_Manager_IP>/api/v1/transport-node-collections/<transport-node-collection-id>?action=retry_profile_realization

Step 7: Re-run compliance check

• Run the vLCM compliance check again and verify if the status is now reported correctly.