Remediation of vLCM cluster fails with error: "NSX is not present in cluster desired state"

Products

• VMware NSX
• VMware vCenter Server

Issue/Introduction

• Following errors received while trying to remediate a vLCM Cluster/Host:  "NSX is not present in cluster desired state" and "Health Check failed":

• Compliance check shows that Image Version doesn't contain the NSX LCP bundle:

• Following log snippets seen in /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log after failed remediation:

info vmware-vum-server[12248] [Originator@6876 sub=EHP] NSX-T API /api/v1/vlcm/esx/health/cluster/perspectives/ready-for-apply/status?action=check called successfully.
info vmware-vum-server[12248] [Originator@6876 sub=EHP] [domain-##] A provider [nsxt_rest] has finished (0 remaining).
info vmware-vum-server[12248] [Originator@6876 sub=EHP] [domain-##] [nsxt_rest] [com.vmware.nsxt.Cluster_tnp_or_tn_exists_check] reported issue: NSX is not present in cluster desired state.

• Following log snippets seen in /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log after compliance check:

 "default_message": "Components NSX LCP Bundle are removed in the image. They will be removed from the host during remediation.",

Environment

VMware NSX
VMware vCenter Server

 

Cause

This happens when NSX is not set as a Solution in the Cluster desired state which can be verified by running the following command on vCenter server (requires root access):

dcli com vmware esx settings clusters software get --cluster <Cluster ID>

Sample output missing component: nsx-lcp-bundle in solutions:

Note: if prompted to login, use the SSO Administrator credentials, eg: [email protected]

Resolution

• Get a list of all "Transport Node Collection(TNC)" objects:

GET https://{{MPIP}}/api/v1/transport-node-collections/

• Refer the TNC object where cluster ID matches with the problematic cluster. e.g.

{
 "results": [
 {
 "compute_collection_id": "515d02f7-####-####-####-############:domain-c##",           -------> Refer "compute_collection_id"
 "transport_node_profile_id": "da61fd4d-####-####-####-############",
 "resource_type": "TransportNodeCollection",
 "id": "5c79f613-####-####-####-############",                                         -------> This is the ID to be used in next API call
 "display_name": "5c79f613-####-####-####-############",
 "tags": [],
 "_system_owned": false,
 "_create_time": 1723777879972,
 "_create_user": "admin",
 "_last_modified_time": 1723777879972,
 "_last_modified_user": "admin",
 "_protection": "NOT_PROTECTED",
 "_revision": 0
 }
 ],
 "result_count": 1
}

• Re-trigger workflow from NSX side:

POST https://{{MPIP}}/api/v1/transport-node-collections/<TNC ID>?action=retry_profile_realization

Additional Information

Related KB: Error: "A general system error occurred: Solution being applied does not exist in the desired image." seen on vCenter Server