Remediation of vLCM cluster fails with error: "NSX is not present in cluster desired state"

Products

• VMware NSX
• VMware vCenter Server

Issue/Introduction

• Following errors received while trying to remediate a vLCM Cluster/Host:  "NSX is not present in cluster desired state" and "Health Check failed":

• Compliance check shows that Image Version doesn't contain the NSX LCP bundle:

• Following log snippets seen in /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log:

info vmware-vum-server[12248] [Originator@6876 sub=EHP] NSX-T API /api/v1/vlcm/esx/health/cluster/perspectives/ready-for-apply/status?action=check called successfully.
info vmware-vum-server[12248] [Originator@6876 sub=EHP] [domain-##] A provider [nsxt_rest] has finished (0 remaining).
info vmware-vum-server[12248] [Originator@6876 sub=EHP] [domain-##] [nsxt_rest] [com.vmware.nsxt.Cluster_tnp_or_tn_exists_check] reported issue: NSX is not present in cluster desired state.

Environment

VMware NSX
VMware vCenter Server

 

Cause

This happens when NSX is not set as a Solution in the Cluster desired state which can be verified by running the following command on vCenter server (requires root access):

dcli com vmware esx settings clusters software get --cluster <Cluster ID>

Sample output missing component: nsx-lcp-bundle in solutions:

Note: if prompted to login, use the SSO Administrator credentials, eg: [email protected]

Resolution

• Get list of all "Transport Node Collection(TNC)" object:

GET https://{{MPIP}}/api/v1/transport-node-collections/

• Refer the TNC object where cluster ID matches with the problematic cluster. e.g.

{
 "results": [
 {
 "compute_collection_id": "515d02f7-####-####-####-############:domain-c##",           -------> Refer "compute_collection_id"
 "transport_node_profile_id": "da61fd4d-####-####-####-############",
 "resource_type": "TransportNodeCollection",
 "id": "5c79f613-####-####-####-############",                                         -------> This is the ID to be used in next API call
 "display_name": "5c79f613-####-####-####-############",
 "tags": [],
 "_system_owned": false,
 "_create_time": 1723777879972,
 "_create_user": "admin",
 "_last_modified_time": 1723777879972,
 "_last_modified_user": "admin",
 "_protection": "NOT_PROTECTED",
 "_revision": 0
 }
 ],
 "result_count": 1
}

• Re-trigger workflow from NSX side:

POST https://{{MPIP}}/api/v1/transport-node-collections/<TNC ID>?action=retry_profile_realization

Additional Information

Related KB:

Error: "A general system error occurred: Solution being applied does not exist in the desired image." seen on vCenter Server