NSX prepared ESXI Host present in VXRail Cluster is stuck in Activating state in the SDDC Manager
Article ID: 413527
Updated On:
Products
Issue/Introduction
- Host is a Part of VXRail Cluster
- Other Hosts in the cluster got Activated and issue is with only one host
- Configuration Status of the problematic host is stuck in Activating forever as seen in the below screenshot
/var/log/vmware/vcf/domainmanager/domainmanager.log
------------------------------------------------------2025-10-07T08:43:49.060+0000 INFO [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.p.a.TransportNodeCollectionResolver,dm-####-9] Waiting for TNC status to change from in-progress state2025-10-07T08:43:49.060+0000 DEBUG [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.h.NsxtCommonOperations,dm-####-9] Waiting 600000 ms for TNC status change from IN-PROGRESS2025-10-07T08:43:49.077+0000 DEBUG [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.p.a.TransportNodeCollectionResolver,dm-####-9] Current state of cluster preparation FAILED_TO_REALIZE2025-10-07T08:43:49.077+0000 DEBUG [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.h.NsxtCommonOperations,dm-####-9] Finished waiting for TNC status change from IN-PROGRESS, currentState is FAILED_TO_REALIZE2025-10-07T08:43:49.097+0000 DEBUG [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.p.a.TransportNodeCollectionResolver,dm-####-9] Current state of cluster preparation FAILED_TO_REALIZE2025-10-07T08:43:49.097+0000 INFO [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.p.a.TransportNodeCollectionResolver,dm-####-9] Waiting for TNC status to switch to success or failure2025-10-07T08:43:49.097+0000 DEBUG [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.h.NsxtCommonOperations,dm-####-9] Waiting 600000 ms for TNC status to change to SUCCESS or FAILURE2025-10-07T08:43:49.120+0000 DEBUG [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.p.a.TransportNodeCollectionResolver,dm-####-9] Current state of cluster preparation FAILED_TO_REALIZE2025-10-07T08:43:49.120+0000 DEBUG [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.h.NsxtCommonOperations,dm-####-9] Finished waiting for TNC status to change to SUCCESS or FAILURE, currentState is FAILED_TO_REALIZE2025-10-07T08:43:49.120+0000 INFO [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.p.a.TransportNodeCollectionResolver,dm-####-9] Tracking cluster level failures and trigger resolve if cluster TNC status is failed2025-10-07T08:43:49.138+0000 DEBUG [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.p.a.TransportNodeCollectionResolver,dm-####-9] Current state of cluster preparation FAILED_TO_REALIZE2025-10-07T08:43:49.138+0000 INFO [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.p.a.TransportNodeCollectionResolver,dm-####-9] Invoking Retry Profile Realization NSX API for TNC id be56####-####-####-####-########da032025-10-07T08:43:49.149+0000 INFO [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.p.a.TransportNodeCollectionResolver,dm-####-9] Waiting for TNC status to change to in-progress or success2025-10-07T08:43:49.149+0000 DEBUG [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.h.NsxtCommonOperations,dm-####-9] Waiting 600000 ms for TNC status to change To IN-PROGRESS or SUCCESS2025-10-07T08:43:49.167+0000 DEBUG [vcf_dm,68e4########################0f2e,9a66] [c.v.v.c.f.p.n.p.a.TransportNodeCollectionResolver,dm-####-9] Current state of cluster preparation FAILED_TO_REALIZE2025-10-07T08:44:24.348+0000 DEBUG [vcf_dm,e223########4529,4ce8] [c.v.e.s.c.s.a.i.InventoryServiceAdapterImpl,http-nio-127.0.0.1-7200-exec-2] Fetching Cluster with ID cf50####-####-####-####-########383f from inventory2025-10-07T08:44:24.426+0000 DEBUG [vcf_dm,e223########4529,4ce8] [c.v.e.s.c.s.a.i.InventoryServiceAdapterImpl,http-nio-127.0.0.1-7200-exec-2] Fetching Cluster with ID cf50####-####-####-####-########383f from inventory2025-10-07T08:44:24.427+0000 DEBUG [vcf_dm,e223########4529,4ce8] [c.v.e.s.c.s.a.i.InventoryServiceAdapterImpl,http-nio-127.0.0.1-7200-exec-2] Fetching vCenter data for cluster wit ID cf50####-####-####-####-########383f from inventory2025-10-07T08:44:24.427+0000 DEBUG [vcf_dm,e223########4529,4ce8] [c.v.e.s.c.s.a.i.InventoryServiceAdapterImpl,http-nio-127.0.0.1-7200-exec-2] Fetching Cluster with ID cf50####-####-####-####-########383f from inventory2025-10-07T08:44:24.428+0000 DEBUG [vcf_dm,e223########4529,4ce8] [c.v.e.s.c.s.a.i.InventoryServiceAdapterImpl,http-nio-127.0.0.1-7200-exec-2] Fetching vCenter with ID 6053####-####-####-####-########a66c from inventory2025-10-07T08:44:24.429+0000 DEBUG [vcf_dm,e223########4529,4ce8] [c.v.e.s.c.s.s.ServiceCredentialsHelper,http-nio-127.0.0.1-7200-exec-2] Getting credentials for target type VCENTER, entity ID 6053####-####-####-####-########a66c and service type SDDC_MANAGER...2025-10-07T08:44:42.677+0000 ERROR [vcf_dm,68e4########################e4dd,fc0c] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-17] [O6F52S] UNABLE_TO_CREATE_TRANSPORT_NODE_COLLECTION Unable to create transport node collection with profile 55af####-####-####-####-########a16c on compute collection 1f43####-####-####-####-########d1a3:domain-c10 through nsxmanager nsxmgr.#####.comcom.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Unable to create transport node collection with profile 55af####-####-####-####-########a16c on compute collection 1f43####-####-####-####-########d1a3:domain-c10 through nsxmanager nsxmgr.#####.com at com.vmware.vcf.common.fsm.plugins.nsxt.policy.action.CreateTransportNodeCollectionAction.execute(CreateTransportNodeCollectionAction.java:190) at com.vmware.vcf.common.fsm.plugins.nsxt.policy.action.CreateTransportNodeCollectionAction.execute(CreateTransportNodeCollectionAction.java:37) at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62)
NSX Manager syslog:
----------------------------2025-10-07T05:43:32.558Z nsxmgr.#####.com NSX 99908 SYSTEM [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] message repeated 2 times: [getClientHeartbeatStatus: client 9519####-####-####-####-########31e6, no record for heartbeat found.]2025-10-07T05:43:32.558Z nsxmgr.#####.com NSX 99908 FABRIC [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] Number of hosts that went into error during TNCollection CRUD operation for compute collection 1f43####-####-####-####-########d1a3:domain-c10: 1.2025-10-07T05:43:32.558Z nsxmgr.#####.com NSX 99908 SYSTEM [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] getClientHeartbeatStatus: client 9519####-####-####-####-########31e6, no record for heartbeat found.2025-10-07T05:43:32.558Z nsxmgr.#####.com NSX 99908 SYSTEM [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] getClientHeartbeatStatus: client 9519####-####-####-####-########31e6, no record for heartbeat found.2025-10-07T05:43:32.558Z nsxmgr.#####.com NSX 99908 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Fabric node status INSTALL_IN_PROGRESS2025-10-07T05:43:32.558Z nsxmgr.#####.com NSX 99908 FABRIC [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Invoking issue finder com.vmware.nsx.management.service_fabric.issue.InProgress@585efa28 on 9519####-####-####-####-########31e62025-10-07T05:43:32.558Z nsxmgr.#####.com NSX 99908 POLICY [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Fabric node status INSTALL_IN_PROGRESS2025-10-07T05:43:32.558Z nsxmgr.#####.com NSX 99908 SYSTEM [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] getClientHeartbeatStatus: client 9519####-####-####-####-########31e6, no record for heartbeat found.
Environment
VMware NSX-T Data Center
VMware NSX
Resolution
This is an environment specific issue. So reboot the ESXi host and trying activating the Host again to fix the issue.
If this KB article do not resolve the issue, raise a support ticket with Broadcom support selecting NSX as the product.
Handling Log Bundles for offline review with Broadcom support.
Feedback
