Attempting to upgrade ESXI using VLCM with in a environment that has NSX-T failed: Health Check for 'Vmware cluster' failed

Products

VMware NSX

Issue/Introduction

When trying to upgrade a ESXI on a cluster that is not NSX-T Prepared will be getting a general error: Health Check for 'Vmware cluster' failed

Environment

- Vcenter

- NSX-T

Cause

Vcenter and the NSX-T manager are not able to communicate due that the Service account and the enable trust setting are not enable on the compute manager, which case the ESXI upgrade to fail.

- Log to be review

From the Vcenter side

vmware-vum-server

2024-09-05T19:24:22.025Z info vmware-vum-server[1680029] [Originator@6876 sub=EHP opID=[] Found cached JWT

2024-09-05T19:24:22.025Z info vmware-vum-server[1680029] [Originator@6876 sub=EHP opID=[] Not on VMC environment. Provider pod is disabled.

2024-09-05T19:24:22.025Z info vmware-vum-server[1680029] [Originator@6876 sub=EHP opID=[] Provider "pod" not executable

2024-09-05T19:24:22.025Z info vmware-vum-server[1397653] [Originator@6876 sub=EHP opID=[] Current NSX-T node: .................

2024-09-05T19:24:22.025Z info vmware-vum-server[1680029] [Originator@6876 sub=EHP opID=[] [domain-] A provider [pod] has finished (3 remaining).

2024-09-05T19:24:22.026Z info vmware-vum-server[1680029] [Originator@6876 sub=EHP opID=[] Creating drs-plugin API provider

2024-09-05T19:24:22.026Z warning vmware-vum-server[1680029] [Originator@6876 sub=EHP opID=[] No timeout config for PluginRequestTimeout

2024-09-05T19:24:22.026Z info vmware-vum-server[1680029] [Originator@6876 sub=EHP opID[] Initializing vAPI session

2024-09-05T19:24:22.026Z info vmware-vum-server[1680029] [Originator@6876 sub=EHP opID=[] Getting 'non-delegable' SSO token

2024-09-05T19:24:22.026Z info vmware-vum-server[1680029] [Originator@6876 sub=EHP opID=[] Acquiring SAML token with extension certificate...

2024-09-05T19:24:22.029Z info vmware-vum-server[1397653] [Originator@6876 sub=EHP opID=[] NSX-T version: 4.1.2.5

2024-09-05T19:24:22.029Z info vmware-vum-server[1397653] [Originator@6876 sub=EHP opID=[] Calling NSX-T API /api/v1/vlcm/esx/health/cluster/perspectives/ready-for-apply/st

atus?action=check (/external-tp/http1/10.240.50.11/443/06B8AE64182782B31AF7877C081BEE31E1150FA9/api/v1/vlcm/esx/health/cluster/perspectives/ready-for-apply/status?action=check).

2024-09-05T19:24:22.042Z info vmware-vum-server[1680039] [Originator@6876 sub=EHP opID=[] [domain-] A provider [wcp] has finished (2 remaining).

------------------------------------------------

Task Failed. Error: Error:

--> com.vmware.vapi.std.errors.error

--> Messages:

--> com.vmware.vcIntegrity.lifecycle.TaskError.HealthCheckFailed<Health Check for 'Cluster name' failed>

-->

2024-09-05T19:24:23.169Z warning vmware-vum-server[1424181] [Originator@6876 sub=TaskStatsCollector] [taskStatsCollector 190] Task type or creation time not present

2024-09-05T19:24:23.169Z info vmware-vum-server[1680038] [Originator@6876 sub=PM.AsyncTask.RemediateClusterTask{1751}] [vciTaskBase 1496] SerializeToVimFault fault:

--> (vmodl.fault.SystemError) {

--> faultCause = (vmodl.MethodFault) null,

--> faultMessage = (vmodl.LocalizableMessage) [

--> (vmodl.LocalizableMessage) {

--> key = "com.vmware.vcIntegrity.lifecycle.TaskError.HealthCheckFailed",

--> arg = (vmodl.KeyAnyValue) [

--> (vmodl.KeyAnyValue) {

--> key = "1",

--> value = "Cluster name"

--> }

--> ],

--> message = <unset>

--> }

--> ],

--> reason = "vLCM Task failed, see Error Stack for details."

--> msg = "{

--> "data": null,

--> "error_type": "ERROR",

--> "messages": [

--> {

--> "args": [

--> "Cluster name"

--> ],

--> "default_message": "Health Check for 'Cluster name' failed",

--> "id": "com.vmware.vcIntegrity.lifecycle.TaskError.HealthCheckFailed"

--> }

--> ]

--> }"

--> }

--> Converted fault:

--> (vim.fault.ExtendedFault) {

--> faultCause = (vmodl.MethodFault) null,

--> faultMessage = (vmodl.LocalizableMessage) [

--> (vmodl.LocalizableMessage) {

--> key = "com.vmware.vcIntegrity.lifecycle.TaskError.HealthCheckFailed",

--> arg = (vmodl.KeyAnyValue) [

--> (vmodl.KeyAnyValue) {

--> key = "1",

--> value = "cluster name"

--> }

--> ],

--> message = <unset>

--> }

--> ],

--> faultTypeId = "SystemError",

--> data = (vim.KeyValue) [

--> (vim.KeyValue) {

--> key = "faultCause",

--> value = ""

--> },

--> (vim.KeyValue) {

--> key = "reason",

--> value = "vLCM Task failed, see Error Stack for details."

--> }

--> ]

--> msg = "{

--> "data": null,

---------------------------------------------

From the NSX-t

Upgrade coordinator.log

2024-09-05T19:24:23.060Z INFO http-nio-127.0.0.1-7442-exec-1 NsxBaseRestController 93446 SYSTEM [nsx@6876 audit="true" comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] UserName:'vpxd-extensio

n-000000000000000000' ModuleName:'common-services' Operation:'POST@/api/v1/vlcm/esx/health/cluster/perspectives/ready-for-apply/status' Operation status: 'failure' Error: General error has

occurred.

2024-09-05T19:24:23.060Z WARN http-nio-127.0.0.1-7442-exec-1 ExceptionHandlerExceptionResolver 93446 Failure in @ExceptionHandler com.vmware.nsxapi.ehp.controller.EHPController#handleBaseException(BaseExce

ption, HttpServletResponse, HttpServletRequest)

java.lang.NullPointerException: null

at com.vmware.nsx.management.common.NsxBaseRestController.handleBaseException(NsxBaseRestController.java:157) ~[libmp_common_platform.jar:?]

at sun.reflect.GeneratedMethodAccessor217.invoke(Unknown Source) ~[?:?]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_382]

2024-09-05T19:24:23.063Z ERROR http-nio-127.0.0.1-7442-exec-1 error_jsp 93446 - [nsx@6876 comp="nsx-manager" errorCode="MP98" level="ERROR" subcomp="upgrade-coordinator"] Uncaught exception

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is com.vmware.nsx.management.common.exceptions.BaseException: Service account is not enabled on VC-000000000000000000000

at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-5.3.33.jar:5.3.33]

at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) ~[spring-webmvc-5.3.33.jar:5.3.33]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:555) ~[servlet-api.jar:4.0.FR]

at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.3.33.jar:5.3.33]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:623) ~[servlet-api.jar:4.0.FR]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209) ~[catalina.jar:9.0.83]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) ~[catalina.jar:9.0.83]

at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-websocket.jar:9.0.83]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178) ~[catalina.jar:9.0.83]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153) ~[catalina.jar:9.0.83]

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:337) ~[spring-security-web-5.6.12.jar:5.6.12]

at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115) ~[spring-security-web-5.6.12.jar:5.6.12]

at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81) ~[spring-security-web-5.6.12.jar:5.6.12]

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12]

at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:122) ~[spring-security-web-5.6.12.jar:5.6.12]

at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:116) ~[spring-security-web-5.6.12.jar:5.6.12]

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12]

at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:109) ~[spring-security-web-5.6.12.jar:5.6.12]

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12]

at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:149) ~[spring-security-web-5.6.12.jar:5.6.12]

at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:346) ~[spring-security-web-5.6.12.jar:5.6.12]

at org.springframework.security.web.authentication.preauth.AbstractPreAuthenticatedProcessingFilter.doFilter(AbstractPreAuthenticatedProcessingFilter.java:142) ~[spring-security-web-5.6.12.jar:5.6.1

Resolution

Enable create Service account and also enable trust on the compute manager

Example:

Upgrade your ESXI host using VLCM ---> Once this is enabled: From Vcenter, you should be able to upgrade the ESXI host using VLCM.

Additional Information

Support articles: