Commissioning Hosts in SDDC Manager Fails with Error, HOST_COMMISSION operation validation failed due to: HOST_COMMISSION operation cannot be performed on resource UUID as these checks have failed "Failed to evaluate" Error

Products

• VMware SDDC Manager

Issue/Introduction

Commissioning Hosts in SDDC Manager Fails with below Error, 

HOST_COMMISSION operation validation failed due to: HOST_COMMISSION operation cannot be performed on resource dae567e6-1024-42cb-9f8c-d369944ab782 as these checks have failed Failed to evaluate lambda class LambdaInputRest { checkSetType: DAY_N_GUARDRAILS domainIds: [dae567e6-1024-42cb-9f8c-d369944ab782] targets: [class CheckTargetRest { type: HOST_COMMISSION resourceId: dae567e6-1024-42cb-9f8c-d369944ab782 }] forceRefresh: true }

 

In operationsmanager logs we see below entries,

yyyy-mm-ddThh:mm:ss.519+0000 DEBUG [vcf_om,66fe55a92ee8ed964e61f08ee595a233,f949] [c.v.vcf.assessment.sdk.LambdaService,pool-3-thread-1] DAY_N_GUARDRAILS check type HOST_COMMISSION 46e47a56-82df-49b0-a8f2-640cfebb777f status is FAIL
yyyy-mm-ddThh:mm:ss.519+0000 ERROR [vcf_om,66fe55a92ee8ed964e61f08ee595a233,f949] [c.v.vcf.assessment.sdk.LambdaService,pool-3-thread-1] Lambda task 46e47a56-82df-49b0-a8f2-640cfebb777f failed due to {"errorCode":"ASSESSMENT_CANNOT_ADD_CONNECTION","arguments":["nsxmgr.domain.com"],"message":"Cannot connect to nsxmgr.domain.com","cause":[{"type":"com.vmware.vapi.std.errors.Unauthorized","message":"Unauthorized (com.vmware.vapi.std.errors.unauthorized) => {\n    messages = [],\n    data = struct => {error_message=The credentials were incorrect or the account specified has been locked., error_code=403, module_name=common-services},\n    errorType = UNAUTHORIZED\n}"}],"referenceToken":"*****"}
yyyy-mm-ddThh:mm:ss.522+0000 ERROR [vcf_om,66fe55a92ee8ed964e61f08ee595a233,f949] [c.v.v.a.s.AssessmentLambdaServiceClient,pool-3-thread-1] Encountered exception while executing guardrail validations
java.lang.IllegalStateException: Failed to evaluate lambda class LambdaInputRest {
    checkSetType: DAY_N_GUARDRAILS
    domainIds: [dae567e6-1024-42cb-9f8c-d369944ab782]
    targets: [class CheckTargetRest {
        type: HOST_COMMISSION
        resourceId: dae567e6-1024-42cb-9f8c-d369944ab782
..
..
        at java.base/java.lang.Thread.run(Thread.java:840)
yyyy-mm-ddThh:mm:ss.522+0000 ERROR [vcf_om,66fe55a92ee8ed964e61f08ee595a233,f949] [c.v.e.s.c.s.a.DayNGuardrailsValidationAction,pool-3-thread-1] Failed to run HOST_COMMISSION validations due to error: HOST_COMMISSION operation cannot be performed on resource dae567e6-1024-42cb-9f8c-d369944ab782 as these checks have failed Failed to evaluate lambda class LambdaInputRest {
    checkSetType: DAY_N_GUARDRAILS
    domainIds: [dae567e6-1024-42cb-9f8c-d369944ab782]
    targets: [class CheckTargetRest {
        type: HOST_COMMISSION
        resourceId: dae567e6-1024-42cb-9f8c-d369944ab782
..
..
yyyy-mm-ddThh:mm:ss.526+0000 ERROR [vcf_om,66fe55a92ee8ed964e61f08ee595a233,f949] [c.v.e.s.o.model.error.ErrorFactory,pool-3-thread-1] [IQNQ0E] DAYN_GUARDRAILS_VALIDATION_FAILED HOST_COMMISSION operation validation failed due to: HOST_COMMISSION operation cannot be performed on resource dae567e6-1024-42cb-9f8c-d369944ab782 as these checks have failed Failed to evaluate lambda class LambdaInputRest {
    checkSetType: DAY_N_GUARDRAILS
    domainIds: [dae567e6-1024-42cb-9f8c-d369944ab782]
    targets: [class CheckTargetRest {
        type: HOST_COMMISSION
        resourceId: dae567e6-1024-42cb-9f8c-d369944ab782
..
..
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: HOST_COMMISSION operation validation failed due to: HOST_COMMISSION operation cannot be performed on resource dae567e6-1024-42cb-9f8c-d369944ab782 as these checks have failed Failed to evaluate lambda class LambdaInputRest {
    checkSetType: DAY_N_GUARDRAILS
    domainIds: [dae567e6-1024-42cb-9f8c-d369944ab782]
    targets: [class CheckTargetRest {
        type: HOST_COMMISSION

 

Environment

SDDC Manager 5.2

Cause

SDDC Manager is unable to connect to NSXT Manager.

Resolution

Check the status of the NSXT manager credentials in SDDC Password Management and make sure they working with below steps and are in ACTIVE state on SDDC UI.

 

View the passwords using lookup_passwords command in SDDC for NSXT_MANAGER, confirm these passwords working on NSXT.

If the NSXT manager's passwords are expired, we can perform the steps mentioned in below doc,

https://knowledge.broadcom.com/external/article/314657/credential-operations-fail-on-nsx-compon.html