New Workload Domain deployment or Adding New ESXi Host in a VxRail environment fails at the 'NSX-T Generate Input data' stage
search cancel

New Workload Domain deployment or Adding New ESXi Host in a VxRail environment fails at the 'NSX-T Generate Input data' stage

book

Article ID: 384469

calendar_today

Updated On:

Products

VMware SDDC Manager / VCF Installer

Issue/Introduction

  • Scenario 1 (Creating New WLD):
     
    • Under the SDDC Manager UI, below error message is available:

      Failed to fetch images from LCM Service with error.
      Remediation Message: Download the necessary images and retry workflow

    • Under SDDC Manager's /var/log/vmware/vcf/domainmanager.log, below error snips are found:

      YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,##################,####] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-##]  [TOKEN] FAILED_TO_FETCH_IMAGES Failed to fetch the images from LCM Service with error: .
      com.vmware.evo.sddc.common.services.error.SddcManagerServicesIsException: Failed to fetch the images from LCM Service with error: .
              at com.vmware.evo.sddc.common.services.adapters.imagemanagementservice.ImageManagementServiceAdapterImpl.getDomainVersionMatrice
      Caused by: com.vmware.cloud.foundation.rest.lcm.runtime.ApiException:
              at com.vmware.cloud.foundation.rest.lcm.runtime.ApiClient.handleResponse(ApiClient.java:788)
              at com.vmware.cloud.foundation.rest.lcm.runtime.ApiClient.execute(ApiClient.java:708)

    • Under SDDC Manager's /var/log/vmware/vcf/lcm/lcm-debug.log, below error snips are found:

      YYYY-MM-DDTHH:MM:SS ERROR [vcf_lcm,##################,####] [c.v.e.s.l.a.r.c.i.u.InventoryUpgradeContro
      ller,http-nio-127.0.0.1-7400-exec-1] In InventoryUpgradeController, Exception in getting all inventory upgrades
      com.vmware.evo.sddc.common.core.error.InvalidStateException: Management Domain not found

      OR

      YYYY-MM-DDTHH:MM:SS INFO  [vcf_lcm,##################,####] [c.v.e.s.l.a.r.c.i.ImageManagementControlle
      r,http-nio-127.0.0.1-7400-exec-#] get version compliance matrix for domain VI
      YYYY-MM-DDTHH:MM:SS ERROR [vcf_lcm,##################,####] [c.v.e.s.l.s.i.ImageManagementServiceImpl,h
      ttp-nio-127.0.0.1-7400-exec-#] MGMT domain collection size 0 is not equal to 1
      YYYY-MM-DDTHH:MM:SS ERROR [vcf_lcm,##################,####] [c.v.e.s.l.s.i.ImageManagementServiceImpl,h
      ttp-nio-127.0.0.1-7400-exec-1] Error while getting version compliance matrix for VI domain - {}
      java.lang.RuntimeException: MGMT domain collection size is not equal to 1

      YYYY-MM-DDTHH:MM:SS INFO  [vcf_dm,##################,####] [c.v.v.v.c.f.a.WaitForVxRailFirstRunToComplete,dm-exec-15]  VxRail first run workflow is completed successfully
      YYYY-MM-DDTHH:MM:SS DEBUG [vcf_dm,##################,####] [c.v.e.s.o.c.c.ContractParamBuilder,dm-exec-##]  Contract task Monitor VxRail First Run .

      ...............

      YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,##################,####] [c.v.e.s.c.s.a.i.ImageManagementServiceAdapterImpl,dm-exec-##]  Error in fetching Images from LCM

  • Scenario 2 (Adding New Host to an Existing Cluster):

    • Under the SDDC Manager UI, below error message is available:

      "Error in Fetching Workflow Options for Addition of ESXi Host to Cluster."

    • Under SDDC Manager's /var/log/vmware/vcf/domainmanager/domainmanager.log, below log snips are found:

      YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,##################,####] [c.v.v.n.c.v.v.ALBClusterValidator,http-nio-127.0.0.1-####-exec-#]  Unable to get releases for domain ########-####-####-####-############. Status Code - 500, Response - {"errorCode":"VCF_ERROR_INTERNAL_SERVER_ERROR","arguments":[],"message":"A problem has occurred on the server. Please retry or contact the service provider and provide the reference token.","causes":[{"type":"com.vmware.evo.sddc.common.core.error.InternalServerErrorException","message":"A problem has occurred on the server. Please retry or contact the service provider and provide the reference token."},{"type":"com.vmware.evo.sddc.lcm.model.error.LcmException"}],"referenceToken":"TOKEN"}

    • Under SDDC Manager's /var/log/vmware/vcf/lcm/lcm-debug.log, below log snips are found:

      YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,##################,####] [c.v.v.n.c.v.v.ALBClusterValidator,http-nio-127.0.0.1-####-exec-#]  Unable to get releases
      for domain ########-####-####-####-###########. Status Code - 400, Response - {"errorCode":"DOMAIN_ID_INVALID","arguments":[],"message":"Provided domain ID is invalid.","referenceToken":"TOKEN"}
      com.vmware.vcf.rest.api.runtime.ApiException:

    • Under SDDC Manager's /var/log/vmware/vcf/lcm/lcm-debug.log, below log snips may also be found:

      YYYY-MM-DDTHH:MM:SS INFO  [vcf_lcm,##################,####] [c.v.v.l.r.a.c.v.m.LcmReleaseController,http-nio-127.0.0.1-####-exec-#] In LcmReleaseController, get release for domain ########-####-####-####-############
      YYYY-MM-DDTHH:MM:SS ERROR [vcf_lcm,##################,####] [c.v.v.l.r.a.c.v.m.LcmReleaseController,http-nio-127.0.0.1-####-exec-#] In LcmReleaseController, failed to get all releases
      com.vmware.evo.sddc.lcm.model.error.LcmException: null

      OR

      YYYY-MM-DDTHH:MM:SS ERROR [vcf_lcm,##################,####] [c.v.e.s.e.h.VcfFallbackErrorController,http-nio-127.0.0.1-####-exec-#] [TOKEN] - Error attributes: {timestamp=Wed May DD HH:MM:SS UTC YYYY, status=500, error=Internal Server Error, exception=java.lang.NullPointerException, trace=java.lang.NullPointerException: Cannot invoke "com.vmware.evo.sddc.lcm.model.internal.VCenter.getVCenterId()" because "vcenter" is null
              at com.vmware.evo.sddc.lcm.adapter.inventory.impl.InventoryClientImpl.getSsoCredentialFromDomain(InventoryClientImpl.java:1331)
              at jdk.internal.reflect.GeneratedMethodAccessor601.invoke(Unknown Source)
              at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.base/java.lang.reflect.Method.invoke(Method.java:569)

      Note: Node add might never start in the GUI. But, hangs on "fetching subtasks"

Environment

  • VCF on VxRail
  • SDDC Manager

Cause

This issue is caused by a null value for the 'type' row in the vCenter table in the SDDC database for the New or Existing WLD vCenter, which is shown as follows:

Resolution

To resolve this issue, please contact Broadcom Customer Support

Powered by Wolken Software