VCF 9.0 Bringup Fails at ESX Host Validation - Failed to Retrieve Compatibility Matrix for Component
search cancel

VCF 9.0 Bringup Fails at ESX Host Validation - Failed to Retrieve Compatibility Matrix for Component

book

Article ID: 403075

calendar_today

Updated On:

Products

VMware SDDC Manager

Issue/Introduction

Step 13. Validate and Deploy fails with errors such as:

Failed to retrieve compatibility matrix for component VMware Cloud Foundation Operations with version 9.0.0.0.########

Failed to retrieve compatibility matrix for component VMware Cloud Foundation Operations fleet management with version 9.0.0.0.#######

Failed to retrieve compatibility matrix for component VMware vCenter with version 9.0.0.0.#######

Failed to retrieve compatibility matrix for component SDDC Manager with version 9.0.0.0.#######

Failed to retrieve compatibility matrix for component VMware NSX with version 9.0.0.0.#######

domainmanager.log:

DEBUG [vcf_dm,<trace-id>,<span-id>] [c.v.e.s.o.c.c.ContractParamBuilder,dm-exec-<thread-id>] Contract task Verify components interop compatibility result: {"response":{"errorCode":"VALIDATE_COMPONENT_INTEROPERABILITY.error","arguments":[],"context":{"severity":"ERROR","bundleName":"com .vmware.evo.sddc.common.validation.errors.messages","validation.taskId":"<validation-task-id>"},"message":"Failed to verify components interoperab ility","nestedErrors":[{"errorCode":"INTEROPERABILITY_CHECK_FAILED.error","arguments":["VMware Cloud Foundation Operations","9.0.0.0.24695812","File with Compatib ility Matrix Content for Compatibility controller VMWARE_COMPAT is not found."],"context":{"severity":"ERROR","bundleName":"com.vmware.evo.sddc.common.validation. errors.messages","validation.taskId":"<validation-task-id>"},"message":"Failed to retrieve compatibility matrix for component VMware Cloud Foundat ion Operations with version 9.0.0.0.24695812"},{"errorCode":"INTEROPERABILITY_CHECK_FAILED.error","arguments":["VMware Cloud Foundation Operations fleet managemen t","9.0.0.0.24695816","File with Compatibility Matrix Content for Compatibility controller VMWARE_COMPAT is not found."],"context":{"severity":"ERROR","bundleName ":"com.vmware.evo.sddc.common.validation.errors.messages","validation.taskId":"<validation-task-id>"},"message":"Failed to retrieve compatibility matrix for component VMware Cloud Foundation Operations fleet management with version 9.0.0.0.24695816"},{"errorCode":"INTEROPERABILITY_CHECK_FAILED.error","argum ents":["VMware vCenter","9.0.0.0.24755230","File with Compatibility Matrix Content for Compatibility controller VMWARE_COMPAT is not found."],"context":{"severity ":"ERROR","bundleName":"com.vmware.evo.sddc.common.validation.errors.messages","validation.taskId":"<validation-task-id>"},"message":"Failed to re trieve compatibility matrix for component VMware vCenter with version 9.0.0.0.24755230"},{"errorCode":"INTEROPERABILITY_CHECK_FAILED.error","arguments":["SDDC Man ager","9.0.0.0.24703748","File with Compatibility Matrix Content for Compatibility controller VMWARE_COMPAT is not found."],"context":{"severity":"ERROR","bundleN ame":"com.vmware.evo.sddc.common.validation.errors.messages","validation.taskId":"<validation-task-id>"},"message":"Failed to retrieve compatibili ty matrix for component SDDC Manager with version 9.0.0.0.24703748"},{"errorCode":"INTEROPERABILITY_CHECK_FAILED.error","arguments":["VMware NSX","9.0.0.0.2473306 3","File with Compatibility Matrix Content for Compatibility controller VMWARE_COMPAT is not found."],"context":{"severity":"ERROR","bundleName":"com.vmware.evo.s ddc.common.validation.errors.messages","validation.taskId":"<validation-task-id>"},"message":"Failed to retrieve compatibility matrix for componen t VMware NSX with version 9.0.0.0.24733063"}]}}
ERROR [vcf_dm,<trace-id>,<span-id>] [c.v.v.c.v.i.CompatibilityControllerAggregator,dm-exec-<thread-id>] None of the controllers are in READY state
ERROR [vcf_dm,<trace-id>,<span-id>] [c.v.v.v.f.a.v.ValidateEsxiVersionAction,dm-exec-<thread-id>] Failed to validate ESXi host version
DEBUG [vcf_dm,<trace-id>,<span-id>] [c.v.e.s.c.v.util.ResponseUtil,dm-exec-<thread-id>] Build validation response: {"errorCode":"ESXI_VERSION_VALIDATION_STATUS.error","arguments":[],"context":{"severity":"ERROR","bundleName":"com.vmware.evo.sddc.common.validation.errors.messages","validation.taskId":"<validation-task-id>"},"message":"Validate ESX Host version and build failed","cause":[{"type":"com.vmware.vcf.compatibility.validator.exception.CompatibilityValidatorException","message":"File with Compatibility Matrix Content for Compatibility controller VMWARE_COMPAT is not found."}]}
DEBUG [vcf_dm,<trace-id>,<span-id>] [c.v.e.s.o.c.c.ContractParamBuilder,dm-exec-<thread-id>] Contract task Validate ESXi version and build result: {"response":{"errorCode":"ESXI_VERSION_VALIDATION_STATUS.error","arguments":[],"context":{"severity":"ERROR","bundleName":"com.vmware.evo.sddc.common.validation.errors.messages","validation.taskId":"<validation-task-id>"},"message":"Validate ESX Host version and build failed","cause":[{"type":"com.vmware.vcf.compatibility.validator.exception.CompatibilityValidatorException","message":"File with Compatibility Matrix Content for Compatibility controller VMWARE_COMPAT is not found."}]}}
ERROR [vcf_dm,<trace-id>,<span-id>] [c.v.e.s.c.util.LocalProcessService,dm-exec-<thread-id>] Local Command Failed with exit value 1.

Environment

VCF 9.0 Installer

VMware Cloud Foundation 9

Cause

Precheck validations are unable to reach the required files.

Resolution

  1. Connect to the Installer appliance via SSH, switch user to 'root'.

  2. Backup the LCM application properties file:

    cp /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties.bak

  3. Modify the LCM application properties file:

    • Open the file for editing:
      vi /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties 
      • Locate the line vcf.compatibility.controllers.compatibilityCheckEnabled= and change its value to false. If the line does not exist, add it to the end of the file:
        vcf.compatibility.controllers.compatibilityCheckEnabled=false
      • Save and exit the file (in vi, press Esc, then :wq!, then Enter).

  4. Restart the LCM service:

    systemctl restart lcm.service

  5. Backup the Domain Manager application properties file:

    cp /etc/vmware/vcf/domainmanager/application-prod.properties /etc/vmware/vcf/domainmanager/application-prod.properties.bak

  6. Modify the Domain Manager application properties file:

    • Open the file for editing:
      vi /etc/vmware/vcf/domainmanager/application-prod.properties
    • Locate the line vcf.domainmanager.validation.enableVvsInteropValidation= and change its value to false. If the line does not exist, add it to the end of the file:
      vcf.domainmanager.validation.enableVvsInteropValidation=false
    • Save and exit the file.

  7. Restart the Domain Manager service:

    systemctl restart domainmanager.service

  8. Verify Services are Running:

    systemctl status lcm.service
    systemctl status domainmanager.service

    Ensure both show as active (running).

  9. Retest the operation that was previously failing due to compatibility checks.

Additional Information

This kb is also applicable in the scenario where VCF 9.x bring up is failing during validation due to below error:

VCF-installer logs:

domainmanager.log

ERROR [vcf_dm,f] [c.v.e.s.o.model.error.ErrorFactory,dm-XX]  [XXXX] COMPATIBILITY_CONTROLLER_EXCEPTION VvsCompatibilityController controller failed with error message : Invalid VMwareProductVersion format: 8.18.6-Ops Fresh Install: 25211474 Ops CP:25211476 Ops Upgrade: 25211477

Powered by Wolken Software