Products and versions are not listed in VCF

Products

• VMware Cloud Foundation

Issue/Introduction

• If VCF hasn't been deployed yet, VCF installer may have below messages under "Binary management" and unable to proceed with deployment:
  
  Initializing binary databbase, please wait...
  
  
• If VCF environment has already been deployed, the patch version may not be listed in SDDC manager.
• Error message in /var/log/vmware/vcf/lcm/lcm-debug.log:
  
  Failed read/parse 'Product Version Catalog' from file /nfs/vmware/vcf/nfs-mount/metadata/productVersionCatalog.json
Error occurred fetching product version catalog file from depot com.vmware.evo.sddc.lcm.model.error.LcmException: Failed read/parse 'Product Version Catalog' from file /nfs/vmware/vcf/nfs-mount/metadata/productVersionCatalog.json
at com.vmware.evo.sddc.Lcm.services.impl.LcmProductVersionCataLogServicelmpl.getexistingProductVersionCataLogFromLocalFile(LcmProductVersionCatalogServiceImpl.java:456)
  
  
• Similar messages can be observed in /var/log/vmware/vcf/domainmanager/domainmanager.log:
  
  Encountered an exception during the call to get LCM releases
org.springframework.web.client.HttpServerErrorException$InternalServerError: 500 : "{"errorCode":"VCF_ERROR_INTERNAL_SERVER_ERROR","arguments":[],"message":"A problem has occurred on the server. Please retry or contact the service provider and provide the reference token.","causes":[{"type":"com.vmware.evo.sddc.lcm.model.error.LcmException","message":"Failed to read product version catalog from the database. Exception: {0}"},{"type":"com.fasterxml.jackson.core.JsonParseException","message":"Illegal character ((CTRL-CHAR, code 0)): only regular white space (\\r, \\n, \\t) is allowed between tokens\n at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 2]"}]

Environment

VMware Cloud Foundation 9.x

Cause

The ProductVersionCatalog.json file is corrupted.

Resolution

The LCM scheduler downloads the ProductVersionCatalog file from the Broadcom depot every five minutes and replaces the local file in the metadata folder. (Only applicable to Online Environments. Please see doc for offline environments) 

1. SSH into the SDDC Manager appliance with the vcf user
2. Delete the ProductVersionCatalog file
   rm -rf /nfs/vmware/vcf/nfs-mount/metadata/productVersionCatalog.json 
   
   

Additional Information

For offline environment, the issue can be fixed by replacing the ProductVersionCatalog.json with a copy from offline depot if the file on offline depot appears to be good.