SDDC Manager upgrade precheck error : vSAN HCL DB on is out of date in SDDC manager UI

Products

• VMware Cloud Foundation

Issue/Introduction

• The pre-check test fails with the error: 
  
  vSAN HCL DB on SDDC Manager is out of date. This is a critical error if vSAN HCL DB has not been updated within the last 90.0 day(s) and a warning if vSAN HCL DB has not been updated within the last 60.0 day(s).
  
  

• On verifying the vsan hcl db update version in vcenter at vcenter-->configure-->vsan-->Update its already at the latest version

• Skyline health checks for vsan "Hardware compatibility" shows green for vSAN HCL up-to-date
• Updating vsan HCL DB via OBTU fails with
• Bundle Transfer Utility Tool failed with error :
  
  Api call to /v1/vsan-hcl/content failed.
Caused by: java.lang.IllegalArgumentException: URI path begins with multiple slashes

 

Environment

• VCF 5.x
• VCF 9.0

Resolution

1. Take snapshot of SDDC Manager VM
2. SSH to SDDC manager using vcf and su to root
3. Export the information from LCM DB using the below command
   
   psql -h localhost -U postgres -d lcm  -c "select * from vsan_hcl_attributes;" > /tmp/hclattributes.txt
   

   If SDDC Manager is 5.1.1 then use the below command
   
   /usr/pgsql/13/bin/psql -U postgres -h localhost -d lcm-c "select * from vsan_hcl_attributes;" > /tmp/hclattributes.txt

4. Review the output of the file you exported /tmp/hclattributes.txt
   
   less /tmp/hclattributes.txt
   

   note down the local_data_timestamp value which records the latest HCL publish date for vsan_hcl_attributes. To convert the local_data_timestamp like 1704067200 to human readable format - run date -d@1704067200. This makes it easier to see when the HCL was last updated.

   Note: if the value is outdated and/or not the latest then you get an error/warning as per the threshold value
   
   
5. Open the below link in your HTML browser:
   1. https://partnerweb.vmware.com/service/vsan/all.json (Right click, Save as)
   2. Save the file as all.json on your local machine. If you are unable to save the file, you must copy the entire content and create a new file with extension".json"
6. Copy the latest all.json file from your local machine to /home/vcf folder onto your SDDC Manager via tools like WinSCP
   
   
7. Use the below command to grab the access token
   
   curl -k --location 'https://<sddcmanager_fqdn>/v1/tokens' --header 'Content-Type: application/json' --data-raw '{"username":"<SSO_USERNAME>","password":"<SSO_PASSWORD>"}'

Example: Access Token JSON Response

When you request a token from SDDC Manager, you get a JSON response like this:

{ "accessToken": "<your_access_token_here>", "refreshToken": { "id": "<your_refresh_token_here>" } }

     8. cd to /home/vcf (where the latest local copy of all.json is placed)

     9. Upload the HCL file using the HCL upload API using the below command

    curl -k -X PUT --location 'https://<sddcmanager_fqdn>/v1/vsan-hcl/content' --header 'Authorization: Bearer <Your_access_token_here from step 7>' --header 'Content-Type: text/plain' [email protected]

     10. Post successful execution of the API the contents of local_data_timestamp in vsan_hcl_attributes at LCM DB will get updated to a new value, please verify the same as stated in step 3

     11. Retry the pre-check to verify that SDDC no longer complain that vSAN HCL DB on SDDC Manager is out of date

     12. Or copy the latest all.json to /home/vcf then use the following command. About lcm-bundle-transfer-util please refer HCL Offline Download for VMware Cloud Foundation=

./lcm-bundle-transfer-util --vsanHclUpload /home/vcf/all.json

Additional Information

• For Updating the vSAN HCL database manually - refer Updating the vSAN HCL database manually
  
  
• Executing the commands from steps 7 and 9 might fail with an error: "curl: (7) Failed to connect to port 80 after 0 ms: Couldn't connect to server" due to a use of http instead of https.
  Modify the command to avoid the issue.
  Or
  Replace the <sddcmanager_fqdn> with localhost instead.
  
  Example:
  
  Use the below command to grab the access token:
     curl -k --location 'http://localhost/v1/tokens' --header 'Content-Type: application/json' --data-raw '{"username":"<SSO_USERNAME>","password":"<SSO_PASSWORD>"}'
  
  Upload the HCL file using the HCL upload API using the below command:
     curl -k -X PUT --location 'http://localhost/v1/vsan-hcl/content' --header 'Authorization: Bearer <put the token grabbed from step 7>' --header 'Content-Type: text/plain' [email protected]