VCF 4.4 Manifest does not contain patch bundles causing skip upgrade to stop at 4.3.1

Products

• VMware Cloud Foundation

Issue/Introduction

Skip level should continue all the way to 4.4 without stopping at 4.3.1

Symptoms:
The skip level upgrade will stop at 4.3.1, showing incorrect bundles available on the UI, and missing 4.4 bundles. Version aliasing is only done up to 4.3.1 versions

Environment

VMware Cloud Foundation 4.4

Cause

When using skip level upgrade to vcf 4.4, the lcmManifest on the setup does not contain patch bundles information for vcf 4.4 release. If the sequence number of the setup's lcmManifest matches the sequence number of the lcmManifest on the depot, it won't be updated with the new patch bundle information that is on the depot. When patch bundles aren't available for a release, lcm assumes it cannot update to that and will stop at the latest release with patch bundles which is at 4.3.1. Although the vcf services will already be upgraded to 4.4, the BOM bundles available will only show up to 4.3.1. Version aliasing for 4.4 is also not completed.

Resolution

A new manifest is published with patch bundles details for VCF 4.4. The sequence number was incremented so it should always overwrite the manifest on the setup. When patch bundles information is correctly received, the correct bundles will show and the version aliasing will be correctly performed. 

Dark site Environments must make sure that LcmManifest is latest,
Refer : https://docs.vmware.com/en/VMware-Cloud-Foundation/4.4/vcf-lifecycle/GUID-8FA44ACE-8F04-47DA-845E-E0863094F7B0.html

Workaround:

- Make sure VC and NSX-T on management domain are stable
    - Login to appliance management console of vCenter and verify health status is shown as green
    - Login to NSX-T manager console and verify there are no red alarms
- Login to SDDC manager UI
- Run Manifests API (GET /v1/manifests) from developer center. Verify that 4.4 release section has patch bundles listed
- Run version aliasing API (PUT /v1/system/settings/version-aliases) with following payload from developer center
 
        {
             "forceUpdate": true,
             "targetVcfVersion": "4.4.0.0"
         } 
 
- Log out of SDDC Manager Ui and login again
- Verify that next available upgrade in MGMT domain is for VCF 4.4 BOM versions

Note :

Please check if all the components are in ACTIVE state before proceeding with VCF 4.4 upgrade. If VXRAIL was in ERROR state , please validate the health of VXRAIL before upgrading