Skip level upgrades are not enabled for product components after VCF is upgraded to 4.3.0.0 / 4.3.1

Products

• VMware Cloud Foundation

Issue/Introduction

After performing skip level upgrade to VCF 4.3.0.0 / 4.3.1 from 4.1.x or 4.2.x, one or more of the following symptoms is observed:

• vRealize bundles do not show up as available for upgrade
• Bundles of previous versions for product components (NSX-T / VC/ ESXi) show up as available for upgrade
• /var/log/vmware/vcf/lcm/lcm-debug.log shows the below message
  

  YYYY-MM-DDTHH:MM:SS.sssZ WARN  [vcf_lcm,60319965ff2c4524,435c] [c.v.e.s.l.s.UpgradeServiceHelper,pool-7-thread-5] NSX-T cluster with ID: vip-nsxt-mgmt.vcf.local, name: vip-nsxt-mgmt.vcf.local, version: {"version":"3.1.2.0.0.17883596"} is partially upgraded
  YYYY-MM-DDTHH:MM:SS.sssZ WARN  [vcf_lcm,60319965ff2c4524,435c] [c.v.e.s.l.s.i.SkipUpgradeServiceImpl,pool-7-thread-5] Skipping aliasing to VCF version 4.3.0.0 since there are partial NSX-T upgrade on the system 

• Upgrade activity log in SDDC Manager UI for VCF upgrade will have message as follows:
  
  
  • /var/log/vmware/capengine/cap-update/install

Skipped enabling skip level upgrade to VCF version 4.3.0.0 since there are partial NSX-T upgrade on the system, please complete NSX-T upgrade first, then call API /v1/system/settings/version-aliases to enable skip level upgrade to the target VCF version

Environment

VMware Cloud Foundation 4.3.1
VMware Cloud Foundation 4.3

Cause

• NSX-T is upgraded out of band on source VCF version
• There were incomplete NSX-T upgrades on source VCF version and VCF is upgraded to 4.3.0.0 / 4.3.1

Resolution

Recommend to follow the workaround.

Workaround:
Follow the below steps to workaround the issue:

1. Complete NSX-T upgrades on remaining workload domains
   • From SDDC Manager UI, navigate to "Updates and Patches" tab of each of the workload domains and verify if any NSX-T upgrade is shown as available
   • If NSX-T upgrade is shown as available, complete the upgrade.
   • Proceed to Step 2 only after NSX-T upgrades are applied.
2. Update version aliasing configuration to enable skip level upgrades to target release
   • Login to SDDC Manager via SSH
   • Prepare request payload for version alias API invocation
     

     vi request.json
             {
                  "forceUpdate": true,
                  "targetVcfVersion": "4.3.0.0"
              }
      

3. Generate access token using following API

   • curl -X POST -H "Content-Type: application/json" -d '{"username": "<SSO_USERNAME>","password": "<SSO_USER_PASSWORD>"}'  http://localhost/v1/tokens

   • Copy value of "accessToken" field from API output
4. Invoke version alias API with request payload and access token

   • curl -H "Content-Type: application/json" -H "Authorization: Bearer <ACCESS_TOKEN>" http://localhost/v1/system/settings/version-aliases -X PUT -d @request.json -H 'Content-Type:application/json' | json_pp

5. Refresh the "Updates and Patches" tab for management domain from SDDC Manager UI