SDDC Manager Cannot Find Bundles for NSX 4.2.3.3 when Patch Planning is set.

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

• After uploading the NSX and ESXi patch bundles, the SDDC Manager reports the following error when attempting to plan patching for NSX version 4.2.3.3 and ESXi version 8.0 U3g:
  "Cannot upgrade to VCF version 5.2.2.0 because bundles 4f03e04a-d29f-4306-973b-720cd5a951e3 are missing for domain <DOMAIN_NAME>". Please upload bundles 4f03e04a-d29f-4306-973b-720cd5a951e3. 
• The patch bundles are downloaded and uploaded successfully using the VCF download Tool (VDT); however, the SDDC Manager UI does not list the bundles in the download history.
• The /nfs/vmware/vcf/nfs-mount/bundle directory on the SDDC Manager contains no folders with these required IDs. Reviewing the manifest.json for each uploaded bundle reveals mismatched async patch IDs (NSX: 8b6a86c3-d90d-41c4-859d-4c8a010d9b6e, ESXi: 9e494f50-26f0-4c7d-a4b2-efcb6bd7eed3).
• The /var/log/vmware/vcf/lcm/lcm-debug.log reports the following: 

YYYY-MM-DDTHH:MM:SS.Z DEBUG [vcf_lcm,69d8cdbb505af2b4e1f41c2e6aa3ea52,7d2b] [c.v.e.s.l.s.i.LcmManifestServiceImpl,Scheduled-2] Required bundles for bundle software type VCENTER from VCF release {"version":"5.2.1.0"} to {"version":"5.2.3.0"} are [
  {
    "bundleSoftwareType": "VCENTER",
    "bundleId": "f16c7dfa-fde0-4ef0-bc91-2c738c9ca38a",
    "sequenceNumber": 7,
    "vcfVersion": {
      "version": "5.2.3.0"
    },
    "toVersion": "8.0.3.00800-25197330",
    "cumulativeFromVcfVersion": "4.1.0.0",
    "bundleManifestExists": false
  }
]
YYYY-MM-DDTHH:MM:SS.Z DEBUG [vcf_lcm,69d8cdbb505af2b4e1f41c2e6aa3ea52,7d2b] [c.v.e.s.l.s.i.LcmManifestServiceImpl,Scheduled-2] Required bundles for bundle software type NSX_T_MANAGER from VCF release {"version":"5.2.1.0"} to {"version":"5.2.3.0"} are [
  {
    "bundleSoftwareType": "NSX_T_MANAGER",
    "bundleId": "4f03e04a-d29f-4306-973b-720cd5a951e3",
    "sequenceNumber": 6,
    "vcfVersion": {
      "version": "5.2.3.0"
    },
    "toVersion": "4.2.3.3.0-25171318",
    "cumulativeFromVcfVersion": "4.5.0.0",
    "bundleManifestExists": false
  }
]
YYYY-MM-DDTHH:MM:SS.Z DEBUG [vcf_lcm,69d8cdbb505af2b4e1f41c2e6aa3ea52,7d2b] [c.v.e.s.l.s.i.LcmManifestServiceImpl,Scheduled-2] Required bundles for bundle software type ESX_HOST from VCF release {"version":"5.2.1.0"} to {"version":"5.2.3.0"} are [
  {
    "bundleSoftwareType": "ESX_HOST",
    "bundleId": "d1c575ce-bcd5-437a-a927-0bd869b27f94",
    "sequenceNumber": 8,
    "vcfVersion": {
      "version": "5.2.3.0"
    },
    "toVersion": "8.0.3-25205845",
    "cumulativeFromVcfVersion": "4.1.0.0",
    "bundleManifestExists": false
  }
]
YYYY-MM-DDTHH:MM:SS.Z DEBUG [vcf_lcm,69d8cdbb505af2b4e1f41c2e6aa3ea52,7d2b] [c.v.e.s.l.d.c.m.LcmManifestClientImpl,Scheduled-2] Found manifest with version 1 and sequence number 112

Environment

VMware Cloud Foundation 5.2.x

Cause

SDDC Manager utilizes separate IDs for upgrade bundles and patch bundles, prioritizing the lcmmanifest.json Bill of Materials (BOM) upgrade IDs over the uploaded async patch bundle IDs. When NSX 4.2.3.3 was incorporated into the VCF 5.2.3 BOM, it was assigned a new upgrade ID. The system searches for this BOM upgrade ID rather than the original async patch ID.

Resolution

1. Log in to the SDDC Manager appliance via SSH as the vcf user and switch to the root user.

2. Delete the previously uploaded async patch bundle using the bundle cleanup utility: python /opt/vmware/vcf/lcm/lcm-app/bin/bundle_cleanup.py 8b6a86c3-d90d-41c4-859d-4c8a010d9b6e

3. Download the NSX 4.2.3.3 bundle specifically targeting the VCF 5.2.3 BOM using the LCM transfer utility: /opt/vmware/vcf/lcm/lcm-tools/bin/lcm-bundle-transfer-util --download --depotDownloadToken <Path_To_Token_File> --sv <VERSION_REQUIRED> --p 5.2.3.0

4. Upload the newly downloaded NSX bundle using the VMware Diagnostics Tool (VDT) on the SDDC manager. 

5. Navigate to the SDDC Manager UI and proceed with the plan patching workflow.

Additional Information

While this specific issue was observed during NSX patching, the same bundle ID mismatch can occur with other components (such as vCenter Server or ESXi) if they were initially released as an async patch and subsequently incorporated into the Bill of Materials (BOM) of a later VMware Cloud Foundation (VCF) release