Upgrade Sequence and Related Issues for VMware Cloud Foundation and vSphere Foundation 9.1

Products

• VMware Cloud Foundation
• VMware vSphere Foundation

Issue/Introduction

A mandatory component upgrade sequence must be followed when upgrading the environment to version 9.1. Workload domains can be upgraded later as a Day-N procedure.

VMware Cloud Foundation (VCF) 9.1 introduces enhancements designed to simplify private cloud infrastructure management:

• VCF Management Services: VCF 9.1 introduces a common runtime and a unified set of components for lifecycle and operational capabilities. This architecture layer is a mandatory, required part of the deployment. For more information, see VCF Management Services.
  • Minimum 12 IP addresses required for deployment.
  • An additional 18 IP addresses can be added afterwards for the following scenarios:
    • New VCF Management Services are added.
    • Scale out of VCF Management services or new nodes as upgrade to future versions take place.
  • A maximum of 30 IP addresses are required, these can be added as new IP ranges in the Lifecycle section of the VCF Operations UI and these ranges do not need to be continuous.
  • All IP Ranges must be on the management network.
  • An internal IP range is used by VCF services runtime, this IP range is 198.18.0.0/15, it is important to ensure this IP range does not overlap with the IP range of your management network. This internal network can be changed to 240.0.0.0/15 or 250.0.0.0/15 by using JSON specification file to deploy.
  • For more information, refer VCF Components FQDNs and IP addresses.
• Decommissioning of the 9.0 Fleet Management Appliance: VCF 9.1 completely replaces the standalone Fleet Management Appliance introduced in version 9.0. It is replaced with two new services—Fleet Lifecycle and SDDC Lifecycle—which run natively within VCF Management Services. The Fleet Management appliance is powered down and ready to be decommissioned.
• Introduction of the Centralized VCF License Server: A new appliance that centralizes license management, offering flexible scaling and local control. The license server is a mandatory component for VCF and vSphere Foundation environments.
  • An internal IP range is used by VCF license server runtime, this IP range is 172.17.0.0/16, it is important to ensure this IP range does not overlap with the IP range of your management network. For more information, see VCF 9.1 License Server console displays a black screen during boot due to Docker network conflict.
• Consolidation of the 9.0 VMware Identity Broker (vIDB): In VCF 9.1, the external multi-node appliance cluster used for vIDB in VCF 9.0 is migrated directly into VCF Management Services, and the standalone VMs are powered down and ready to be decommissioned.

Now available: The VCF Upgrade Planning Tool provides a custom-tailored upgrade planning experience to help customers confidently navigate their VCF 9.1 upgrade journey, whether you are starting from a vSphere-based deployment or some combination including NSX, Aria Automation to a full VCF environment.

Environment

• VMware vSphere Foundation 9.1.
• VMware Cloud Foundation 9.1.

Resolution

Table of Contents

1. Upgrading from VMware Cloud Foundation 5.2.x or 9.0.x to 9.1
2. Upgrading to vSphere Foundation 9.1
3. Common Related Issues
4. Upgrade Paths to VCF 9.1

Follow the appropriate upgrade guide below based on the current version.

Upgrading from VMware Cloud Foundation 5.2.x or 9.0.x to 9.1

1. Upgrading from VMware Cloud Foundation 5.2.x to 9.1
2. Upgrading from VMware Cloud Foundation 9.0.x to 9.1

Upgrading to vSphere Foundation 9.1

1. Upgrading to vSphere Foundation 9.1
2. Upgrading from standalone components of vSphere 8 and Aria Operations 8 to 9.1

Common Related Issues

An incorrect upgrade path can trigger the errors described in the following KBs.

1. VCF 9.1 upgrade binaries do not show up in VCF Operations 9.0.
2. Unable to license vCenter after upgrading to 9.1, One or more vCenter instances are not connected to a license server.
3. Licenses could not be assigned to vCenters, license assignment failure on VCF 9.1.
4. Upgrading VCF Operations from 9.0.2 to 9.1 returns sync error when upgrading ESXi host.
5. Upgrading VCF 9.0 environment to 9.1 fails when trying to "Import VCF Operations in Fleet Lifecycle" due to VCF Operations for Networks certificate SAN issue.

Upgrade Paths to VCF 9.1.

#	Existing deployment type	Option A	Option B
1	vCenter and ESX only deployment version 8.x or 9.0.x (with or without vSAN)	Use Installer workflows to upgrade to VCF or VVF.  If you have VCF entitlement – you can upgrade to full VCF or just VVF. Benefit: all VCF Fleet Management capabilities.	Download and install VCF Operations. Install license server through VCF Operations workflows. Get new license entitlement. Upgrade vCenters. Upgrade ESX hosts. Benefit: upgrade to latest version of vSphere and leverage new vSphere capabilities such as Memory Tiering and new VCF Operations capabilities.
2	vCenter and ESX only deployment version 8.x or 9.0.x (with or without vSAN) Aria Operations 8.x or VCF Operations 9.0.x	Use Installer workflows to upgrade to VCF or VVF.  If you have VCF entitlement – you can upgrade to full VCF or just VVF. Benefit: all VCF Fleet Management capabilities.	In-place upgrade Aria Operations to VCF Operations. Install license server through VCF Operations workflows. Get new license entitlement. Upgrade vCenter. Upgrade ESX hosts. Benefit: upgrade to latest version of vSphere and leverage new vSphere capabilities such as Memory Tiering and new VCF Operations capabilities.
3	vCenter and ESX only deployment version 8.x or 9.0.x (with or without vSAN) Aria Operations 8.x or VCF Operations 9.0.x (if no Aria Operations or VCF Operations, download and install VCF Operations 9.1 first as 1 above) NSX 4.x	Use VCF Installer converge workflows to upgrade to VCF. Perform the convert/import operations through the VCF Installer workflows.	There is no option to upgrade independent components. When NSX is present, the only option is A, using VCF Installer workflows with convert/import operations.
4	vCenter and ESX only deployment version 8.x or 9.0.x (with or without vSAN) Aria Operations 8.x or VCF Operations 9.0.x (if no Aria Operations or VCF Operations, download and install VCF Ops 9.1 first as 1 above) Aria Automation 8.x With or without NSX 4.x	Use VCF Installer converge workflows to upgrade to VCF. Perform the convert/import operations through the VCF Installer workflows.	There is no option to upgrade independent components. When NSX is present, the only option is A, using VCF Installer workflows with convert/import operations.

 

#	Existing deployment type	Upgrade path to 9.1
5	VCF 5.2.x or 9.0.x using SDDC Manager	Upgrade VCF Operations, then follow LCM workflows to upgrade to VCF 9.1.
6	vRealize Suite Lifecycle Manager 8.x with VMware Aria Automation and VMware Identity Manager	Upgrade VCF Operations, then follow LCM workflows to upgrade to VCF 9.1. VMware Aria Automation in-place upgrade to VCF Automation 9.1. vRealize Suite Lifecycle Manager and VMware Identity Manager continues to work with VCF Automation 9.1. No impact to blueprints and single sign-on post upgrade to VCF Automation 9.1. Day 2: Manually change the identity configuration to new VMware Identity Broker. Decommission vRealize Suite Lifecycle Manager and VMware Identity Manager. VMware Identity Manager to VMware Identity Broker migration is made available.
7	VCF 9.0.x with VCF Fleet Management Appliance	VCF Fleet Management Appliance is no longer available in 9.1. The capability of the Fleet LCM is now moved to VCF Management Services. Do not use VCF Fleet Management Appliance to do any upgrades, it can be decommissioned once the environment is upgraded to VCF 9.1.
8	VCF 9.0.x with VMware Identity Broker	Upgrade VCF Operations, then follow LCM workflows to upgrade to VCF 9.1. If VMware Identity Broker 9.0.x is on the management network, no downtime required during upgrade. If VMware Identity Broker 9.0.x is on a non-mgmt network (regular vlan or NSX overlay) – contact Support for assistance.
9	Aria Log Insight 8.x	Logs capability is now a service part of VCF Operations and VCF Management Services. There is no longer a separate appliance for Log Management. Aria Log Insight continues work after all the management components are upgraded to VCF 9.1. Follow the VCF Installer workflow to upgrade to VCF 9.1 and enable Log Management Service through VCF Operations workflows. Follow the VCF Operations workflows to migrate the logs from Aria Log Insight to the new VCF Log Management 9.1 and decommission Aria Log Insight.
10	Aria Operations for Networks 6.x	Aria Operations for Networks appliance is still available in VCF 9.1. You cannot upgrade Aria Operations for Networks through its UI. Upgrade of Aria Operations for Networks only available through VCF Operations Fleet LCM workflows.