Planned Activity Precheck Review Case for VMware Cloud Director

Products

• VMware Cloud Director

Issue/Introduction

Note: The Planned Activity Precheck Review outlined in this KB is available only to Advanced Support customers with a Support Account Manager (SAM) and/or Dedicated/Designated Support Engineer (DTSE/DSE). 

The Planned Activity Precheck Review is provided on a best effort basis and is intended to assist with identified issues and precheck failures. While Broadcom Support strives to highlight potential risks, this review is not exhaustive and may not identify all issues or environment-specific considerations. It is not a commitment to participate during the planned activity nor is it a certification or validation of a runbook. The customer remains responsible for validating their activity plan and ensuring their environment if fully prepared for the change. 

Prerequisites before opening a Planned Activity Precheck Review for VMware Cloud Director:

• Cases must be opened at least 10 business days in advance of the upgrade window.
• Cases should be opened as Low-P4.
• Collect a multi-cell Cloud Director log bundle and upload to the new case.
• Read the target version Release Notes and be aware of Known Issues.
• Reference the Interop Matrix to confirm the target Cloud Director version is compatible with other integrations in the environment and plan to upgrade those if necessary. Also confirm the Cloud Director upgrade path is supported.
• Have a development environment upgraded before production to test that any customization still functions as expected on the latest version.
• If this is a VCF environment, open a Planned Activity Precheck Review Case with VCF Support.
• Validate there are backups of VMware Cloud Director.
• Be aware that SHA-1 certificates are not supported at or beyond 10.5.1 and replace those certificates prior to the upgrade if applicable.
• Be familiar with Upgrade to VCD 10.6 GA fails installing few RPMs though upgrade is reported as successful.
• Ensure there is enough disk space on the appliances, before starting the upgrade. 

VMware by Broadcom Support will:

• Validate the target version and advise if a newer one is available.
• Review provided logs for known issues that may impact the upgrade.
• If a Planned Activity Precheck Review case is not raised with sufficient notice of 10 days, it may not be possible to offer log review and precheck resolution within the planned timeframe. 
• Once the review is complete, the Planned Activity Precheck Review case will be closed.
  • Two days prior to the upgrade, either you or your SAM/DTSE/DSE will open a maintenance case. This case will track the upgrade and can be uplifted to a Critical - P1 in the event of a production down outage during the upgrade. 
  • The next available support engineer will engage with you. Broadcom will not be able to provide dedicated support to be on standby during the activity. 

Please complete the following template when opening the Case: 

• Upgrade Date & Time:
• Cloud Director from version:
• Cloud Director to version:
• VCF from version:
• VCF to version:
• 2 or more upgrade paths required?

Upgrade Recommendations:

• It is recommended to retain locally the Cloud Director logs collected when opening the Planned Activity Precheck Review Case until the upgrade has completed successfully.
• Prepare a contingency plan to allow the upgrade maintenance window to be extended should a failure occur.
  • Schedule a maintenance window for the entire upgrade so that VMs can be powered off to make backups.  Make sure there are good backups just prior to the upgrade.  A good backup is one that restores have been tested and everything boots as expected. Powered off snapshots can provide a quick rollback point and are also required before starting the upgrade.
• Note: The order of Powering Off/On of Nodes is Extremely Important
  • Off: Primary, then Standby Nodes
  • On: Standby Nodes, then Primary
    • This is to prevent situations where the Primary WAL file is no longer suitable for the Standbys to join the cluster and lead to READ_ONLY_PRIMARY issues.
      • Reverting from a Cloud Director Appliance Deployment with Database HA to a Cloud Director Appliance Deployment Without Database HA
      • After restoring the VMware Cloud Director Appliance Cells from snapshot the UI is not accessible
• Before the power off/on order during the VMware Cloud Director upgrade, it is recommended to update the failover mode to manual via the VCD API. This can be changed back after the upgrade is complete. 
• If 2 or more upgrade should be performed in line, take the latest snapshots after last upgrade completed.

 

Environment

• VMware Cloud Director 10.6.x

Resolution

Open a Low - P4, Planned Activity Precheck Review case via Broadcom Support portal. 

The purpose of a Planned Activity Precheck Review case is to prepare in advance for the VMware Cloud Director upgrade.

If an issue is experienced during upgrade that causes a production down outage, Advanced Support customers should uplift the severity of the case to a Critical - P1.

This is not an exhaustive health check of the environment. If this is required, engage VMware by Broadcom Professional Services.

Additional Information

For further Planned Activity Precheck Review guidance, please see the Planned Activity Precheck Review - Broadcom Advanced Support KB which details the process and includes all VCF specific products this is offered for.