VMware Cloud Director database upgrade fails at task 4 with "java.lang.IllegalArgumentException: state"

Products

• VMware Cloud Director

Issue/Introduction

When attempting to upgrade the VMware Cloud Director (VCD) database, the upgrade utility fails to at task 4 with the following error:

 java.lang.IllegalArgumentException: state

Subsequent attempts to rerun the upgrade utility abort immediately, returning the following output indicating an incomplete upgrade state:

Welcome to the VMware Cloud Director upgrade utility
Verify that you have a valid license key to use the version of the
VMware Cloud Director software to which you are upgrading.
This utility will apply several updates to the database. Please
ensure you have created a backup of your database prior to continuing.
Do you wish to upgrade the product now? [Y/N] Y
Examining database at URL: jdbc:postgresql://##.####.##:####/vcloud?socketTimeout=##gssl=true
A previous attempt to upgrade the database was incomplete. Please restore your database from a backed up copy and try again.

Environment

• VMware Cloud Director 10.6.x

Cause

This issue occurs because the VMware Cloud Director (VCD) cell appliances were manually Powered ON prior to executing the database upgrade. Powering ON the appliances automatically starts the vmware-vcd services, which interferes with the database schema upgrade process.

Resolution

The recommended resolution is to revert the environment to previous snapshots and execute the upgrade again.

1. Revert to Pre-Upgrade Snapshots.
2. Perform the Appliance Upgrade.
3. Before performing the database upgrade, you must ensure the cell service is completely shut down on all nodes. Log into each cell and verify the service status:

   service vmware-vcd status

4. If the service is still in an active/running state on any cell, forcefully stop it:

   service vmware-vcd stop

5. Once you have confirmed that the vmware-vcd service is completely stopped across all cells, log into the Primary cell and execute the database upgrade utility:

   /opt/vmware/vcloud-director/bin/upgrade

Additional Information

Upgrading to VMware Cloud Director 10.6