vCenter upgrade to 8.x fails with a firstboot error due to "RESTART value (XXXXXXXXXX) cannot be greater than MAXVALUE (4294965000)"

Products

• VMware vCenter Server

Issue/Introduction

• Attempts to upgrade vCenter Server Appliance to 8.0.x are failing with a firstboot error:

VMware VirtualCenter failed firstboot.

• When reviewing the firstboot logs, you can find the following errors
• In /var/log/firstboot/firstbootInfrastructure.log:

  XXXX-XX-XXTXX:XX:XX.203Z INFO firstbootInfrastructure Running firstboot script: ['/opt/vmware/bin/python', '/usr/lib/vmware-vpx/firstboot/vpxd_firstboot.py', '--action', 'firstboot', '--compkey', 'vpxd', '--errlog', '/var/log/firstboot/vpxd_firstboot.py_XXXX_stderr.log']
  XXXX-XX-XXTXX:XX:XX.602Z INFO firstbootInfrastructure [Failed] /usr/lib/vmware-vpx/firstboot/vpxd_firstboot.py is complete
  XXXX-XX-XXTXX:XX:XX.605Z WARNING firstbootInfrastructure Bug component info file does not exist
  XXXX-XX-XXTXX:XX:XX.605Z INFO firstbootInfrastructure Firstboot duration: 335 sec
  XXXX-XX-XXTXX:XX:XX.606Z WARNING firstbootInfrastructure stopping status aggregation...
  XXXX-XX-XXTXX:XX:XX.605Z INFO firstbootInfrastructure First boot is a failure
  XXXX-XX-XXTXX:XX:XX.606Z ERROR firstbootInfrastructure Installation of vCenter server failed with firstboot scripts

• In /var/log/firstboot/vpxd_firstboot.py_###_stdout.log:

  XXXX-XX-XXTXX:XX:XX.896Z  Getting value for install-parameter: db.dsn
  XXXX-XX-XXTXX:XX:XX.519Z  Failed to run inplace database upgrade - Error while executing ./Upgrade-v2017-to-v2018/postgresql/upgrade_PostgreSQL.sql, reason: Statement failure(rc=-1).
  ALTER SEQUENCE IF EXISTS VPX_EVENT_SEQ INCREMENT BY 1000 MINVALUE 1 MAXVALUE 4294965000 CYCLE;
  ^^^^^^^^^^
   1 [22023](1) ERROR: RESTART value (8588097201) cannot be greater than MAXVALUE (4294965000);
  Error while executing the query
  Error while executing ./Upgrade-v2017-to-v2018/postgresql/VCDB_views_PostgreSQL.sql:1345, reason: Statement failure(rc=-1).
  create or replace view VPXV_HOST_DATASTORE as
  select ENTITY_ID as HOST_ID,
          DS_ID, ACCESSIBLE, MOUNT_PATH, MOUNT_ID, MOUNT_MODE, MOUNTED, VMKNIC_NAME, VMKNIC_ACTIVE,
          CONNECTIONS
  from VPX_ENTITY, VPX_DS_ASSIGNMENT
  where ENTITY_ID = ID
     and TYPE_ID = 1
  ^^^^^^^^^^
   
  1 [42703](1) ERROR: column "vmknic_name" does not exist;
  Error while executing the query
  ERROR: Statement failure(rc=-1).
  create or replace view VPXV_HOST_DATASTORE as
  select ENTITY_ID as HOST_ID,
          DS_ID, ACCESSIBLE, MOUNT_PATH, MOUNT_ID, MOUNT_MODE, MOUNTED, VMKNIC_NAME, VMKNIC_ACTIVE,
          CONNECTIONS
  from VPX_ENTITY, VPX_DS_ASSIGNMENT
  where ENTITY_ID = ID
     and TYPE_ID = 1

Environment

VMware vCenter Server 8.0.x

Cause

This happens when vpx_event_seq contains values that are too high and needs to be reset.

Resolution

Before attempting the steps below, please make sure to create a fresh offline snapshot of the vCenter Server Appliance (VCSA) in powered off state. If the affected VCSA is part of an Enhanced Linked Mode replication setup, please be aware that offline snapshots of all other ELM nodes are required as well.

To fix the issue, open an SSH connection to the source VCSA, then:

1. Stop the vCenter Server service:

   # service-control --stop vpxd

2. Run the following command to regenerate vpx_event_seq:

   # /opt/vmware/vpostgres/current/bin/psql -U postgres VCDB -c "ALTER SEQUENCE VPX_EVENT_SEQ RESTART WITH 1" 

3. Clean-up the event tables in the vCenter DB by following the steps outlined in Delete old tasks, events and statistics data in vCenter Server 5.x, 6.x, 7.x and 8.x
4. Restart the vCenter Server service:

   # service-control --stop vpxd

5. Restart the upgrade process