vCenter upgrade from 8.0 U3h to 9.0.1 fails with an error "Encountered an internal error. see /var/log/firstboot/vpostgres-firstboot.py_#####_stderr.log"
search cancel

vCenter upgrade from 8.0 U3h to 9.0.1 fails with an error "Encountered an internal error. see /var/log/firstboot/vpostgres-firstboot.py_#####_stderr.log"

book

Article ID: 423649

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

  1. Upgrading from vCenter Server 8.0 Update 3h/3i (versions 8.0.3.00700 and newer) to vCenter Server 9.0.0 or 9.0.1 fails during Stage 2 (Migration).
    •  Stage 1 (Appliance Deployment) completes successfully.
    • Stage 2 fails during the vPostgres firstboot sequence.
    • The following error is recorded in the target vCenter log file 
      • Encountered an internal error. see /var/log/firstboot/vpostgres-firstboot.py_#####_stderr.log
        Resolution
        This is an unrecoverable error, please retry install. If you encounter this error again, please search for these symptoms in the VMware Knowledge Base for any known issues and possible resolutions. If none can be found, collect a support bundle and open a support request.
         
        A problem has occurred. The source vCenter Server might have been Powered Off during this process.
    • The following error can be found in the logs of the failed target vCenter:
      • var/log/firstboot/failed:

        Failed to start services. Firstboot Error.

        fbInstall.json
        {
            "progress": 12,
            "progress_message": {
                "id": "install.ciscommon.component.starting",
                "translatable": "Starting %(0)s...",
                "args": [
                    "VMware Postgres"
                ],
                "localized": "Starting VMware Postgres..."
            },
            "status": "error",
            "info": [],
            "warning": [],
            "question": null,
            "error": {
                "detail": [
                    {
                        "id": "install.ciscommon.internal.error",
                        "translatable": "Encountered an internal error.\n\n%(0)s",
                        "args": [
                            "see /var/log/firstboot/vpostgres-firstboot.py_13885_stderr.log"
                        ],
                        "localized": "Encountered an internal error.\n\nsee /var/log/firstboot/vpostgres-firstboot.py_#####_stderr.log"
                    }
                ],
                "componentKey": "vpostgres",
                "problemId": null,
                "resolution": {
                    "id": "install.default.failure.resolution",
                    "translatable": "This is an unrecoverable error, please retry install. If you encounter this error again, please search for these symptoms in the VMware Knowledge Base for any known issues and possible resolutions. If none can be found, collect a support bundle and open a support request.",
                    "localized": "This is an unrecoverable error, please retry install. If you encounter this error again, please search for these symptoms in the VMware Knowledge Base for any known issues and possible resolutions. If none can be found, collect a support bundle and open a support request."
                }
            },
            "start_time": "yyyy-mm-ddT00:00",
            "end_time": "yyyy-mm-ddT00:00"
        }


        var/log/firstboot/firstbootStatus.json:

        {
            "totalSteps": 49,
            "stepsStarted": 7,
            "stepsCompleted": 7,
            "finalStatus": "failure",
            "runTime": [
                {
                    "visl-support-firstboot": "0:00:03"
                },
                {
                    "vdtc_firstboot": "0:00:00"
                },
                {
                    "vmafd-firstboot": "0:00:24"
                },
                {
                    "vmon-firstboot": "0:00:01"
                },
                {
                    "envoy_hgw_firstboot": "0:00:01"
                },
                {
                    "rhttpproxy_firstboot": "0:00:01"
                },
                {
                    "vpostgres-firstboot": "0:00:04"
                }
            ],
            "failedSteps": "vpostgres-firstboot",  <==== Failed service!!!
            "stepsCompletedList": "visl-support-firstboot,vdtc_firstboot,vmafd-firstboot,vmon-firstboot,envoy_hgw_firstboot,rhttpproxy_firstboot,vpostgres-firstboot"
        }

         

    • Error log for failed vpostgres service:
      • var/log/firstboot/vpostgres-firstboot.py_#####_stderr.log
      • yyyy-mm-ddT00:00  Upgrade import failed due to error Error while deploying roles: psql.bin:/storage/seat/cis-export-folder/vcdb/VCDB_roles.dump:5: error: invalid command \restrict
        yyyy-mm-ddT00:00  Upgrade import step failed
        yyyy-mm-ddT00:00  vPostgres firstboot(action=firstboot) failed

Environment

vCenter Server  Appliance 8.0U3h

vCenter server Appliance 8.0.3i

 

Cause

This issue is classified as a back-in-time upgrade, which is currently unsupported for the following reasons:

  • Release Timing: vCenter Server versions 9.0.0 and 9.0.1 were released prior to the release of vCenter Server 8.0 Update 3h and 8.0.3i.
  • Interoperability: Per the VMware Interoperability Matrix, a migration path from a newer build of an older major version to an older build of a newer major version is not permitted.

 

Resolution

There is currently no workaround for this failure. Please follow these recommendations:

  • Defer Upgrade: Upgrading from vCenter 8.0 Update 3h/3i to vCenter 9.0.0 or 9.0.1 is not supported.
  • Future Compatibility: Support for this migration path will only be available after a vCenter 9.x patch, maintenance, or minor version release that post-dates the release of vCenter 8.0.3i.
  • VMware Cloud Foundation (VCF) Planning: If you intend to upgrade to VCF 9.x before January 2026, do not apply the vCenter Server 8.0 Update 3h or 8.0.3i updates to your current environment.
  • Recovery: If an upgrade has already failed, power off the new 9.0.x appliance and restore the source vCenter 8.0.3i from a pre-upgrade snapshot or backup.

Additional Information

Product Interoperability Matrix for upgrade