Updating/Patching vCenter Server from 8.0U1 to 8.0U2 or later releases fails with "ERROR: P0001,Failed to add primary key pk_vpx_dual on table vc.vpx_dual"
search cancel

Updating/Patching vCenter Server from 8.0U1 to 8.0U2 or later releases fails with "ERROR: P0001,Failed to add primary key pk_vpx_dual on table vc.vpx_dual"

book

Article ID: 345087

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Patching vCenter Server from 8.0U1 to 8.0U2 or later releases  fails with the error: vcdb.ndu.expand.generic.error

  • Upon checking var/log/vmware/applmgmt/PatchRunner.log we may notice similar entries:
psql.bin:/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/payload/components-script/vcdb/ndu/vcdb_expand.sql:1434: NOTICE:  l_alter_col_crt_idx: create unique index pk_vpx_dual on vc.vpx_dual(dummy)
psql.bin:/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/payload/components-script/vcdb/ndu/vcdb_expand.sql:1434: ERROR:  P0001,Failed to add primary key pk_vpx_dual on table vc.vpx_dual - sqlstate: 23505, sqlerrm: could
 not create unique index "pk_vpx_dual"
CONTEXT:  PL/pgSQL function vcdb_ndu_cln_processor(text[]) line 20 at RAISE
YYYY-MM-DDTHH:MM:SSZ vcdb:Patch ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'vcdb:Patch' failed.
Traceback (most recent call last):
  File "/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/py/vmware_b2b/patching/executor/hook_executor.py", line 74, in executeHook
    executionResult = systemExtension(args)
  File "/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/libs/sdk/extensions.py", line 106, in __call__
    result = self.extension(*args)
  File "/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/libs/sdk/extensions.py", line 123, in _func
    return func(*args)
  File "/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/payload/components-script/vcdb/__init__.py", line 188, in patch
    _run_expand()
  File "/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/payload/components-script/vcdb/__init__.py", line 120, in _run_expand
    raise UserError(cause=cause)
patch_errors.UserError: Failed to extend the database state.
YYYY-MM-DDTHH:MM:SSZ ERROR vmware_b2b.patching.phases.patcher Patch hook Patch got ComponentWrapperError.
Traceback (most recent call last):
  File "/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 208, in patch
    _patchComponents(ctx, userData, statusAggregator.reportingQueue)
  File "/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 90, in _patchComponents
    executeComponentHook(Hook.Patch, ctx, c, userData, reportingQueue)
  File "/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/py/vmware_b2b/patching/executor/execution_facade.py", line 97, in executeComponentHook
    result = executeHook(c.patchScript, hook, args,
  File "/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/py/vmware_b2b/patching/executor/execution_facade.py", line 53, in executeHook
    result = executor.executeHook(scriptFile, hook, args, reportQueue, reportIdentifier)
  File "/storage/updatemgr/software-updatees3zxzat/stage/scripts/patches/py/vmware_b2b/patching/executor/hook_executor_process.py", line 119, in executeHook
    raise ex
patch_errors.ComponentError
YYYY-MM-DDTHH:MM:SSZ WARNING root stopping status aggregation...
YYYY-MM-DDTHH:MM:SSZ ERROR __main__ Patch vCSA failed
  • While checking var/log/vmware/applmgmt/upgrade_hook_PatchHook we may notice similar entries:
{
    "progress": ##,
    "progress_message": {
        "detail": [
            {
                "id": "vcdb.ndu.expand.generic.error",
                "translatable": "Failed to extend the database state.",
                "localized": "Failed to extend the database state."
            }
        ],
        "componentKey": "vcdb:Patch",
        "problemId": null,
        "resolution": null
    },
    "status": "error",
    "info": [],
    "warning": [],
    "question": null,
    "error": {
        "detail": [
            {
                "id": "vcdb.ndu.expand.generic.error",
                "translatable": "Failed to extend the database state.",
                "localized": "Failed to extend the database state."
            }
        ],
        "componentKey": "vcdb:Patch",
        "problemId": null,
        "resolution": null
    },
    "start_time": "YYYY-MM-DDTHH:MM:SSZ",
    "end_time": "YYYY-MM-DDTHH:MM:SSZ"
}





Environment

VMware vCenter Server 8.0.1.x

Cause

The primary key creation failed because the table vc.vpx_dual contains duplicate rows with identical values. SQL tables with primary keys must ensure that each row has a unique value. During the upgrade process, an additional row was unintentionally inserted into the table, causing the conflict and preventing the creation of the primary key.

Resolution

Steps to Resolve the issue:

 

  • Revert to the snapshot taken before updating the vCenter.

  • Take a new snapshot, as interaction with the vCenter Server database (VCDB) will occur.

  • Open an SSH session to the vCenter VM and execute the following commands in the order specified below:

    • Stop the necessary services

      service-control --stop vmware-vpxd && service-control --stop vmware-content-library

      This stops the vpxd and content library services. The vCenter Server UI will be inaccessible until the services are started.

    • View existing entries in the vpx_ext_upgrade table

      /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres -c "\x" -c "select * from vpx_ext_upgrade;"

    • Delete existing entries from the vpx_ext_upgrade table

      /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres -c "delete from vpx_ext_upgrade;"

    • Verify that the vpx_ext_upgrade table is empty

      /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres -c "\x" -c "select * from vpx_ext_upgrade;"

    • Insert a new value into the vpx_ext_upgrade table

      /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres -c "insert into vpx_ext_upgrade (status) values (0);"

    • Verify the table has the correct value (only one entry)

      /opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres -c "\x" -c "select * from vpx_ext_upgrade;"

    • Restart the necessary services

      service-control --start vmware-vpxd && service-control --start vmware-content-library

  • Retry the vCenter update workflow after taking a new snapshot of the vCenter Server instance.

 

 

Note: Before making any changes to a vCenter Server database, ensure to create an offline snapshot of the entire vCenter Server instance. If the vCenter is part of an Enhanced Linked Mode (ELM) setup. For a standalone vCenter, ensure that a snapshot is taken without memory.

Note:  For any clarifications or assistance, please contact Broadcom Support and refer to this KB article.

Additional Information

As the required edits involve changes to the vCenter Server's database, which may be potentially destructive, it is advisable for a Broadcom Support Engineer to perform this task, or for the task to be carried out while a Broadcom Support Engineer is present on the call.

Powered by Wolken Software