VPXD crashes due to ERROR: duplicate key value violates unique constraint "pk_n_vm_config_info"
search cancel

VPXD crashes due to ERROR: duplicate key value violates unique constraint "pk_n_vm_config_info"

book

Article ID: 316427

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

To resolve the issue related to the DB crash due to a duplicate value in VCDB

Symptoms -

  • VPXD crashes with a VCDB error
  • vpxd.log entries -
yyyy-mm-dd error vpxd[44699] [Originator@6876 sub=Default opID=HB-host-226@232599-3eddce8f] An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Error: Error[VdbODBCError] (-1) "ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_n_vm_config_info"

--> DETAIL: Key (id)=(xxxxxx) already exists.;

--> Error while executing the query" is returned when executing SQL statement "INSERT INTO VPX_NON_ORM_VM_CONFIG_INFO (ID,CHANGE_VERSION,CHANGE_TRACKING_ENABLED,CPU_HOT_ADD_ENABLED,CPU_HOT_REMOVE_ENABLED,MEM_HOT_ADD_ENABLED,HARDWARE_NUM,HARDWARE_MEMORY,HARDWARE_CORES,VIRTUAL_ICH7M_PRESENT,VIRTUAL_SMC_PRESENT,TOOLS_BEFORE_GUEST_STANDBY_FLG,TOOLS_BEFORE_GUESTSHUTDOWN_FLG,TOOLS_TOOLS_UPGRADE_POLICY,TOOLS_AFTER_RESUME_FLG,TOOLS_AFTER_POWER_ON_FLG,TOOLS_SYNC_TIME_WITH_HOST_FLG,TOOLS_TOOLS_VERSION,TOOLS_LASTINSTALL_COUNTER,GUEST_FULL_NAME,INSTANCE_UUID,UUID,ANNOTATION,VERSION,TEMPLATE_FLG,M"

yyyy-mm-dd panic vpxd[44699] [Originator@6876 sub=Default opID=HB-host-226@232599-3eddce8f]
  • postgresql-*.log entries -
yyyy-mm-dd 63a869b6.b616 777020269 VCDB vc ERROR:  duplicate key value violates unique constraint "pk_n_vm_config_info"
yyyy-mm-dd 63a869b6.b616 777020269 VCDB vc DETAIL:  Key (id)=(xxxxxx) already exists.
yyyy-mm-dd 63a869b6.b616 777020269 VCDB vc STATEMENT:  INSERT INTO VPX_NON_ORM_VM_CONFIG_INFO (ID,CHANGE_VERSION,CHANGE_TRACKING_ENABLED,CPU_HOT_ADD_ENABLED,CPU_HOT_REMOVE_ENABLED,ME
M_HOT_ADD_ENABLED,HARDWARE_NUM,HARDWARE_MEMORY,HARDWARE_CORES,VIRTUAL_ICH7M_PRESENT,VIRTUAL_SMC_PRESENT,TOOLS_BEFORE_GUEST_STANDBY_FLG,TOOLS_BEFORE_GUESTSHUTDOWN_FLG,TOOLS_TOOLS_UPGRADE_POLICY,TOOLS_
AFTER_RESUME_FLG,TOOLS_AFTER_POWER_ON_FLG,TOOLS_SYNC_TIME_WITH_HOST_FLG,TOOLS_TOOLS_VERSION,TOOLS_LASTINSTALL_COUNTER,GUEST_FULL_NAME,INSTANCE_UUID,UUID,ANNOTATION,VERSION,TEMPLATE_FLG,MEM_ALLOCATION
_OVERHEAD_LIMIT,MEMORY_ALLOCATION_RESERVATION,MEMORY_ALLOCATION_LIMIT,MEM_ALLOCATION_SHARES_SHARES,MEM_ALLOCATION_SHARES_LEVEL,HOT_PLUG_MEMORY_LIMIT,FILES_LOG_DIRECTORY,FILES_SNAPSHOT_DIRECTORY,FILES
_VM_PATH_NAME,FILES_SUSPEND_DIRECTORY,ALTERNATE_GUEST_NAME,SWAP_PLACEMENT,LOCATION_ID,GUEST_ID,BOOT_OPTIONENTER_BIOSSETUP_FLG,BOOT_OPTION_BOOT_DELAY,BOOT_OPTION_RETRY_ENABLED_FLG,BOOT_OPTION_RETRY_DE
LAY,BOOT_OPTION_EFI_SB_ENABLED,BOOT_OPTION_NETWORK_PROTOCOL,CPU_ALLOCATION_RESERVATION,CPU_ALLOCATION_LIMIT,CPU_ALLOCATION_SHARES_SHARES,CPU_ALLOCATION_SHARES_LEVEL,DEF_PO_OPS_DEFAULT_RESET_TYPE,DEF_
PO_O_DEFAULT_SUSPEND_TYPE,DEF_POWER_OPS_SUSPEND_TYPE,DEF_POWER_OPS_STANDBY_ACTION,DEF_PO_O_DEF_POWER_OFF_TYPE,DEFAULT_POWER_OPS_RESET_TYPE,DEF_POWER_OPS_POWER_OFF_TYPE,H_PLUG_MEMORY_INCREMENT_SIZE,NA
ME,NPIV_TEMP_DISABLED,MODIFIED,DISK_UUID_ENABLED_FLG,SNAPSHOT_DISABLED_FLG,RECORD_REPLAY_ENABLED_FLG,FAULT_TOLERANCE_TYPE,RUN_WITH_DEBUG_INFO_FLG,VIRTUAL_MMU_USAGE,VIRTUAL_EXEC_USAGE,ENABLE_LOGGING_F
LG,SNAPSHOT_LOCKED_FLG,HT_SHARING,DISABLE_ACCELERATION_FLG,FL_SNAPSHOT_POWER_OFF_BEHAVIOR,FLAGS_MONITOR_TYPE,USE_TOE_FLG,FIRMWARE,MAX_MKS_CONNECTIONS,NESTED_HV_ENABLED,VIRTUAL_PMC_ENABLED,SCHED_HW_UP
GRADE_POLICY,SCHED_HW_UPGRADE_STATUS,LATENCY_LEVEL,GUEST_AUTOLOCK_ENABLED,MEM_RES_LOCKED_MAX,MESSAGE_BUS_TUNNEL_ENABLED,GUEST_INTEGRITY_INFO_ENABLED,MIGRATE_ENCRYPTION,VM_ID,CREATE_DATE,VVTD_ENABLED,
VBS_ENABLED,FT_ENCRYPTION_MODE,GUEST_MONITORING_MODE_INFO_ENABLED,TOOLS_SYNC_TIME_WITH_HOST_ALLOWED_FLG,SEV_ENABLED) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21
,$22,$23,$24,$25,$26,$27,$28,$29,$30,$31,$32,$33,$34,$35,$36,$37,$38,$39,$40,$41,$42,$43,$44,$45,$46,$47,$48,$49,$50,$51,$52,$53,$54,$55,$56,$57,$58,$59,$60,$61,$62,$63,$64,$65,$66,$67,$68,$69,$70,$7
1,$72,$73,$74,$75,$76,$77,$78,$79,$80,$81,$82,$83,$84,$85,$86,$87,$88,$89,$90,$91,$92,$93,$94)

Environment

  • vCenter Server 6.7
  • vCenter Server 7.0 
  • vCenter Server 8.0 

Cause

A duplicate value in the VCDB causes the VPXD service to crash every minute.

Resolution

  • After identifying the affected Key ID value from the vpxd and postgres service logs follow the steps below.
  • vpxd.log entries -
"ODBC error: (23505) - ERROR: duplicate key value violates unique constraint "pk_n_vm_config_info"

--> DETAIL: Key (id)=(xxxxxx) already exists.;

Workaround/Resolution -

  1. SSH to the vCSA via root
  2. Stop the VPXD service

service-control --stop vpxd

  1. Connect to the VCDB:

/opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres

  1. Run the following command to identify the VM ID associated with the stated Key ID:
select vm_id from vpx_vm_text where id=<Key_ID_in_vpxd/postgres_logs>;

Example:
select vm_id from vpx_vm_text where id=xxxxxx;

Output:

 vm_id
-------
 xxxx
(1 row)

  1. Delete the entries from the following tables:

delete from VPX_COMPUTE_RESOURCE_DAS_VM where VM_ID=<vm_id_from_step4>;
delete from VPX_COMPUTE_RESOURCE_DAS_VM where VM_ID=<vm_id_from_step4>;
delete from VPX_COMPUTE_RESOURCE_DRS_VM where VM_ID=<vm_id_from_step4>;
delete from VPX_COMPUTE_RESOURCE_ORC_VM where VM_ID=<vm_id_from_step4>;
delete from VPX_VM_SGXINFO where VM_ID=<vm_id_from_step4>;
delete from VPX_GUEST_DISK where VM_ID=<vm_id_from_step4>;
delete from VPX_VM_VIRTUAL_DEVICE where ID=<vm_id_from_step4>;
delete from VPX_VM_DS_SPACE where VM_ID=<vm_id_from_step4>;
delete from VPX_NON_ORM_VM_CONFIG_INFO where ID=<vm_id_from_step4>;
delete from VPX_NORM_VM_FLE_FILE_INFO where VM_ID=<vm_id_from_step4>;
delete from VPX_VDEVICE_BACKING_REL where VM_ID=<vm_id_from_step4>;
delete from VPX_VIRTUAL_DISK_IOFILTERS where VM_ID=<vm_id_from_step4>;
delete from VPX_VM_STATIC_OVERHEAD_MAP where VM_ID=<vm_id_from_step4>;
delete from VPX_VM_TEXT where VM_ID=<vm_id_from_step4>;
delete from VPX_VM where ID=<vm_id_from_step4>;
delete from VPX_ENTITY where ID=<vm_id_from_step4>;

Note: some of the delete commands may output '0 deleted', this is normal. Continue to proceed with the remaining delete commands.

  1. Disconnect from the VCDB
\q 
  1. Start the VPXD service
service-control --start vpxd

Note: If the above command fails to start vpxd, restart all services instead

service-control --stop --all; service-control --start --all
  1. Monitor the start of the VPXD service or all services in a separate SSH instance 

watch service-control --status all

Additional Information

Impact/Risks -

  • Following the resolution section of the KB requires making alterations to the vCenter Server database (VCDB), please ensure the appropriate snapshots and/or backups are in place before proceeding.

Powered by Wolken Software