VMware Update Manager Database Reset for LCCM causes FLB deployment to fail
search cancel

VMware Update Manager Database Reset for LCCM causes FLB deployment to fail

book

Article ID: 430437

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

A reset of the VMware Update Manager Database results in the LCCM database losing the "vm_placement_policy" column and the following relations:

system_vms_cluster_solution_spec_vm_placement_policies
system_vms_cluster_solution_spec_vm_networks
system_vms_cluster_solution_spec_vm_datastores

This inconsistency in the LCCM database will critically impact LCCM operations.

Symptoms:

You see the following messages in the logs (/var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log):

2026-02-12T19:55:09.873Z info vmware-vum-server[690561] [Originator@6876 sub=com.vmware.vcIntegrity.lifecycle.lccm.SetSolutionTask] [SolutionsSvcSetTask, 71] Task:com.vmware.vcIntegrity.lifecycle.lccm.SetSolutionTask ID:52169e6c-5de3-9b70-881d-5bb4a08be3b9. Storing solution "solution-1" for cluster "domain-c1"
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] Execute result code: -1
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] SQL execution failed: INSERT INTO SYSTEM_VMS_CLUSTER_SOLUTION_SPEC_VM_PLACEMENT_POLICIES (CLUSTER_ID, SOLUTION_ID, VM_PLACEMENT_POLICY) VALUES (?, ?, ?)
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] Execution elapsed time: 2 ms
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] Statement diagnostic data from driver is 42703:0:1:ERROR: column "vm_placement_policy" of relation "system_vms_cluster_solution_spec_vm_placement_policies" does not exist;
--> Error while preparing parameters
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] Bind parameters:
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] [0]datatype: 7, size: 12, arraySize: 0
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] value = "domain-c9255"
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] [1]datatype: 7, size: 25, arraySize: 0
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] value = "foundation-loadbalancer-1"
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] [2]datatype: 7, size: 19, arraySize: 0
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] value = "VM_VM_ANTI_AFFINITY"
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [Vdb::IsRecoverableErrorCode] Unable to recover from 42703:1
2026-02-12T19:55:09.877Z error vmware-vum-server[690561] [Originator@6876 sub=Default] [VdbStatement] SQLError was thrown: "ODBC error: (42703) - ERROR: column "vm_placement_policy" of relation "system_vms_cluster_solution_spec_vm_placement_policies" does not exist;
--> Error while preparing parameters" is returned when executing SQL statement "INSERT INTO SYSTEM_VMS_CLUSTER_SOLUTION_SPEC_VM_PLACEMENT_POLICIES (CLUSTER_ID, SOLUTION_ID, VM_PLACEMENT_POLICY) VALUES (?, ?, ?)"
2026-02-12T19:55:09.878Z info vmware-vum-server[690561] [Originator@6876 sub=com.vmware.vcIntegrity.lifecycle.lccm.SetSolutionTask] [Task, 505] Task:com.vmware.vcIntegrity.lifecycle.lccm.SetSolutionTask ID:52169e6c-5de3-9b70-881d-5bb4a08be3b9. Finalizing Task
2026-02-12T19:55:09.878Z info vmware-vum-server[690561] [Originator@6876 sub=com.vmware.vcIntegrity.lifecycle.lccm.SetSolutionTask] [Task, 345] Task:com.vmware.vcIntegrity.lifecycle.lccm.SetSolutionTask ID:52169e6c-5de3-9b70-881d-5bb4a08be3b9. Task Finalization completed.
2026-02-12T19:55:09.878Z error vmware-vum-server[690561] [Originator@6876 sub=com.vmware.vcIntegrity.lifecycle.lccm.SetSolutionTask] [Task, 424] Task:com.vmware.vcIntegrity.lifecycle.lccm.SetSolutionTask ID:52169e6c-5de3-9b70-881d-5bb4a08be3b9. Task Failed. Error: Error:
-->    com.vmware.vapi.std.errors.error
-->    com.vmware.vcIntegrity.lifecycle.TaskError.UnknownError<Task failed with Unknown Error: Unknown Exception>
--> Messages:
-->

You see the following messages in the postgres log (/var/log/vmware/vpostgres/postgresql.log):

2026-01-14 15:07:24.942 UTC 6967b126.369f 0 VCDB postgres [local] 13983 320NOTICE:  relation "system_vms_cluster_solution_spec_vm_placement_policies" already exists, skipping
2026-01-14 15:07:24.942 UTC 6967b126.369f 0 VCDB postgres [local] 13983 321NOTICE:  relation "system_vms_cluster_solution_spec_vm_networks" already exists, skipping
2026-01-14 15:07:24.942 UTC 6967b126.369f 0 VCDB postgres [local] 13983 322NOTICE:  relation "system_vms_cluster_solution_spec_vm_datastores" already exists, skipping
...
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 4ERROR:  relation "system_vms_cluster_solution_spec_vm_placement_policies" does not exist
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 5STATEMENT:  ALTER TABLE system_vms_cluster_solution_spec_vm_placement_policies ADD COLUMN IF NOT EXISTS
VM_PLACEMENT_POLICY lccm_vm_placement_policy NOT NULL DEFAULT 'VM_VM_ANTI_AFFINITY';
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 6ERROR:  relation "system_vms_cluster_solution_spec_vm_placement_policies" does not exist
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 7STATEMENT:  ALTER TABLE system_vms_cluster_solution_spec_vm_placement_policies
ALTER COLUMN VM_PLACEMENT_POLICY DROP DEFAULT;
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 8ERROR:  relation "system_vms_cluster_solution_spec_vm_placement_policies" does not exist
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 9STATEMENT:  ALTER TABLE ONLY system_vms_cluster_solution_spec_vm_placement_policies
ADD CONSTRAINT system_vms_cluster_solution_spec_vm_placement_policies_pkey
PRIMARY KEY (cluster_id, solution_id, vm_placement_policy);
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 10ERROR:  relation "system_vms_cluster_solution_spec_vm_placement_policies" does not exist
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 11STATEMENT:  ALTER TABLE ONLY system_vms_cluster_solution_spec_vm_placement_policies
ADD CONSTRAINT system_vms_cluster_solution_spec_vm_cluster_id_solution_id_fkey
FOREIGN KEY (cluster_id, solution_id) REFERENCES system_vms_cluster_solution_spec(cluster_id, solution_id)
ON DELETE CASCADE;
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 12ERROR:  relation "system_vms_cluster_solution_spec_vm_networks" does not exist
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 13STATEMENT:  ALTER TABLE ONLY system_vms_cluster_solution_spec_vm_networks
ADD CONSTRAINT system_vms_cluster_solution_spec_v_cluster_id_solution_id_fkey1
FOREIGN KEY (cluster_id, solution_id) REFERENCES system_vms_cluster_solution_spec(cluster_id, solution_id)
ON DELETE CASCADE;
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 14ERROR:  relation "system_vms_cluster_solution_spec_vm_datastores" does not exist
2026-01-29 14:48:50.247 UTC 697b7352.bb24 0 postgres postgres [local] 47908 15STATEMENT:  ALTER TABLE ONLY system_vms_cluster_solution_spec_vm_datastores
ADD CONSTRAINT system_vms_cluster_solution_spec_v_cluster_id_solution_id_fkey2
FOREIGN KEY (cluster_id, solution_id) REFERENCES system_vms_cluster_solution_spec(cluster_id, solution_id)
ON DELETE CASCADE;
...
2026-02-12 19:55:09.875 UTC 698e1a76.6981e 7775420 VCDB vumuser [local] 432158 118ERROR:  column "vm_placement_policy" of relation "system_vms_cluster_solution_spec_vm_placement_policies" does not exist at character 94
2026-02-12 19:55:09.875 UTC 698e1a76.6981e 7775420 VCDB vumuser [local] 432158 119STATEMENT:  INSERT INTO SYSTEM_VMS_CLUSTER_SOLUTION_SPEC_VM_PLACEMENT_POLICIES (CLUSTER_ID, SOLUTION_ID, VM_PLACEMENT_POLICY) VALUES ($1, $2, $3)

Environment

VC 9.0.1.0

Resolution

This issue is fixed in 9.1.

Workaround for 9.0 is as follows.  Note that this procedure will correct for two possible scenarios, necessitating running the script more than once.

1.  Take offline snapshots of the vCenter, or ensure you have proper backups before continuing.

2.  Execute the attached script on the VC using the following command:

psql -U postgres VCDB -f script.sql

3.  The expected output upon successful execution is:

SET
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE

4.  Run the following commands on the VC:

pg_dump -U postgres VCDB --schema-only > /tmp/dump-before.sql

python /usr/lib/vmware-updatemgr/bin/updatemgr-utility.py reset-db

pg_dump -U postgres VCDB --schema-only > /tmp/dump-after-reset.sql

psql -U postgres VCDB -f script.sql

pg_dump -U postgres VCDB --schema-only > /tmp/dump-after-script.sql

diff -du /tmp/dump-before.sql /tmp/dump-after-script.sql

The database schema comparison before and after the script should show no differences.

Additional Information

Attachment: script.sql

File Hashes:

SHA256 - 1E24C06123931C8995F38241FD89ED1BB88DD34B6398E7211D1BE90E0F8487A9

MD5 - 9646D939F306A3118D81CC479F3089A5

Attachments

script.sql get_app
Powered by Wolken Software