VMware VirtualCenter Server service fails with the error: Violation of PRIMARY KEY constraint 'PK_VPX_GUEST_DISK'

Products

• VMware vCenter Server

Issue/Introduction

Overview

The vCenter Server database is case insensitive. This issue occurs when a Linux virtual machine presents two duplicate mount points that are distinguished by case to the vCenter Server database.

 

For example, the Linux guest sees the following as two separate mount points:

• /dev/mapper/mount1 on /opt/Folder1
• /dev/mapper/mount2 on /opt/folder1

To vCenter Server, however, the two mount points appear to have the same value. This violates primary key constraints and causes the VirtualCenter Server service to fail.

Note: Only Microsoft SQL exhibits this behavior. Oracle and IBM DB2 use case sensitive databases and therefore do not exhibit this behavior.

Symptoms:

• The VMware VirtualCenter Server service fails to start
• The VMware VirtualCenter Server service is up and running, but hosts report the warning:
  
  Cannot synchronize host. database temporarily unavailable or has network problems.
   
• In the vpxd log of vCenter Server, you see entries similar to:
  
  An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Check database connectivity before restarting. Error: Error[VdbODBCError] (-1) "ODBC error: (23000) - [Microsoft][SQL Native Client][SQL Server]Violation of PRIMARY KEY constraint 'PK_VPX_GUEST_DISK'. Cannot insert duplicate key in object 'dbo.VPX_GUEST_DISK'." is returned when executing SQL statement "INSERT INTO VPX_GUEST_DISK (VM_ID, PATH, CAPACITY, FREE_SPACE) VALUES (?, ?, ?, ?)"

Environment

VMware VirtualCenter 2.5.x
VMware vCenter Server 4.0.x
VMware vCenter Server 4.1.x

Resolution

The script attached to this article prevents the VMware VirtualCenter Server service from failing when this issue occurs.

To permanently resolve this issue:

1. Download 1020317_recreate_vpx_guest_disk.zip (attached to this article).
2. Extract the SQL script.
3. Stop the VMware VirtualCenter Server service.
4. Open up and run the recreate_vpx_guest_disk.sql file within the SQL Enterprise Manager Tool on the SQL Server.
5. Start the VMware VirtualCenter Server service.

Workaround:

If you cannot apply the resolution, workaround this issue by performing one of these options:

• Ensure that you use unique folder names for all mount points.
• Uninstall VMware Tools from the virtual machine that has multiple mount points. When VMware Tools is not available to send the partition information to vCenter Server, this issue does not occur.

However, before you can apply the workaround, you must determine which virtual machine is causing the problem. You may already know this information, but determining the virtual machine can be difficult in large environments.

To identify the virtual machine that is causing the failure:

1. Enable trivia logging on vCenter Server. For more information, see Enabling trivia logging in VirtualCenter (1001584).
2. When vCenter Server fails, open the vpxd.log file and identify the thread number that is causing the service to crash. For example:
   
   [<date> <time> 'App' 1234 error] An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Check database connectivity before restarting. Error: Error[VdbODBCError] (-1) "ODBC error: (23000) - [Microsoft][SQL Native Client][SQL Server]Violation of PRIMARY KEY constraint 'PK_VPX_GUEST_DISK'. Cannot insert duplicate key in object 'dbo.VPX_GUEST_DISK'." is returned when executing SQL statement "INSERT INTO VPX_GUEST_DISK (VM_ID, PATH, CAPACITY, FREE_SPACE) VALUES (?, ?, ?, ?)".</time>
   
   In this example, the thread number is 1234 .
    
3. Take the thread number and search backwards in the log until you see a set of log lines that look similar to:
   
   [<date><time> 'PropertyProvider' 1234 trivia] RecordOp 2: summary.guest.guestFullName, vm-123456
   </time>[<date><time> 'PropertyProvider' 1234 trivia] RecordOp 2: guest.guestFullName, vm-123456
   </time>[<date><time> 'PropertyProvider' 1234 trivia] RecordOp 2: guest.disk, vm-123456
   </time>[<date><time> 'PropertyProvider' 1234 trivia] RecordOp 2: guest.screen, vm-123456</time>
   
   This identifies the virtual machine that is causing the failure. In this example, the virtual machine is vm-123456 .
    
4. To determine the virtual machine name, login to the vCenter Server database server and run the following query against the vCenter Server database:
   
   SELECT NAME FROM VPX_ENTITY WHERE ID = '123456';
   
   where '123456' is the number of the virtual machine identified in Step 3. The query results return the virtual machine name.