vCenter Server fails during virtual machine snapshot operations

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• vCenter Server fails while a virtual machine snapshot operation is in progress.
• This issue occurs when the vCenter Server database is Microsoft SQL Server.
• The vpxd.log file contains an error similar to:
  
  An unrecoverable problem has occurred, stopping the VMware VirtualCenter service. Error: Error[VdbODBCError] (-1) "ODBC error: (23000) - [Microsoft][SQL Server Native Client 10.0][SQL Server]Violation of PRIMARY KEY constraint 'PK_FLE_FILE_INFO'. Cannot insert duplicate key in object 'dbo.VPX_VM_FLE_FILE_INFO'. The duplicate key value is (154315, 12)." is returned when executing SQL statement "INSERT INTO VPX_VM_FLE_FILE_INFO WITH (ROWLOCK) (VM_ID, KEY_VAL, NAME, FILE_SIZE, TYPE) VALUES (?, ?, ?, ?, ?)"
   
• This issue may be more prevalent when multiple virtual machine snapshot operations are happening at the same time.
• This issue may be more prevalent if the same virtual machine (or virtual machines) have snapshots taken frequently.
• If you restart vCenter Server, it functions normally, but then the issue re-occurs.

Environment

VMware vCenter Server 4.1.x
VMware vCenter Server 5.0.x
VMware vCenter Server 6.0.x

Resolution

This issue is resolved in vCenter Server 5.1. You can download the latest release, available at VMware Downloads.

 

To work around this issue, delete the primary key constraint and create a clustered index:

1. Connect to the vCenter Server database using SQL Management studio.
2. Take a backup of the existing vCenter Server database. For more information, see Backing up the vCenter Server database running on Microsoft SQL or SQL Express server (2012138).
3. Execute this SQL statement against the vCenter Server database:
   
   IF EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[dbo].[VPX_VM_FLE_FILE_INFO]') AND name = N'PK_FLE_FILE_INFO')
   ALTER TABLE [dbo].[VPX_VM_FLE_FILE_INFO] DROP CONSTRAINT [PK_FLE_FILE_INFO]
   GO
    
4. Execute this SQL statement against the vCenter Server database:
   
   CREATE CLUSTERED INDEX [VPX_VM_FLE_FILE_INFO_ID_KEY_VAL] ON [dbo].[VPX_VM_FLE_FILE_INFO]
   (
   [VM_ID] ASC,
   [KEY_VAL] ASC
   )
   GO

Note: If you implement this workaround, duplicate information related to virtual machine snapshots may exist in the vCenter Server database for a very short period of time. This causes inaccurate values to be displayed for Space Used and Snapshot Space under the Storage Views in the vSphere Client until the snapshot is deleted.

Additional Information

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box.
Backing up the vCenter Server database running on Microsoft SQL or SQL Express server