Reducing the size of the vCenter Server database when the rollup scripts take a long time to run

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:
When the rollup jobs are running very slowly as a result of the vCenter Server database size, you may experience these symptoms:

• The vCenter Server database is very large (50 GB or more) and the row count in the VPX_HIST_STAT tables is very high (800 million lines or more).
• Rollup scripts are running very slowly or may not complete successfully.
• Performance Charts may not display recent data.
• Gaps are appearing in the Performance Charts.
• When this issue is encountered, you may see this error message in the Windows Event Viewer on vCenter Server:

Stats insertion failed for entity esxhost.domain.com due to ODBC error

Environment

VMware vCenter Server 4.1.x
VMware vCenter Server 5.1.x
VMware VirtualCenter 2.5.x
VMware vCenter Server 4.0.x
VMware vCenter Server 5.0.x
VMware vCenter Server 5.5.x

Cause

This issue occurs due to the stats collection level setting.

Verify the current stats collection level, and if this is set to a collection level higher than 2, then this should be set back to a lower stats collection level of 1.

Resolution

If the vCenter Server database is very large (50 GB or more) and the row count in the VPX_HIST_STAT# table is very high (800 million lines or more), the database rollup scripts may have difficulty handling the amount of data. Rollup scripts run faster if you reduce the size of the vCenter Server database.

Before you reduce the size of the vCenter Server database, you may want to defragment the indexes on your Microsoft SQL database server. 

Truncating all performance data from vCenter Server 5.x and 6.x

For truncating data in vCenter Server and vCenter Server Appliance 5.x and 6.x, see Delete old tasks, events and statistics data in vCenter Server 5.x and 6.x (2110031)

Reducing the size of the vCenter Server database 2.x

To reduce the size of the vCenter Server database:

Warning: This procedure erases all historical data. If you want to retain some historical performance data instead of deleting all of it, see Delete old tasks, events and statistics data in vCenter Server 5.x and 6.x (2110031)
 

1. Ensure that you have a good backup of the vCenter Server database. Do not skip this step.
2. Shut down the VMware VirtualCenter Server service and any other services (such as VMware VDI or VMware Lab Manager) that use the database. For more information, see Stopping, starting, or restarting vCenter services (1003895).
   
   Note: Again, ensure that you have a recent backup of the vCenter Server database before continuing. Do not skip this step.
    
3. Truncate the VPX_HIST_STAT1 table and corresponding VPX_SAMPLE_TIME1 table. After completing the truncate, verify if the rollup jobs are now completing successfully. For more information, see step 4.
   
   Warning: The truncate function is destructive. VMware highly recommends that a Database Administrator perform this step.
   
   To truncate the tables, execute these SQL statements:
   
   truncate table VPX_HIST_STAT1;
   
   Note: In vCenter Server 5.1and 5.5, the table name is VPX_HIST_STAT1_n.
   
   truncate table VPX_SAMPLE_TIME1;
   
   If the rollup jobs do not complete successfully, it may be necessary to truncate the VPX_HIST_STAT2 table, and the corresponding table, VPX_SAMPLE_TIME2. Continue truncating the VPX_HIST_STAT[1-4] tables and corresponding VPX_SAMPLE_TIME[1-4] tables until the rollup jobs are completing successfully.
   
   Note: These additional tables in vCenter Server 4.1 and 5.0 can also be truncated to further reduce the vCenter Server database size. The VPX_TEMPTABLEx tables are a staging/cache area for the performance data before they are processed and moved to VPX_HIST_STAT1.
   
   truncate table VPX_TEMPTABLE0;
truncate table VPX_TEMPTABLE1;
truncate table VPX_TEMPTABLE2;
    
4. To run the rollup scripts perform these steps:
    
   1. Using SQL Management Studio, connect to the SQL database for vCenter Server.
   2. Navigate to SQL Server Agent > Jobs.
   3. Select the individual rollup jobs, right-click and select Start Job at Step.
       
5. Start the VMware VirtualCenter Server service and any other service service that you stopped in step 2.