Investigating the health of a vCenter Server database
search cancel

Investigating the health of a vCenter Server database

book

Article ID: 306738

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

For troubleshooting purposes, it may be necessary to verify the health of the vCenter Server database server. This article provides steps to eliminate the common causes for problems with a database server, including full disk space, transaction log maintenance, index fragmentation, and other common database problems.

Symptoms:
  • vCenter Server does not start.
  • vCenter Server is slow to respond.
  • vCenter Server fails after an indefinite amount of time.
  • When connected to a Microsoft SQL database, the vpxd.log file of vCenter Server contains an entry similar to:

    ODBC error: (42000) - [Microsoft][SQL Native Client][SQL Server]Could not allocate space for object <object> in database '<database>' because the '<filegroup>' filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup." is returned when executing SQL statement "<sql statement>".
     
  • When connected to an Oracle database, the vCenter Server vpxd.log contains an entry similar to:

    ODBC error: (HY000) - [Oracle][ODBC][Ora]ORA-01653: unable to extend table by <size> in tablespace
    is returned when executing SQL statement <SQL Statement>.
     
  • You see this error when attempting to connect to the SQL database server:

    ODBC error: (08S01) - [Microsoft][SQL Native Client]Named Pipes Provider: No process is on the other end of the pipe.
     
  • The VirtualCenter Server service cannot authenticate with the database.
  • You cannot connect to the database repository through SQL Server or Oracle.
  • The vCenter Server database is corrupt.
Note: The vpxd.log files are located in %ALLUSERSPROFILE%\Application Data\VMware\VMware VirtualCenter\Logs.
</object>

Environment

VMware VirtualCenter 2.5.x
VMware vCenter Server 5.5.x
VMware vCenter Server 5.0.x
VMware vCenter Server 4.1.x
VMware vCenter Server 6.0.x
VMware vCenter Server 5.1.x
VMware VirtualCenter 2.0.x
VMware vCenter Server 4.0.x
VMware vCenter Server 6.7.x
VMware vCenter Server 6.5.x

Resolution

Validate that each troubleshooting step below is true for your environment. The steps and sections are ordered in the most appropriate sequence to isolate the issue and identify the proper resolution. Do not skip a step or a section.

The article is divided into several sections to allow for targeted troubleshooting depending on the symptoms. The sections are: VMware recommends that you start with the General Troubleshooting section to verify common configuration problems which might impact the other sections of the article.

Note: If you perform a corrective action in any of these steps, restart the VMware VirtualCenter Server service to validate the change. For more information, see Stopping, starting or restarting the vCenter server service (1003895).
 

General Troubleshooting

To validate the basic configuration of the vCenter Server database:
  1. Verify that there is adequate disk space available on the volume that is storing the database files to ensure correct operation of the database. If there is not adequate space available on the physical volume that stores the database files, free up disk space.
     
  2. If you are using Microsoft SQL Server, verify that the transaction logs for the vCenter Server database are healthy and are backed up on a regular basis. For more information, see https://docs.microsoft.com/en-us/sql/relational-databases/logs/the-transaction-log-sql-server?view=sql-server-2017                                                                                                                    
  3. If you are using Oracle, verify that there is adequate space available in the tablespace for growth of the database. For more information, see Extending the tablespace on an Oracle database (1003982).
     
  4. Validate the authentication to the vCenter Server database. The vCenter Server service may not be able to authenticate with the database if:
  5. You may not be able to connect to the database repository through SQL Server or Oracle. If vCenter Server disconnects from the SQL or Oracle database, the database server can maintain a lock on the vCenter Server database. You may need to restart the SQL Service or Oracle instance to free the lock.
 

Troubleshooting growth of the vCenter Server database

There are many circumstances when the vCenter Server database grows excessively. Growth of the vCenter Server database is due to data being collected and stored in the database. This data falls into three categories:
  • Performance data
  • Log of tasks that were performed
  • Log of events that occurred
Performance data, in most cases, causes the most growth of the database. When troubleshooting excessive growth of the database, start by examining where the growth is occurring. You can use this to determine how to troubleshoot the issue.

It may be necessary to truncate the information in the database. However, this should never be the first step. Diagnose the issue completely, first, if at all possible. In many cases, the amount of data is too large to process in the given amount of time. For example, if there is more data that is being collected than can be processed in the same period of time, or if it takes several months to process the data, it may be better to simply truncate the data. In these cases, it is dependent on your business requirements as well as the cause of the growth.  

Troubleshooting the database when vCenter Server is slow

vCenter Server is a database application and is therefore subject to the same issues as similar applications. These are common issues that may occur:
  • Slow response when retrieving performance data, tasks, events, or inventory information.
  • Only real time performance data is available.
  • You see the error:

    Performance data is currently not available for this entity
To troubleshoot these issues:
  1. If you are running a Microsoft SQL database, perform a defragmentation of the performance data indexes on the vCenter Server database. This ensures that the data can be accessed as quickly as possible. For more information, see Defragmenting VMware VirtualCenter or vCenter Server performance data indexes on a Microsoft SQL database (1003990).
     
  2. Validate whether the vCenter Server database server has adequate resources available. Consult the vSphere Installation Guide for your version of vCenter Server, along with the vendor documentation for the database. The resolution may vary depending on the load and the database being used.

    It may be beneficial to separate vCenter Server and the database if they are running on the same server and if there is a large amount of activity in the environment. Resource contention can cause a noticeable decrease in vCenter Server performance.

    These steps may also help improve database performance:
     
    1. Separate the database and transaction / redo logs to different physical drives. This helps avoid contention on the disks because it increases the number of write operations that can be done at the same time.
    2. Avoid the use of statistics collection levels above level 2, unless troubleshooting a specific problem. The amount of data collected at the higher levels can cause both slow performance of the database and excessive growth.
    3. Avoid running other applications on the server which is hosting vCenter Server. This can appear to be a database problem. However, it can be a factor in slow performance of the database due to resource contention on the vCenter Server system.
    4. Do not use SQL Server Express. It is only designed to run a limited configuration and quickly hits the 4 GB database size limit. This causes a decrease in performance, which can then lead to failure of the service.
For more information on performance recommendations, see the VMware vCenter Server Performance and Best Practices document.
 

Reinitializing the vCenter Server database

A reinitialization of the vCenter Server database resets it to the default configuration, as if the vCenter Server was newly installed. Resetting the database may be required in these situations:
  • A rebuild of vCenter Server is required
  • Data corruption is suspected
  • Due to a request from VMware Support
To reset the database configuration:

Note: This procedure modifies the database. Ensure to take a backup of the database before proceeding.
 
  1. Stop the VMware VirtualCenter Server service. For more information, see Stopping, starting, or restarting vCenter Services (1003895).
     
  2. Go to Start > Run, type cmd, and click OK. The Command Prompt window opens.
     
  3. Navigate to the vCenter Server program files directory. By default directory is:

    C:\Program Files\VMware\Infrastructure\VirtualCenter\
     
  4. Run the command:

    vpxd.exe -b
     
  5. Start the VMware VirtualCenter Server service. For more information, see Stopping, starting, or restarting vCenter Services (1003895).

Note: If the issue still exists after attempting the steps in this article:
  1. Gather diagnostic information. For more information, see Collecting diagnostic information for VMware products (1008524).
     
  2. File a support request with VMware Technical Support and note this Knowledge Base article ID (1003979) in the problem description. For more information on filing a Support Request, see Filing a Support Request in Customer Connect (2006985).


Additional Information

Note: If you encounter this error in the vpxd.log when attempting to connect to the SQL database server:

ODBC error: (08S01) - [Microsoft][SQL Native Client]Named Pipes Provider: No process is on the other end of the pipe

And a corresponding Event ID 17187 is recorded in the Windows Event log on the SQL Server at approximately the same time, the Event Log code indicates:

SQL Server is not ready to accept new client connections; the connection has been closed

This suggests a possible delay is necessary between the time when the SQL service starts and when the attempt is made to start the VirtualCenter Server services.

Before performing additional troubleshooting, restart the SQL Server service, wait approximately 60 seconds, then start the VirtualCenter Server service. If it starts successfully, start the VMware Management Webservices service. For more information on preventing a race condition between the VirtualCenter Server service and the SQL database service, see VMware VirtualCenter Server service does not start automatically after reboot (1007669).


For translated versions of this article, see: Defragmenting VMware VirtualCenter or vCenter Server performance data indexes on a Microsoft SQL database
Purging old data from the database used by vCenter Server
Determining where growth is occurring in the vCenter Server database
Verifying jobs and stored procedures in VMware vCenter Server 4.1.x
Investigando el estado del servidor de bases de datos de VirtualCenter
Verifying vCenter Server 4.0 jobs and stored procedures
Verifying vCenter Server 2.5 jobs and stored procedures
vCenter Server データベースの健全性の調査
Investigação de integridade de um banco de dados do vCenter Server
调查 vCenter Server 数据库的运行状况

Powered by Wolken Software