vCenter Server Services hang on startup after upgrading to vCenter Server 5.1

Products

• VMware

Issue/Introduction

Symptoms:

• After upgrading to vCenter Server 5.1, the VMware VirtualCenter Server service displays as Starting and fails to complete the start of the service
• The VMware VirtualCenter Server service does not successfully stop or start.
• The final log in the vpxd.log file shows:
  
  CoreDump: Writing minidump
  
  
• A backtrace near the bottom of the vpxd.log file starts with:
  
  2012-09-13T11:53:46.802+02:00 [04468 info 'vmmoVm' opID=SWI-c5103928] [VmMo::SetComputeCompatibilityDirty] vm vm-5345 is marked dirty
  2012-09-13T11:53:46.786+02:00 [05112 warning 'win32vpxdOsLayer_win32' opID=SWI-56f04b9c] [VpxUnhandledException] Backtrace
  
  Or
  
  2012-10-05T00:25:38.080-04:00 [04032 info 'licenselicenseUtil' opID=SWI-2aee8353] Unable to decode license XXXXX-XXXXX-XXXXX-XXXXX-XXXXX with error 5 using path dir://C:\ProgramData\VMware\VMware VirtualCenter\licenses\site\VMware ESX Server\5.0\5.0.1.4
  warning 'win32vpxdOsLayer_win32' opID=SWI-78887cd0] [VpxUnhandledException] Backtrace
  --> backtrace[00] rip 000000018018977a
  --> backtrace[01] rip 0000000180100c98
  --> backtrace[02] rip 0000000180101fae
  --> backtrace[03] rip 0000000140a7387d
  --> backtrace[04] rip 0000000077519450
  
  
  
• Reconnecting hosts after upgrading to vCenter Server 5.1 results in the error:
  
  general error occurred: invalid encrypted data size

Cause

The issue is caused by a key length mismatch between the vCenter Server certificate and the encrypted host passwords stored in the vCenter Server database. Starting with vCenter Server 4.1 and continuing with vCenter 5.1, self-signed certificates are generated with 2048-bit key length. In vCenter Server 2.5-4.0, they were anywhere from 512-bit to 1024-bit.

Resolution

This issue is resolved in VMware vCenter Server 5.1.0a released October 25, 2012. For further information, see Resolved Issues section the VMware vCenter Server 5.1.0a Release Notes. To download this version, go to the VMware vSphere download site.

If you cannot upgrade, you can work around this issue by updating the encrypted host passwords in the database.

To update the encrypted host passwords:

1. Identify that the length of the encrypted password for any host does not meet the 2048-bit key length:
   
   For SQL Server:
   
   SQL query: SELECT ID, LEN(PASSWORD), PASSWORD FROM VPX_HOST;
   
   For Oracle:
   
   SQL query: SELECT ID, LENGTH(PASSWORD), PASSWORD FROM VPX_HOST;
   
   Password lengths of 173 represent encryption with a 1024-bit key, and lengths of 345 represent encryption with a 2048-bit key.
   
   If any lengths are found to be less than 345, continue to step 2.
   
   If all lengths are 0 or 345, no further action is required.
   
   
2. Clear out the stored encrypted passwords.
   
   SQL query: UPDATE VPX_HOST SET PASSWORD='';
   
   Note: To limit the scope of the update statement:
   
   SQL query: UPDATE VPX_HOST SET PASSWORD='' where ID = <where ID value from Step1 = 173>;
   
   
3. Restart vCenter Server.
4. Connect to vCenter Server and reconnect all disconnected hosts.
   
   As hosts are reconnected, their encrypted passwords will be repopulated into the database with the new key length.

Note: You may have to re-enter the root password when reconnecting the host to vCenter Server.

Additional Information

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box vCenter Server 5.1 へのアップグレード後に vCenter Server の起動が途中で失敗する
升级到 vCenter Server 5.1 后，vCenter Server 服务在启动时挂起