Adding an ESXi 5.0 host to vCenter Server 5.0 fails with the error: A general system error occurred

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

This article describes a specific issue. If you experience all of the above symptoms, consult the sections below. If you experience some, but not all, of these symptoms, your issue is not related to this article.

Search the Knowledge Base for your symptoms or file a support request. For more information, see Filing a Support Request in Customer Connect (2006985).

Symptoms:

• You cannot add an ESXi 5.0 host to vCenter Server 5.0
• Adding an ESXi 5.0 host to vCenter Server 5.0 fails
• You see the error:
  
  Timed waiting for vpxa to start
  
  
• In the vpxd.log file, you see a backtrace similar to:
  
  A general system error occurred: Unexpected exception reading HTTP response body: class Vmacore::Http::TruncatedResponseException(While determining chunk size, truncated HTTP response.) with trace:
  backtrace[00] rip 000000018013d40a (no symbol)
  backtrace[01] rip 00000001800ffa38 (no symbol)
  backtrace[02] rip 00000001800fffee (no symbol)
  backtrace[03] rip 000000018008794b (no symbol)
  backtrace[04] rip 000000018003ed6f (no symbol)
  backtrace[05] rip 000000018004349e (no symbol)
  backtrace[06] rip 00000000005d2989 (no symbol)
  backtrace[07] rip 00000000005a35d7 (no symbol)
  backtrace[08] rip 00000000005a3bd9 (no symbol)
  backtrace[09] rip 000000014060f6af (no symbol)
  backtrace[10] rip 000000014061702a (no symbol)
  backtrace[11] rip 0000000140617b10 (no symbol)
  backtrace[12] rip 000000014060f827 (no symbol)
  backtrace[13] rip 00000001406187c4 (no symbol)
  backtrace[14] rip 000000000051263c (no symbol)
  backtrace[15] rip 000000000060a2c7 (no symbol)
  backtrace[16] rip 00000001400d7d20 (no symbol)
  backtrace[17] rip 00000001400d890c (no symbol)
  backtrace[18] rip 0000000180153dad (no symbol)
  backtrace[19] rip 00000001801552d4 (no symbol)
  backtrace[20] rip 000000018014dc65 (no symbol)
  backtrace[21] rip 00000000708a2fdf (no symbol)
  backtrace[22] rip 00000000708a3080 (no symbol)
  backtrace[23] rip 0000000076bbf56d (no symbol)
  backtrace[24] rip 0000000076cf2cc1 (no symbol)
  while parsing serialized value of type string at line 17, column 374505
  while parsing property "uuid" of static type string
  while parsing serialized DataObject of type vim.vm.device.VirtualDisk.FlatVer2BackingInfo
  at line17, column 374240
  while parsing property "parent" of static type VirtualDiskFlatVer2BackingInfo
  while parsing serialized DataObject of type vim.vm.device.VirtualDisk.FlatVer2BackingInfo
  at line 17, column 373871 while parsing property "parent" of static type VirtualDiskFlatVer2BackingInfo
  while parsing serialized DataObject of type vim.vm.device.VirtualDisk.FlatVer2BackingInfo at line 17, column 373502
  while parsing property "parent" of static type VirtualDiskFlatVer2BackingInfo
  
  
• The host does not get added to vCenter Server.
• The vpxd.log file contains entries similar to:
  
  T12:53:11.394+01:00 [04536 info 'Default' opID=CE9B4629-00000205] [VpxLRO] -- ERROR task-5638 -- domain-c7 -- vim.ClusterComputeResource.addHost: vmodl.fault.SystemErr
  
  --> Result: --> (vmodl.fault.SystemError) { --> dynamicType = <unset>, --> faultCause = (vmodl.MethodFault) null, --> reason = "Timed waiting for vpxa to start", --> msg = "",
  
  
• The hostd.log file (located at /var/log/) contains entries similar to:
  
  [FFD40B90 verbose 'Proxysvc Req00049'] New proxy client SSL(TCP(local=X.X.X.X:443, peer=X.X.X.X:59734))
  
  [309EBB90 warning 'Proxysvc Req00049'] Connection to localhost:8089 failed with error N7Vmacore15SystemExceptionE(Connection refused).
  
  
• The vobd.log file (located at /var/log/) contains entries similar to:
  
  [UserWorldCorrelator] XXXXXXXXXXXus: [esx.problem.vpxa.core.dumped] /usr/lib/vmware/vpxa/bin/vpxa crashed (1 time(s) so far) and a core file may have been created at /var/core/vpxa-zdump.003. This may have caused connections to the host to be dropped.
  
  [UserWorldCorrelator] XXXXXXXXXXXus: [vob.uw.core.dumped] /usr/lib/vmware/vpxa/bin/vpxa(86449) /var/core/vpxa-zdump.003
  
  
• In the vpxd.log, you see entries similar to:
  
  [05188 info 'Default' opID=899032E5-00000ABD-7c] [VpxLRO] -- ERROR task-9481 -- vm-87 -- vim.VirtualMachine.reconfigure: vmodl.fault.InvalidRequest:
  --> Result:
  --> (vmodl.fault.InvalidRequest) {
  --> dynamicType = <unset>,
  --> faultCause = (vmodl.MethodFault) null,
  --> msg = "XML element depth exceeds configured maximum 100
  
  
• You are unable to add an RDM to a virtual machine

Cause

This issue occurs when a virtual machine on the host that is added to vCenter Server has more than 32 snapshots. Due to the number of snapshots, the vpxa process runs out of memory when vpxa runs queries on the virtual machine configuration.

Resolution

To work around this issue, remove snapshots from the virtual machine that has more than 32 snapshots.

To remove snapshots from the virtual machine:

1. Connect to the ESXi host using SSH. For more information, see Using ESXi Shell in ESXi 5.0 and 5.1 (2004746).
2. Determine if the virtual machine is using snapshots by running the command:
   
   find /vmfs/volumes/*/* -iname *delta*
   
   For more information, see Determining if a virtual machine is using snapshots (1004343).
   
   
3. Remove the snapshots from the virtual machine either by consolidating or using the Delete All option.
   
   For more information, see Consolidating snapshots in ESX/ESXi 3.x and 4.x (1007849).
   
   For more information on cloning virtual machine disks with snapshots, see Cloning individual virtual machine disks via the ESX/ESXi host terminal (1027876).
   
   
4. Re-add the host to vCenter Server.

Alternatively, you can modify the vpxa.cfg file (located at /etc/vmware/vpxa/) on the affected host to increase the thread stack size.

To increase the thread stack size:

1. Connect to the ESXi host using SSH. For more information, see Using ESXi Shell in ESXi 5.0 and 5.1 (2004746).
2. Change directory and back up the current vpxa.cfg file by running the commands:
   
   cd /etc/vmware/vpxa
   cp vpxa.cfg vpxa.cfg.orig
   
   
3. Open the vpxa.cfg file using a text editor.
4. Increase the ThreadStackSizeKb value to 1024.
   
   Note: The default value is:
   
   <ThreadStackSizeKb>128</ThreadStackSizeKb>.
   
   For example:
   
   <Config>
   <vmacore>
   <threadPool>
   <ThreadStackSizeKb>1024</ThreadStackSizeKb>
   </threadPool>
   </vmacore>
   </Config>
   
   
5. Save and close the vpxa.cfg file.
6. Use this command to restart the vpxa agent for the configuration changes to take affect.
   
   /etc/init.d/vpxa restart
   
   
7. After the changes are applied, add/reconnect the host to vCenter Server.
8. Resolve the snapshot issue in the virtual machine using the earlier procedure.
9. After the snapshots are consolidated, revert the changes made to the vpxa.cfg file in step 4.

Additional Information

You can use additional commands to find snapshot information. For more information, see Using ESXi Shell in ESXi 5.0 and 5.1 (2004746).

To find out if one or more virtual machines has a large number of snapshots, run the command:

find /vmfs/volumes/ -name *delta* -print

To find out how many delta files there are per host, perform a word count by running the command:

find /vmfs/volumes/*/* -iname *delta* | wc -l

Determining if a virtual machine is using snapshots
Consolidating snapshots in ESX/ESXi 3.x and 4.x
Location of vCenter Server log files
Cloning individual virtual machine disks via the ESX/ESXi host terminal
Using ESXi Shell in ESXi 5.x and 6.x
How to file a Support Request in Customer Connect
ESXi 5.0 ホストを vCenter Server 5.0 に追加すると、次のエラーで失敗する: 一般システム エラーが発生しました
将 ESXi 5.0 主机添加到 vCenter Server 5.0 失败，并显示以下错误：出现了常规系统错误