Cannot connect host to vCenter due to error: string too large for database
search cancel

Cannot connect host to vCenter due to error: string too large for database

book

Article ID: 327022

calendar_today

Updated On:

Products

VMware vCenter Server VMware vSAN

Issue/Introduction

This article is created for the purpose of resolving and preventing 'string too large for database' errors in vCenter.

Symptoms:

Attempting to connect an ESXi host to vCenter times out and you see error similar to:

string too large for database

In the C:\ProgramData\VMware\VMware VirtualCenter\Logs\vpxd.log, you see entries similar to:


error vpxd[7FC341374700] [Originator@6876 sub=Default opID=AddHostWizard-apply-10967-ngc:70003754-72] [Vdb::VdbField] Invalid value written to column DEVICE_INFO_SUMMARY in table VPX_VM_VIRTUAL_DEVICE
error vpxd[7FC341374700] [Originator@6876 sub=Default opID=AddHostWizard-apply-10967-ngc:70003754-72] [Vdb::VdbField] String too large: 512 > max(510)
error vpxd[7FC341374700] [Originator@6876 sub=MoHost opID=AddHostWizard-apply-10967-ngc:70003754-72] [HostMo::Reconnect] Got method fault: N5Vmomi5Fault11SystemError9ExceptionE(vmodl.fault.SystemError)
[context]zKq7AVECAAAAAIStcgAednB4ZAAAeF4rbGlidm1hY29yZS5zbwAAEBcbAMppGAGCAHF2cHhkAAF4D3EBGuJ9AcALfgEUs64BmHStAcqDrQHTzHwBQc58AcVvfQEjdH0BdM95AQx7egEqg3oB7NBYAbuNWYLD4QUBbGlidmltLXR5cGVzLnNvAAEuw3QBlbpzAQqwcwF71HMBsid0AEesIwAuryMAN
7crA1R0AGxpYnB0aHJlYWQuc28uMAAEHYwObGliYy5zby42AA==[/context]
2018-05-09T14:38:37.309Z info vpxd[7FC341374700] [Originator@6876 sub=vpxLro opID=AddHostWizard-apply-10967-ngc:70003754-72] [VpxLRO] -- FINISH task-108756
2018-05-09T14:38:37.309Z info vpxd[7FC341374700] [Originator@6876 sub=Default opID=AddHostWizard-apply-10967-ngc:70003754-72] [VpxLRO] -- ERROR task-108756 -- host-20 -- vim.HostSystem.reconnect: vmodl.fault.SystemError:
--> Result:
--> (vmodl.fault.SystemError) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = <unset>,
--> reason = "string too large for database"
--> msg = ""
--> }
--> Args:
-->
--> Arg cnxSpec:
--> (vim.host.ConnectSpec) {
--> hostName = "host-3.vmware.local",
--> port = -1,
--> sslThumbprint = <unset>,
--> userName = "root",
--> password = (not shown),
--> vmFolder = 'vim.Folder:617870ee-75a2-4a71-813b-989537a5135d:group-v3',
--> force = true,
--> vimAccountName = "vpxuser",
--> vimAccountPassword = (not shown),
--> managementIp = <unset>,
--> lockdownMode = <unset>,
--> hostGateway = (vim.host.GatewaySpec) null
--> }
--> Arg reconnectSpec:



In the /var/log/vpxa.log for the host you see entries similar to:

verbose vpxa[740EB70] [Originator@6876 sub=vpxaVmprovUtil opID=AddHostWizard-apply-6826-ngc:70001896-a0-d4] [TranslateDatastoreMoref] vim.Datastore:vsan:52dc0a41a225df28-022d933f1cb6db1d -> vim.Datastore:ds://
/vmfs/volumes/vsan:52dc0a41a225df28-022d933f1cb6db1d/
verbose vpxa[740EB70] [Originator@6876 sub=vpxaMoVm opID=AddHostWizard-apply-6826-ngc:70001896-a0-d4] Existing ISO file name: SW_DVD5_Win_Svr_Std_and_DataCtr_2012_64Bit_English_Core_MLF_X18-27588_01130f0e-0b02
-461b-8c67-321155940509.ISO

verbose vpxa[740EB70] [Originator@6876 sub=PropertyProvider opID=AddHostWizard-apply-6826-ngc:70001896-a0-d4] RecordOp ASSIGN: info.progress, session[529342b7-3a6b-cb5c-b9c6-cd2a19400b91]52308476-734d-7a42-31d
d-8806d9b711de. Applied change to temp map.


Environment

VMware vCenter Server 6.x
VMware vCenter Server 6.5.x
VMware vSAN 6.x
VMware vCenter Server 6.7.x

Cause

This issue is triggered when there are ISOs with lengthy names mounted to VMs when using Content Library on VSAN Datastores.

The vCenter Database table 'VPX_VM_VIRTUAL_DEVICE' has a column called 'DEVICE_INFO_SUMMARY' which is only designed to handle a certain number of characters.

This column contains the paths to the devices that are mounted on virtual machines.

By default, the VSAN datastore path is longer than the paths to VMFS or NFS.

Content Library also adds an extension to ISO files. 

When using both of the above, having naturally large named ISOs can surpass the column character limit for 'DEVICE_INFO_SUMMARY'.

Resolution

Upgrade vCenter to version 7.0 or higher.


Workaround:
If an upgrade is not possible then follow the below steps.
STEP 1: 

Identify the virtual machines:

PowerCLI:

Connect-VIServer -Server
VCENTER_FQDN_or_IP -User USERNAME

Enter password for specified user name.

Run the following command :

Get-VM | FT Name, @{Label="ISO file"; Expression = { ($_ | Get-CDDrive).ISOPath }}​

The command output will give you the names & paths to the virtual machines with ISOs mounted


ESXi CLI (SSH/ESXi Shell):

Run the following command on the affected Host(s):

grep -i ISO /vmfs/volumes/*/*/*.vmx

The command output will give you the names & paths to the VMs with ISOs mounted



GUI (HOST UI):

Check each VM by right clicking one at a time and going to 'Edit Settings'.

Check the CD/DVD drive to see if there is an ISO mounted.



STEP 2: 

REMOVE THE PROBLEMATIC ISO FILES:

ESXi CLI (SSH/ESXi Shell):

Type the following command to edit vmx file using the paths found in STEP 1 (BACKUP THE VMX FILE FIRST - cp VM_NAME.vmx VM_NAME.vmx.bak):

vi /vmfs/volumes/VSAN_DATASTORE/VM_DIRECTORY/VM_NAME.vmx

Type 'i' for INSERT mode.

Move to line referencing ISO file using Arrow Keys on your Keyboard.

Remove the ISO name by going to the end of the line and using backspace.

Hit the 'ESC' key to exit INSERT mode.

Type ':' followed by 'wq' and hit ENTER.


GUI (HOST UI):

Right Click VM in question and select 'Edit Settings'.

Change the CD/DVD drive to Client Device




STEP 3:

 
Once the ISO has been unmounted from the VM, you can rename the ISO to a shorter name so it can be used in the vSAN environment. 

Additional Information

Impact/Risks:
This issue will render the ESXi hosts disconnected/unusable in vCenter until action is taken.