Updating vCenter Server 7.0 to a newer patch fails with error "invalid type expected string instead got none type"
search cancel

Updating vCenter Server 7.0 to a newer patch fails with error "invalid type expected string instead got none type"

book

Article ID: 321523

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

  • The vCenter is or was previously running in Enhanced Link Mode (ELM) 

  • When trying to update vCenter 7.0 to an newer patch fails on error  "invalid type expected string instead got none type"

  • Patchrunner.log, located in /var/log/vmware/applmgmt/, contains the following error stack:
    YYYY-MM-DDTHH:MM:SS.xxxZ  vmdir:Patch ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'vmdir:Patch' failed.
Traceback (most recent call last):
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/py/vmware_b2b/patching/executor/hook_executor.py", line 74, in executeHook
executionResult = systemExtension(args)
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/libs/sdk/extensions.py", line 106, in __call__    result = self.extension(*args)
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/libs/sdk/extensions.py", line 123, in _func return func(*args)
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/payload/components-script/vmdir/__init__.py", line 384, in patch
DoUpgrade(ctx)
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/payload/components-script/vmdir/__init__.py", line 418, in DoUpgrade
 _postInstallScripts(ctx)
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/payload/components-script/vmdir/__init__.py", line 151, in _postInstallScripts
update_vmdir()
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/payload/components-script/vmdir/__init__.py", line 161, in update_vmdir
util.start_service()
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/payload/components-script/vmdir/utils.py", line 390, in start_service
raise Exception("Failed to start service : %s" % self.__VMDIR_SERVICE_NAME)
Exception: Failed to start service : VMwareDirectoryService >>>> VMdir service failed to start so we need to check vmdird-syslog.log to see why it's failed
YYYY-MM-DDTHH:MM:SS.xxxZ  ERROR vmware_b2b.patching.phases.patcher Patch hook Patch got ComponentWrapperError.
Traceback (most recent call last):
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 203, in patch
_patchComponents(ctx, userData, statusAggregator.reportingQueue)
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 85, in _patchComponents
executeComponentHook(Hook.Patch, ctx, c, userData, reportingQueue)
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/py/vmware_b2b/patching/executor/execution_facade.py", line 98, in executeComponentHook
reportQueue, identifier, expectedResultType)  File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/py/vmware_b2b/patching/executor/execution_facade.py", line 53, in executeHook    result = executor.executeHook(scriptFile, hook, args, reportQueue, reportIdentifier)
File "/storage/seat/software-updateb8yc0fgg/stage/scripts/patches/py/vmware_b2b/patching/executor/hook_executor_process.py", line 119, in executeHook raise expatch_errors.ComponentError
YYYY-MM-DDTHH:MM:SS.xxxZ  WARNING root stopping status aggregation...
YYYY-MM-DDTHH:MM:SS.xxxZ  ERROR __main__ Patch vCSA failed
  • Below errors are seen under , /var/log/vmware/vmdir/vmdird-syslog.log 

    YYYY-MM-DDTHH:MM:SS.xxx348+00:00 info vmdird  t@140566389462848: Trying to get topology from host <vCenterFQDN> ... >>> the stale node
YYYY-MM-DDTHH:MM:SS.xxx600+00:00 err vmdird  t@140566389462848: VmDirSafeLDAPBindEx to (ldap://<vCenterFQDN>:389) failed. SRP(9234)
YYYY-MM-DDTHH:MM:SS.xxx830+00:00 err vmdird  t@140566389462848: _VmDirRestoreInstance: cannot get restored USN from partners, error code: 9275.  >>>> trying to connect to the stale node but fails
YYYY-MM-DDTHH:MM:SS.xxx870+00:00 info vmdird  t@140566389462848: No partners are available. Will start in read-only mode.
YYYY-MM-DDTHH:MM:SS.xxx860+00:00 info vmdird  t@140566389462848: VmDirCheckRestoreStatus: RestoreStatus - (2)
YYYY-MM-DDTHH:MM:SS.xxx754+00:00 info vmdird  t@140566389462848: VmDirCheckRestoreStatus: targetState - (2)
YYYY-MM-DDTHH:MM:SS.xxx287+00:00 err vmdird  t@140566389462848: VmDirSafeLDAPBindEx to (ldap://<vCenterFQDN>:389) failed. SRP(9234)
YYYY-MM-DDTHH:MM:SS.xxx377+00:00 warning vmdird  t@140566389462848: Domain functional level could not be validated with a partner host



Environment

VMware vCenter Server 7.0.x

Cause

  • If the vCenter was linked with another vCenter server that was decommissioned incorrectly, there is a chance for it to leave stale entries in the database.
  • This is problematic as the vmdir service still thinks that the decommissioned vCenter is connected and tries to reach it but fails.
  • Validate that if there is stale host node using the command:
    /usr/lib/vmware-vmdir/bin/vdcrepadmin -f showservers -h localhost -u administrator
  • If the undesired node is in the same SSO domain then clear the stale node

Resolution

Note: Take offline snapshots of vCenter prior to proceeding.  If this is an ELM configured vCenter take offline snapshots of all vCenter participating in ELM. 

  1. Log in to the vCenter Server Appliance through SSH.
  2. Type shell and press Enter.
  3. Run the cmsso-util unregister command to unregister the stopped vCenter Server instance:
    cmsso-util unregister --node-pnid vCSA_System_Name --username administrator@your_domain_name --passwd vCenter_Single_Sign_On_password
  4. Run the update again

Additional Information

Impact/Risks:
Deleting anything can be destructive so be sure to take backup/precautions for VC prior to unregistering an old node. 

Powered by Wolken Software