Unable to install vmtools or ESXi patch using lifecycle manager fails with: An error occured during host configuration /usr/sbin/esxupdate returned with exit status :32 in vSphere 7.x
search cancel

Unable to install vmtools or ESXi patch using lifecycle manager fails with: An error occured during host configuration /usr/sbin/esxupdate returned with exit status :32 in vSphere 7.x

book

Article ID: 370236

calendar_today

Updated On:

Products

VMware vSphere ESX 7.x VMware vCenter Server VMware vCenter Server 7.0

Issue/Introduction

  • Unable to install VMtools or ESXi patch via lifecycle manager.
  • No issues found while patching via command line.
  • Update initiated by new baseline with "automatically updates" unchecked also fails.
  • Error received on the UI: An error occured during host configuration
    /usr/sbin/esxupdate returned with exit status :32

Log location: /var/log/vmware/vmware-updatemgr/vum-server/vmware-vum-server.log

The patching fails with error in logs stating its failing to install the nsx vibs:



2023-11-03T14:57:59.488+08:00 info vmware-vum-server[177403] [Originator@6876 sub=HostUpdateDepotManager] [vibDownloaderImpl 361] Downloaded and validated vibs successfully
2023-11-03T14:57:59.488+08:00 info vmware-vum-server[177403] [Originator@6876 sub=HostUpdateDepotManager] [patchDepotManager 692] Vib file: /storage/updatemgr/patch-store/hostupdate/vmw/vib20/esx-nsxv/VMware_bootbank_esx-nsxv_6.7.0-0.0.17493714.vibis already downloaded (and being copied).

2023-12-20T11:03:25.637+08:00 info vmware-vum-server[422112] [Originator@6876 sub=HostUpdateDepotManager] [scanHostxxx] List of updates for installation/staging on host: xxx (entity: host-xxx):
--> -----------------------------------------------
--> (VMware_bootbank_esx-nsxv_6.7.0-0.0.17493714, VMware_locker_tools-light_12.3.5.22544099-22545121)
--> ------------------------------------------------
2023-12-20T11:03:25.637+08:00 info vmware-vum-server[422112] [Originator@6876 sub=SingleHostRemediateTask.SingleHostRemediateTask{375}] [singleHostRemediateTaskBase 392] 
--> ------------------------------------------------
--> (VMware_bootbank_esx-nsxv_6.7.0-0.0.17493714, VMware_locker_tools-light_12.3.5.22544099-22545121)
--> ------------------------------------------------

The baseline VM Tools does not contain NSX-v

 
2023-10-27T12:24:14.282+08:00 info vmware-vum-server[09460] [Originator@6876 sub=BaselineMgr] [baselineMgr 5497] Baseline 'VMWare Tools 
12.3.0' with ID '21' has 'TOOLS-22236987', updates

However, the EAM baselines contain NSX-V vib in the updates list:

2023-10-27T12:24:14.308+08:00 info vmware-vum-server[09460] [Originator@6876 sub=BaselineMgr] [baselineMgr 5497] Baseline 'ESX Agent Manager 
:: 56d8c027-####-####-####-########ad1' with ID '10' has 'nsxv-datapath-6.7-17493714', 'nsxv-datapath-6.5-17493714', updates.

2023-10-27T12:24:14.350+08:00 info vmware-vum-server[09460] [Originator@6876 sub=BaselineMgr] [baselineMgr 5497] Baseline 'ESX Agent Manager 
:: bfd62c04-####-####-####-########09e' with ID '8' has 'nsxv-datapath-6.7-17493714', 'nsxv-datapath-6.5-17493714', updates.

2023-10-27T12:24:14.385+08:00 info vmware-vum-server[09460] [Originator@6876 sub=BaselineMgr] [baselineMgr 5497] Baseline 'ESX Agent Manager 
:: 82035b01-####-####-####-########808' with ID '9' has 'nsxv-datapath-6.7-17493714', 'nsxv-datapath-6.5-17493714', updates.

 

 

Environment

Infra has NSX installed.

Cause

vSphere ESX Agent Manager (EAM) may stage and install VIBs through VUM depending on the VIB type.
There are legacy VIBs that are installed through VUM and there are vLCM VIBs that are installed through vLCM.

Eam.log 

2023-11-03T04:45:29.547Z |  INFO | vim-monitor | ExecutorUtils.java | 36 | JOB SUBMITED: [#19808345] StageAgencyJob(Agency ID: bfd62c04-4cd6-4e07-962d-546687f6a09e, Depots: [https://x.x.x.x/bin/vdn/vibs-6.4.10/6.5-17493714/vxlan.zip, https://x.x.x.x/bin/vdn/vibs-6.4.10/6.7-17493714/vxlan.zip], Scope: [ClusterComputeResource:domain-c2307], PM available: false, VUM available: true)
2023-11-03T04:45:29.576Z |  INFO | agency-1 | AuditedJob.java | 106 | JOB STARTED: [#19808345] StageAgencyJob(Agency ID: bfd62c04-4cd6-4e07-962d-546687f6a09e, Depots: [https://x.x.x.x/bin/vdn/vibs-6.4.10/6.5-17493714/vxlan.zip, https://x.x.x.x/bin/vdn/vibs-6.4.10/6.7-17493714/vxlan.zip], Scope: [ClusterComputeResource:domain-c2307], PM available: false, VUM available: true)

2023-11-03T04:45:47.950Z | ERROR | host-292524-0 | VibJob.java | 300 | Cannot access vib package at https://x.x.x.x/bin/vdn/vibs-6.4.10/6.7-17493714/vxlan.zip
com.vmware.eam.security.trust.CannotConnect: Cannot open connection.

2023-11-03T04:45:54.197Z |  INFO | agency-1 | StageAgencyJob.java | 144 | [19808345:6e2079cf85b23eba] Staging agency finished: JobResult[parsedDepots=Result[value=null, exception=com.vmware.eam.exception.CannotHostUrlException: URL:https://x.x.x.x/bin/vdn/vibs-6.4.10/6.5-17493714/vxlan.zip cant be accessed], vumRes=VumResult[operation=NO_OP, uploadedVibs=Result[value=[], exception=null], baselineRecRes=Result[value=null, exception=null]], pmRes=PmResult[operation=NO_OP, clustersRes=[], depotsRes=Result[value=null, exception=null]]]
2023-11-03T04:45:54.197Z |  INFO | agency-1 | AuditedJob.java | 106 | JOB COMPLETED: [#19808345] StageAgencyJob(Agency ID: bfd62c04-4cd6-4e07-962d-546687f6a09e, Depots: [https://x.x.x.x/bin/vdn/vibs-6.4.10/6.5-17493714/vxlan.zip, https://x.x.x.x/bin/vdn/vibs-6.4.10/6.7-17493714/vxlan.zip], Scope: [ClusterComputeResource:domain-c2307], PM available: false, VUM available: true)

2023-11-03T04:45:54.196Z |  INFO | manage-depots-LRO | Workflow.java | 121 | [LoadDepot:96e7463f47bae98] FAILED
com.vmware.eam.exception.CannotHostUrlException: URL:https://x.x.x.x/bin/vdn/vibs-6.4.10/6.5-17493714/vxlan.zip cant be accessed
 at com.vmware.eam.job.UrlHostJob.call(UrlHostJob.java:107) ~[eam-server.jar:?]
 at com.vmware.eam.job.UrlHostJob.call(UrlHostJob.java:33) ~[eam-server.jar:?]
 at com.vmware.eam.async.impl.AuditedJob.call(AuditedJob.java:58) ~[eam-server.jar:?]
 at com.vmware.eam.async.impl.FutureRunnable.run(FutureRunnable.java:55) [eam-server.jar:?]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_362]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_362]
 at java.lang.Thread.run(Thread.java:750) [?:1.8.0_362]
Caused by: com.vmware.eam.security.trust.CannotConnect: Cannot open connection.
 at com.vmware.eam.security.trust.impl.TrustedLeaf.get(TrustedLeaf.java:141) ~[eam-server.jar:?]
 at com.vmware.eam.envoy.route.impl.RouteProvider.provide(RouteProvider.java:193) ~[eam-server.jar:?]
 at com.vmware.eam.envoy.route.impl.RouteProvider.provide(RouteProvider.java:299) ~[eam-server.jar:?]
 at com.vmware.eam.job.UrlHostJob.call(UrlHostJob.java:96) ~[eam-server.jar:?]
 ... 6 more
Caused by: java.net.NoRouteToHostException: No route to host (Host unreachable)
 at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_362]

 

 

 

Resolution

  1. Remove the EAM baseline. 
  2. Remove the EAM agency.
  3. Connect to VCDB and check the table:

VPX_EXT_DATA for EXT_ID = 'com.vmware.vim.eam'

Output is similar to the below code and shows baselines which is seen on the vCenter from old NSX V manager:


'ESX Agent Manager :: 56d8c027-####-####-####-########ad1' 

'ESX Agent Manager :: bfd62c04-####-####-####-########09e'

'ESX Agent Manager :: 82035b01-####-####-####-########808'

Follow the steps below:

  1. Take snapshot of the vCenter.
  2. Stop the below services:
    >> vmware-vpxd
    >>eam
  3. Delete the agent details from the VCDB:

    Example:
    delete from VPX_EXT_DATA where ext_id = 'com.vmware.vim.eam' and data_key like '%56d8c027-####-####-####-########ad1%';             
  4. Restart vCenter services and validate the lifecycle manager to see if the old baseline of EAM is deleted.
  5. Attach the VMware tools baseline and remediate the host. 
Powered by Wolken Software