Upgrade failed for <Hostname> host. Please reboot the host immediately to discard the unfinished update.
search cancel

Upgrade failed for <Hostname> host. Please reboot the host immediately to discard the unfinished update.

book

Article ID: 420497

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

When upgrading NSX components via SDDC Manager, the ESXi host upgrade fails.

The following error string is observed in the Host Upgrade failure log:

Upgrade failed for <Hostname> host. Please reboot the host immediately to discard the unfinished update.
Unexpected error while upgrading upgrade unit: Install of offline bundle failed on host ############ with error : [LiveInstallationError] VMware_bootbank_nsx-esx-datapath_4.2.3.1.0-8.0.24954728: Error in running [/etc/init.d/nsx-datapath-dl start upgrade]: Return code: 1 Output: start upgrade begin Exception: Traceback (most recent call last): File "/etc/init.d/nsx-datapath-dl", line 1354, in <module> DualLoadUpgrade() File "/etc/init.d/nsx-datapath-dl", line 1152, in DualLoadUpgrade LoadKernelModules() File "/etc/init.d/nsx-datapath-dl", line 228, in LoadKernelModules nsxesxutils.loadModule(modName) File "/usr/lib/vmware/nsx-esx-datapath/lib64/python/nsxesxutils.py", line 607, in loadModule raise Exception('Failed to load module %s: %s' % Exception: Failed to load module nsx-esx-80u3/nsxt-core-24954728: vmkmod: VMKModLoad: VMKernel_LoadKernelModule(nsxt-core-24954728): Out of memory Cannot load module nsx-esx-80u3/nsxt-core-24954728: Out of memory It is not safe to continue. Please reboot the host immediately to discard the unfinished update. cause = ('nsx-lcp-bundle(4.2.3.1.0-8.0.24954728)', 'nsx-datapath-dl', 'Error in running [/etc/init.d/nsx-datapath-dl start upgrade]:\nReturn code: 1\nOutput: start upgrade begin\nException:\nTraceback (most recent call last):\n File "/etc/init.d/nsx-datapath-dl", line 1354, in <module>\n DualLoadUpgrade()\n File "/etc/init.d/nsx-datapath-dl", line 1152, in DualLoadUpgrade\n LoadKernelModules()\n File "/etc/init.d/nsx-datapath-dl", line 228, in LoadKernelModules\n nsxesxutils.loadModule(modName)\n File "/usr/lib/vmware/nsx-esx-datapath/lib64/python/nsxesxutils.py", line 607, in loadModule\n raise Exception(\'Failed to load module %s: %s\' %\nException: Failed to load module nsx-esx-80u3/nsxt-core-24954728: vmkmod: VMKModLoad: VMKernel_LoadKernelModule(nsxt-core-24954728): Out of memory\nCannot load module nsx-esx-80u3/nsxt-core-24954728: Out of memory\n\n') vibs = ['VMware_bootbank_nsx-esx-datapath_4.2.3.1.0-8.0.24954728'] Please refer to the log file for more details.. Please refer 'https://kb.vmware.com/s/article/91383' article for troubleshooting steps.

The following entries are captured in the vmkernel.log:

2025-11-09T11:06:03.329Z In(182) vmkernel: cpu38:54935941)Loading module nsxt-core-24954728 ...
2025-11-09T11:06:03.330Z In(182) vmkernel: cpu38:54935941)NumaCodeRep: 1112: Failed to allocate read-only code pages for address 0x42001e400000: Out of memory
2025-11-09T11:06:03.330Z Wa(180) vmkwarning: cpu38:54935941)WARNING: MemRO: 946: Failed to grow region by 131072/79699968 bytes: Out of memory
2025-11-09T11:06:03.330Z Wa(180) vmkwarning: cpu38:54935941)WARNING: Elf: 3284: Kernel based module load of nsxt-core-24954728 failed: Out of memory <no mem for Read-Only sections>
2025-11-09T10:23:50.137Z Wa(180) vmkwarning: cpu29:2183058)WARNING: VmAnon: vm 2183058: 1141: anon mpn allocation failed
2025-11-09T10:23:50.137Z Wa(180) vmkwarning: cpu4:2183060)WARNING: VmAnon: vm 2183058: 1141: anon mpn allocation failed
2025-11-09T10:23:50.137Z Wa(180) vmkwarning: cpu11:2183061)WARNING: VmAnon: vm 2183058: 1141: anon mpn allocation failed
2025-11-09T10:23:50.137Z Wa(180) vmkwarning: cpu25:2183062)WARNING: VmAnon: vm 2183058: 1141: anon mpn allocation failed
2025-11-09T10:23:53.201Z In(182) vmkernel: cpu23:54919977)UserCartel: 2920: sh: Execute non-ins

Note: The upgrade precheck executing /tmp/nsx-esx-precheck may still pass successfully, even though memory allocation warnings like anon mpn allocation failed are logged right before the precheck execution in vmkernel.0:

Environment

VMware NSX

Cause

The upgrade process fails because there is insufficient large page memory available to allocate the read-only code pages required by the incoming NSX kernel modules (nsxt-core).

Resolution

This issue is resolved in the NSX 9.0.2 release.

Workaround

If you experience this failure during an active upgrade cycle, execute the following steps to manually clear the state and finish the installation:

  1. Remove the unfinished VIBs from the failing ESXi host.
  2. The NSX Removal Command: nsxcli -c del nsx
  3. Reboot the ESXi host immediately to discard the incomplete update
  4. Once the host is back online and in a clean state, re-initiate the NSX component installation/upgrade from SDDC Manager.
Powered by Wolken Software