Failed to apply Component: an error occurred while enabling service nsx-datapath
search cancel

Failed to apply Component: an error occurred while enabling service nsx-datapath

book

Article ID: 407934

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

  • Unable to install nsx vibs on the ESXi host using vLCM.
  • From /var/run/log/lifecycle.log 
    • YYYY-MM-DDTHH:MM:SSZ Db(15) lifecycle[2733219]: HostImage:1454 installer LiveImageInstaller failed: VMware_bootbank_nsx-esx-datapath_4.1.1.0.0-8.0.22224315: VMware_bootbank_nsx-esx-datapath_4.1.1.0.0-8.0.22224315: Error in running [/etc/init.d/nsx-datapath start install]:
YYYY-MM-DDTHH:MM:SSZ Db(15)[+] lifecycle[2733219]: Return code: 1
YYYY-MM-DDTHH:MM:SSZ Db(15)[+] lifecycle[2733219]: Output: start install begin
YYYY-MM-DDTHH:MM:SSZ Db(15)[+] lifecycle[2733219]: Exception:
YYYY-MM-DDTHH:MM:SSZ Db(15)[+] lifecycle[2733219]: Traceback (most recent call last):
YYYY-MM-DDTHH:MM:SSZ Db(15)[+] lifecycle[2733219]:   File "/etc/init.d/nsx-datapath", line 2000, in <module>
YYYY-MM-DDTHH:MM:SSZ Db(15)[+] lifecycle[2733219]:     checkLKMs()
YYYY-MM-DDTHH:MM:SSZ Db(15)[+] lifecycle[2733219]:   File "/etc/init.d/nsx-datapath", line 1631, in checkLKMs
YYYY-MM-DDTHH:MM:SSZ Db(15)[+] lifecycle[2733219]:     raise Exception(msg)
YYYY-MM-DDTHH:MM:SSZ Db(15)[+] lifecycle[2733219]: Exception: Failed to load kernel modules: nsxt-ipfix-22224315, nsxt-dvfilterUser-22224315, nsxt-core-22224315, nsxt-vdrb-22224315, nsxt-livetrace-22224315, nsxt-switch-security-22224315, nsxt-fc-22224315, nsxt-spf-22224315, nsxt-vswitch-22224315, nsxt-vsip-22224315, nsxt-ens-22224315, nsxt-vdl2-22224315, nsxt-traceflow-22224315, nsxt-latency-22224315, nsxt-obsrv-22224315, nsxt-bfd-22224315, nsxt-kcp-22224315
YYYY-MM-DDTHH:MM:SSZ Db(15)[+] lifecycle[2733219]:
YYYY-MM-DDTHH:MM:SSZ Db(15)[+] lifecycle[2733219]: It is not safe to continue. Please reboot the host immediately to discard the unfinished update.. Clean up the installation.
  • From /var/run/log/vmkernel.log 
    • YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu17:2733744)Admission failure in path: host/vim/vmvisor/settingsd-task-forks/python.2733219:python.2733760:uw.2733760
YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu17:2733744)UserWorld 'python' 2733760 with cmdline 'unknown', parent 2733744
YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu17:2733744)started from 'python' 2733744 with cmdline '/usr/bin/python /usr/lib/vmware/nsx-esx-datapath/bin/jumpstart.py', parent 2733743
YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu17:2733744)started from 'sh' 2733743 with cmdline '/bin/sh -c LANG=en_US.UTF-8  /usr/lib/vmware/nsx-esx-datapath/bin/jumpstart.py
YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel:    ', parent 2733742
YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu17:2733744)started from 'jumper2' 2733742 with cmdline '/sbin/jumper2 --plugin=nsx-datapath.json', parent 2733741
YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu17:2733744)started from 'jumper2' 2733741 with cmdline '/sbin/jumper2 --plugin=nsx-datapath.json', parent 2733219
YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu17:2733744)started from 'python' 2733219 with cmdline '/usr/bin/python /usr/lib/vmware/lifecycle/bin/imagemanagerctl.py software --apply --task-id 526c9929-83b3-2ffa-4bc9-200bc22916d4 --depot http://hostname.$
YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu17:2733744)started from 'init' 2097869 with cmdline '/bin/init', parent 0
YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu17:2733744)uw.2733760 (3341962) requires 700 KB, asked 700 KB from python.2733219 (3337353) which has 322220 KB occupied and 340 KB available.
  • From UI, we will observe below error.

Environment

  • VMware vSphere 8.0 U3

Cause

  • Insufficient sizing of the memory resource pool of one of the services during the installation of VIBs in an NSX deployment might cause the deployment to fail. In the /var/run/log/vmkernel.log  you see an error such as:  
    • YYYY-MM-DDTHH:MM:SSZ In(182) vmkernel: cpu68:2107749)Admission failure in path: host/vim/vmvisor/settingsd-task-forks/python.2107554:python.2107764:uw.2107764

 

Resolution

This is a known issue and it is resolved in ESXi 8.0 update 3g build 24859861. 

Reference: VMware ESXi 8.0 Update 3g Release Notes

Workaround:

  • Run the below command to increase the memory to 400 MB. 
    • localcli --plugin-dir=/usr/lib/vmware/esxcli/int sched group setmemconfig -g host/vim/vmvisor/settingsd-task-forks -m 400 -i 0 -l -1 -u mb
    • Note: In some scenario you might need to increase the memory more than 400 MB, accordingly you will increase the memory for the pool.
  • Perform the remediation of the image to install the NSX VIBs on the ESXi host.

NOTE: Please reboot the ESX host after any failed remediation attempt. Remediation will still fail even after performing the above workaround if the host was not rebooted since last failed attempt. 

Powered by Wolken Software