NSX-T VIBs fail to install/upgrade, due to insufficient space in bootbank on ESXi host.
Article ID: 324231
Updated On:
Products
VMware NSX
Issue/Introduction
Symptoms:
- ESXi 6.7
- When attempting to install NSX-T on ESXi host, it may fail.
- ESXi host logs (esxupdate.log) display message(s) similar to:
20**-**-**T13:37:50Z esxupdate: 5557508: BootBankInstaller.pyc: ERROR: The pending transaction requires 245 MB free space, however the maximum supported size is 239 MB.^@
Environment
VMware NSX-T Data Center
VMware NSX-T Data Center 2.x
VMware NSX-T Data Center 2.x
Cause
This issue occurs when the bootbank / altbootbank runs out of space on the ESXi host. There is no supported procedure to increase the size of either bootbank on the ESXi host.
Some ESXi images provided by third-party vendors may include VIBs which are not in use and can be relatively large in size. This can result in insufficient space in bootbank/alt-bootbank when installing/upgrading any VIBs.
Some ESXi images provided by third-party vendors may include VIBs which are not in use and can be relatively large in size. This can result in insufficient space in bootbank/alt-bootbank when installing/upgrading any VIBs.
Resolution
This issue is resolved in ESXi 7.0 which introduced a larger bootbank for both upgraded and newly installed ESXi hosts.
Workaround:
To work around this issue, free the additional space on the bootbank by un-installing non-required VIBs on the ESXi host using the 'esxcli software vib remove -n <vib-name>' command.
1) Check the free space in bootbank / altbootbank:
2) Check which partition (bootbank or altbootbank) contain the VIBs last updated by running the below commands:
If bootbank has the higher value for “updated”, as in the example above, then that partition has the latest update to the VIBs. Otherwise altbootbank would have the latest update to the VIBs.
3) NSX software space requirements, note reduced sizing from 3.0
NSX-T 2.4.0 - VMware ESXi 6.5 - 32.17 MB
NSX-T 2.4.0 - VMware ESXi 6.7 - 32.55 MB
NSX-T 2.5.0 - VMware ESXi 6.5 - 37.24 MB
NSX-T 2.5.0 - VMware ESXi 6.7 - 37.64 MB
NSX-T 3.0.0 - VMware ESXi 6.5 - 29.16 MB
NSX-T 3.0.0 - VMware ESXi 6.7 - 29.42 MB
NSX-T 3.0.0 - VMware ESXi 7.0 - 29.43 MB
As well as this space requirement for the NSX VIBs, ESX install/upgrade framework has the following requirements:
For an NSX-T upgrade, an additional 20MB free space buffer is required in the partition with the highest updated value (check step 2 above to find the partition with highest updated value).
For a new install of NSX-T, an additional 10MB free space buffer is required in the partition with highest updated value.
4) To determine the currently installed VIBs in bootbank/altbootbank partitions run the following command:
Workaround:
To work around this issue, free the additional space on the bootbank by un-installing non-required VIBs on the ESXi host using the 'esxcli software vib remove -n <vib-name>' command.
1) Check the free space in bootbank / altbootbank:
ls -l ... lrwxrwxrwx 1 root root 49 Aug 9 19:06 altbootbank -> /vmfs/volumes/d7607071-826ffd54-8cd6-69cc3f3840ff drwxr-xr-x 1 root root 512 Aug 9 19:04 bin lrwxrwxrwx 1 root root 49 Aug 9 19:06 bootbank -> /vmfs/volumes/42910973-563a8bff-11f1-af4f22ba2855 ...
To see how much space is used and free:
df -h Filesystem Size Used Available Use% Mounted on VMFS-6 383.8G 125.0G 258.7G 33% /vmfs/volumes/stgb01 vfat 285.8M 172.9M 112.9M 60% /vmfs/volumes/5c669304-eda3d71f-1224-0050560302fa vfat 249.7M 149.6M 100.1M 60% /vmfs/volumes/d7607071-826ffd54-8cd6-69cc3f3840ff -> partitions used for altbootbank, see above vfat 249.7M 150.5M 99.2M 60% /vmfs/volumes/42910973-563a8bff-11f1-af4f22ba2855 -> partitions used for bootbank, see above
2) Check which partition (bootbank or altbootbank) contain the VIBs last updated by running the below commands:
grep updated /bootbank/boot.cfg
updated=8
grep updated /altbootbank/boot.cfg
updated=7
If bootbank has the higher value for “updated”, as in the example above, then that partition has the latest update to the VIBs. Otherwise altbootbank would have the latest update to the VIBs.
3) NSX software space requirements, note reduced sizing from 3.0
NSX-T 2.4.0 - VMware ESXi 6.5 - 32.17 MB
NSX-T 2.4.0 - VMware ESXi 6.7 - 32.55 MB
NSX-T 2.5.0 - VMware ESXi 6.5 - 37.24 MB
NSX-T 2.5.0 - VMware ESXi 6.7 - 37.64 MB
NSX-T 3.0.0 - VMware ESXi 6.5 - 29.16 MB
NSX-T 3.0.0 - VMware ESXi 6.7 - 29.42 MB
NSX-T 3.0.0 - VMware ESXi 7.0 - 29.43 MB
As well as this space requirement for the NSX VIBs, ESX install/upgrade framework has the following requirements:
For an NSX-T upgrade, an additional 20MB free space buffer is required in the partition with the highest updated value (check step 2 above to find the partition with highest updated value).
For a new install of NSX-T, an additional 10MB free space buffer is required in the partition with highest updated value.
4) To determine the currently installed VIBs in bootbank/altbootbank partitions run the following command:
ls -lSrh /bootbank/ ls -lSrh /altboobank/
Note: If the ESXi server was installed using a software image provided by a third party, you need to contact the third party and work with them to identify the VIBs which can be safely removed.5) Once the VIBs to be removed have been determined, remove the unused VIBs using command:
esxcli software vib remove -n <vib-name>
If any issues are encountered during the above process, file a support request with VMware support and quote this Knowledge base article (74864) in the problem description.
6) Once the required VIBs have been removed and bootbank / altbootbank has the required space to proceed as indicated in (Step 3), proceed with install or upgrade.
Feedback
Yes
No
Powered by
