Host stuck on NSX unistall failed state

Products

• VMware NSX

Issue/Introduction

• Uninstallation of NSX-T from an ESXi host fails via the Manager UI.
  
• Error message indicates a java.rmi.RemoteException involving an input/output error at the /scratch/lifecycle path.
  Failed to uninstall the software on host. Unable to remove vib on host.
java.rmi.RemoteException: [OSError] [Errno 5] Input/output error:
'/scratch/lifecycle' Please refer to the log file for more details.
  

Environment

VMware NSX

Cause

• Esxi kernel is unable to communicate with the physical storage device backing the scratch partition.
• Local storage controller malfunctions or filesystem corruption on the boot media prevent successful vib removal operations.

Resolution

• Fix any underlying storage device issue on the ESXi host.
• Perform Start search resync all if any stale host is seen on NSX manager UI.

Additional Information

Installing NSX on Host fails with "already exists" error or upgrade fails with "Failed to get Host status"