vSAN Health Service Fails to Start following vCenter Server 8.0 Patch Update

Products

• VMware vCenter Server 8.0
• VMware vCenter Server

Issue/Introduction

• After patching or upgrading vCenter Server to version 8.0 Update 3, the vmware-vsan-health service fails to start. While the patching process may report success, the service manager indicates a failure during the startup phase of the vSAN health component.

• Review of the /log/vmware/applmgmt/patchrunner.log shows the following error:
  
  

  YYYY-MM-DDT 15:20:02.207Z INFO service_manager Command '[['/bin/service-control', '--start', 'vmware-vsan-health']]' has exit-code='1'
  stderr: Error executing start on service vsan-health. Details {
      "detail": [
          {
              "id": "install.ciscommon.service.failstart",
              "translatable": "An error occurred while starting service '%(0)s'",
              "args": ["vsan-health"],
              "localized": "An error occurred while starting service 'vsan-health'"
          }
      ]
  }

Environment

VMware vCenter Server 8.x

Cause

The failure is caused by a Python ImportError within the vSAN management module. Specifically, the system is unable to import the name isXvcHciMeshV2Enabled from the VsanConfigUtil module. This typically occurs when legacy or incompatible .py files remain in the vSAN health library directory, conflicting with the compiled versions required for the new build.

/log/vmware/vsan-health/vmware-vsan-health-service.log:

YYYY-MM-DDT 15:08:13.045Z ERROR vsan-mgmt[######] [VsanVcModuleImporter::startImport opID=noOpId] Issues with Importing VSAN extension VsanVcObjectSystem__ext_init__ with error: cannot import name 'isXvcHciMeshV2Enabled' from 'VsanConfigUtil' (###/usr/lib/vmware-vpx/vsan-health/pyMoVsan/VsanConfigUtil.pyc)
ImportError: cannot import name 'isXvcHciMeshV2Enabled' from 'VsanConfigUtil' (###/usr/lib/vmware-vpx/vsan-health/pyMoVsan/VsanConfigUtil.pyc)

Resolution

To resolve this issue, needs to remove the conflicting Python source files to allow the service to utilize the correct libraries provided by the upgrade.

1. Take a Snapshot: Ensure there is a valid, powered-off snapshot of the vCenter Server Appliance (VCSA) before proceeding.

2. Access VCSA via SSH: Log in to the vCenter Server as root.

3. Navigate to the Library Directory:

   cd /usr/lib/vmware-vpx/vsan-health/pyMoVsan/
   

4. Identify and Move Conflicting Files: If the .py source files are present (which may conflict with the updated .pyc files), move them to a backup location:

   mv VsanVcObjectSystemImpl.py /var/core/
   mv VsanConfigUtil.py /var/core/
   

5. Restart the vSAN Health Service:

   service-control --restart vmware-vsan-health
   

6. Verify Service Status: Ensure the service now starts successfully and the vSAN Health UI is accessible in the vSphere Client.

Additional Information

In some instances, this conflict arises if manual modifications were previously made to these scripts or if the upgrade process failed to clean up the source files from the previous version.