vSphere Update Manager fail to update firmware due to incompatible controller utility installed on ESXi host

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Updating controller firmware on an ESXi Server host using vSphere Update Manager might leave the ESXi Server host in Maintenance Mode with General vSAN error.
• You might see this error for the vSAN: Update firmware task:

General event: Runtime error: invalid literal for int() with base10: 'NotImplemented'

OR

General event: Runtime error: (vim.fault.VsanFault) { dynamicType = <unset>, dynamicProperty = (vmodl.DynamicProperty) [], msg = <unset>, faultCause = <unset>, faultMessage = (vmodl.LocalizableMessage) [ (vmodl.LocalizableMessage) { dynamicType = <unset>, dynamicProperty = (vmodl.DynamicProperty) [], key = 'com.vmware.vsan.patch.msg.installfailure', arg = (vmodl.KeyAnyValue) [], message = "There was an error resolving dependencies.

• In this situation, the Update Manager may not be able to update the firmware and remediation process will be terminated.

Environment

VMware vSphere ESXi 6.7

Cause

The issue can be caused by an incompatible controller utility installed on the ESXi Server host. During the firmware update process, Update Manager will first install a VMware verified controller utility onto the ESXi Server host in order to flash the controller firmware. If an incompatible controller utility is already installed on ESXi Server host, the installation of the VMware verified controller utility may fail which cause the firmware remediation process to bail out and leave the host in maintenance mode.

Note: The primary controller utilities covered in this article are:

• perccli

Resolution

To resolve this issue, if the installed controller utility is incompatible with vSphere Update Manager, manually uninstall the incompatible controller utility from ESXi Server hosts and restart the firmware remediation.

Additional Information

Q: What is a VMware verified controller utility?

A VMware verified controller utility is a specific version of the controller utility, that VMware has verified the controller firmware update function with. For every VMware certified controller firmware that Update Manager supports updating, there is a VMware verified controller utility which Update Manager will use to flash the controller.
 
Q: Who is likely to be affected by this issue?

If customer has used Configuration Assist to install controller utility or update firmware, or manually installed controller utility, customer is more vulnerable to this issue, as long as the vendor has a new but incompatible version of controller utility listed in VMware Compatibility Guide that is being consumed by the vSphere Update Manager.

Q: How can I determine which version of the controller utility is compatible with vSphere Update Manager?

Every certified controller firmware has an associated controller utility, to determine the exact controller utility that Update Manager will use to update firmware, user can navigate to the Download vendor firmware tool page to get the utility download link for confirmation.

1.  Click Download vendor firmware tool.

 

2.  Click Enter your own location option, a download link of the controller utility will be shown to guide user who has no Internet Access to download the tool and then upload to the VUM server.