vCenter Server 7.0 U3 update fails at 80% during postInstallHook phase

Products

• VMware vCenter Server

Issue/Introduction

• While updating/patching vCenter Server from the VMware Management Interface (VAMI), you see the progress of the update/patch stop at 80%
• You may see a failure message in the VAMI similar to "Invalid Type, Expected String, Instead got None Type" or "Exception occurred in postInstallHook"

Environment

The vCSA VM has an NFS mount point used at some point for Content Library.

• Content Library Storage

Cause

/var/log/vmware/applmgmt/PatchRunner.log 

• We see a permission error when the script is trying to change ownership of the NFS mount:

2024-10-17 13:57:56,666.666Z content-library:Patch ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'content-library:Patch' failed.
Traceback (most recent call last):
  File "/storage/seat/software-updatelq_gks8u/stage/scripts/patches/payload/components-script/content-library/base.py", line 101, in execute
    self.executor(patch_context)
  File "/storage/seat/software-updatelq_gks8u/stage/scripts/patches/payload/components-script/content-library/base.py", line 218, in wrapper
    patch_method(*args, **kwargs)
  File "/storage/seat/software-updatelq_gks8u/stage/scripts/patches/payload/components-script/content-library/patches/cls66.py", line 57, in patch_executor_6_7_0
    change_owner(CONTENT_LIBRARY_CONFIG_HOME, usr_id=cls_uid, grp_id=cls_grp_id)
  File "/storage/seat/software-updatelq_gks8u/stage/scripts/patches/payload/components-script/content-library/patches/cls66.py", line 67, in change_owner
    os.chown(root_dir, usr_id, grp_id)
PermissionError: [Errno 1] Operation not permitted: '/etc/vmware-content-library/nfsmounts/library-nfs-mount-#######-####-####-####-########'

 

Running df -h you see that '/etc/vmware-content-library/nfsmounts/library-nfs-mount-#######-####-####-####-########' NFS mount attached to the vCSA VM

Resolution

After pulling a log bundle from the vCSA VM (SSH to the vCSA VM, run vc-support, use WinSCP or other FTP client to copy the log bundle off the vCSA VM) and reverting the vCSA VM to its snapshot taken before the failed update/patch, unmount the NFS share from the vCSA VM and try the update/patch again.

• SSH into the vCSA VM
  • umount /etc/vmware-content-library/nfsmounts/library-nfs-mount-#######-####-####-####-########