'lookupsvc:Patch' failed while patching vCenter Server

Products

• VMware vCenter Server 7.0

Issue/Introduction

• vCenter Server patch fails with "lookupsvc:Patch failed" error
• Similar log snippets will be found under /var/log/vmware/applmgmt/PatchRunner.log
  
  =========STDERR=============
  yyyy-mm-ddThh-mm-ssZ lookupsvc:Patch INFO patch_01 yyyy-mm-ddThh-mm-ssZ ERROR starting lookupsvc rc: 2, stdout: , stderr: Start service request failed. Error: Service crashed while starting
  yyyy-mm-ddThh-mm-ssZ Lookup Service firstboot failed
  yyyy-mm-ddThh-mm-ssZ lookupsvc:Patch ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'lookupsvc:Patch' failed.


• /var/log/vmware/applmgmt/update_microservice.log
  
  yyyy-mm-ddThh-mm-ssZ,632 - 37072 -      extensions::        __call__: 107 -  DEBUG - The component script returned 'None'
  yyyy-mm-ddThh-mm-ssZ,636 - 21351 -      update_b2b::        _runScriptHook: 389 -  DEBUG - update script result file '{
    "end_time": "yyyy-mm-ddThh-mm-ssZ",
    "error": {
       "componentKey": "lookupsvc:Patch",
       "detail": [
          {
            "args": [
               ""
            ],
            "id": "patch.internal.error.text",
            "localized": "Internal error occurs during execution of update process .",
            "translatable": "Internal error occurs during execution of update process %(0)s."
          }
       ],
       "problemId": null,
       "resolution": {
          "id": "patch.internal.error.resolution",
          "localized": "Send upgrade log files to VMware technical support team for further assistance.",
          "translatable": "Send upgrade log files to VMware technical support team for further assistance."
       }
    },
    "info": [],
    "progress": 5,
    "progress_message": {
    "componentKey": "lookupsvc:Patch",
    "detail": [
       {
       "args": [
          ""
         ],
         "id": "patch.internal.error.text",
         "localized": "Internal error occurs during execution of update process .",
         "translatable": "Internal error occurs during execution of update process %(0)s."
       }
    ],
    "problemId": null,
    "resolution": {
       "id": "patch.internal.error.resolution",
    }
  
  
• /var/log/vmware/lookupsvc/lookupsvc.log for the same time states permission denied:
  
  Starting service process with pid: 43199.
  /usr/java/jre-vmware/bin/lookupsvc: line 86: /storage/vmware-vmon/lookupsvc.start.cmd: Permission denied

Cause

Inadequate file permissions on /storage/vmware-vmon/lookupsvc.start.cmd

File /storage/vmware-vmon/lookupsvc.start.cmd is created as part of the pre-start script from /etc/vmware/vmware-vmon/svcCfgfiles/lookupsvc.json

Umask basically sets the default permissions for newly created files and directories.

Enter "umask" on ssh console, the default value is 0022 for vCenter Server 7.0 if the value is any different the new files created would have an issue with permissions. 

Resolution

Take appropriate snapshot and backup of the vCenter Server before proceeding.

1. SSH to vCenter Server.
2. Run the below command to change the umask to original value 022  (else section of the file)
   
   # vi /etc/profile.d/umask.sh
   
   Example :
   
   # By default, the umask should be set.
   if [ "$(id -gn)" = "$(id -un)" -a $EUID -gt 99 ]; then
     umask 002
   else
     umask 022
   fi
   
   
3. Execute "source /etc/profile.d/umask.sh"
4. Terminate and create a new SSH session
5. Check umask, expected output is 0022
6. Can also test new file creation using,
   
      $VMWARE_CLOUDVM_RAM_SIZE -J lookupsvc -O /tmp/2.txt
      stat /tmp/2.txt
   
   
7. The new file should have the correct permissions of 644
8. Take fresh snapshot and proceed to update the vCenter Server