vCenter services fail to start following a reboot

Products

• VMware vCenter Server

Issue/Introduction

Upon a reboot of the vCenter appliance vmon services fail to start


The following error is observed in vmon.log

/var/log/vmware/vmon/vmon.log

Service pre-start command failed with exit code 1
File "/usr/lib/vmware-observability/scripts/obs_prestart_vapi.py", line 11, in <module>
<observability-vapi> Service pre-start commands stderr: from cis svcaccount_prestart_util import setup_service_account
File "/usr/lib/vmware/site-packages/cis/svcaccount_prestart_util.py", line 28, in <module>
from cix.cisreglib import vmafdClient, LookupServiceClient
File "/usr/lib/vmware/site-packages/cis/cisreglib.py", line 32, in <module>
from pyVmomi import (lookup, SoapStubAdapter, vmodl, dataservice,
ImportError: cannot import name 'lookup' from 'pyVmomi' (/usr/lib/python3.10/site-packages/pyVmomi/__init__.py

lsdoctor will also fail to execute with the following observed

Traceback (most recent call last):
  File "/tmp/lsdoctor-251006/lsdoctor.py", line 19, in <module>
    from lib.utils.import *
  File "/tmp/lsdoctor-251006/lib/utils.py", line 54, in <module>
    from pyVmomi imort (lookup, SoapStubAdapter, vmodl, dataservice.
  ImportError: cannot name 'lookup' from 'pyVmomi' (/usr/lib/python3.10/site-packages/pyVmomi/___init___.py)

To validate python module on problematic vCenter use command below.

# python -V
Python 3.7.5

# echo $PYTHONPATH

# echo $VMWARE_PYTHON_PATH
/usr/lib/vmware/site-packages

# python -c 'import pyVmomi; print(pyVmomi.__file__)'
/usr/lib/python3.7/site-packages/pyvmomi-8.0.2.0.1-py3.7.egg/pyVmomi/__init__.py

# PYTHONPATH=$VMWARE_PYTHON_PATH:$PYTHONPATH python -c 'import pyVmomi; print(pyVmomi.__file__)'
/usr/lib/vmware/site-packages/pyVmomi/__init__.py

Environment

vCenter 8.x

Cause

Command

python -c 'import pyVmomi; print(pyVmomi.__file__)'

should fail with the below output

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'pyVmomi'

If the command returns an output similar to below it indicates an incorrect public version of pyVmomi has been installed

/usr/lib/python3.10/site-packages/pyVmomi/__init__.py

Path /usr/lib/python3.10/site-packages/pyVmomi does not exist in vCenter 8.x


The default location for pyVmomi is /usr/lib/vmware/site-packages/pyVmomi on vCenter Server. It is not supported to change/modify/install the default python library on vCenter

Resolution

To resolve this issue, please restore the vCenter from backup from before the module was manually installed. For reference see: Overview of Backup and Restore Options in vCenter Server 6.x/7.0.x/8.0.