Inventory is not visible in vSphere UI

Products

• VMware vCenter Server

Issue/Introduction

• vCenter web-UI inventory is not visible
  
  •  
• Error displayed "Could not connect to one or more vCenter Server systems" at the top of the vCenter UI (as displayed in screenshot above)
• Below services shows failed post restarting the services using command : service-control --start --all && service-control --stop --all 

  Service-control failed. Error: Failed to start services in profile ALL. RC-1, stderr-failed to start sps, updatemgr, wcp, vpxd, vsan-health, vstats services, Error: Operation times out

• Error observed in/var/log/vmware/vpxd/vpxd.log

  [VpxMain] Setting OpenSSL verify locations for CAFiles = CAPath=/etc/ssl/certs
  
  info  vpxd (######) (Original@xxx sub=default) Creating  SSL Contexts
  error vpxd (######) (Original@xxx sub=default) Failed to initialize the SSL context: N7Vmacore6CryptoExeceptionE (Crypto Exception: error:8xxxxxxx: system library: No such file or directory: unable to load BIO)
  
  --> [context] ################/#################+############
  xxxxxxxxxxxxxxxx ==[/context]
  panic vpxd [xxxxx] (Original@xxx sub=default) 
  
  --->
  ---> Panic: Failed to initialize the SSL context.
  ---> Backtrace:
  ---> [backtrace begin] product: VMware VirtualCenter, version 8.03, build-xxxxxx, tag: vpxd. cpu: x86_64; os: linux, buildType: release
  --->
  --->

Environment

vCenter 7.0.x

vCenter 8.0.x

Cause

Missing rui.crt and rui.key on directory /etc/vmare-vpx/ssl folder on vCenter

Resolution

• Take a snapshot of the vCenter VM (Take offline snapshots if the vCenter is in linked mode)
• SSH to vCenter VM and generate a machine.crt and machine.key files

  /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store MACHINE_SSL_CERT --alias _MACHINE_CERT --output /var/tmp/machine.crt
  
  /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store MACHINE_SSL_CERT --alias _MACHINE_CERT --output /var/tmp/machine.key

 

• Verify both machine crt and key file exist:

  cd /var/tmp

• Make a backup of any existing rui.crt on /etc/vmware-vpx/ssl folder 

  mv rui.crt rui_backup.crt

• Copy machine crt and name as rui.crt and rui.key on /etc/vmware-vpx/ssl directory
  

  cp /var/tmp/machine.crt  /etc/vmware-vpx/ssl/rui.crt
  
  cp /car/tmp/machine.key /etc/vmware-vpx/ssl/rui.key

• Start vmware-vpxd service

  service-control --start vpxd

• Start all remaining services

  service-control --start --all