vstats service fails to start

Products

• VMware vCenter Server

Issue/Introduction

• VAMI backups are failing
• vstats service in stopped state
• attempting to start the vstats service fails
  
  service-control --start vstats: Got an error while executed the cmd
  
  
• /var/log/vmware/vstats/vstats-stderr.log
  
  

  2024-07-02T09:02:19Z    ERROR   ssolib/helper.go:78     Failed executing shell command  {"vStats": "MVP", "cmd": "/usr/lib/vmware-vmafd/bin/vecs-cli", "args": ["entry", "getcert", "--store", "vpxd-extension", "--alias", "vpxd-extension"], "stdout": "", "stderr": "vecs-cli failed. Error 5: Possible errors: \nLDAP error: Compare False \nWin Error: Operation failed with error ERROR_ACCESS_DENIED (5) \n", "error": "exit status 5"}
  vstats/ssolib.runCommand
          src/vstats/ssolib/helper.go:78
  vstats/ssolib.getX509KeyPair
          src/vstats/ssolib/signer.go:102
  vstats/ssolib.GetHOKSigner
          src/vstats/ssolib/signer.go:69
  vstats/vchelper.(*reloginRoundTripper).login
          src/vstats/vchelper/client.go:302
  vstats/vchelper.installReloginRoundTripper
          src/vstats/vchelper/client.go:270
  vstats/vchelper.newSAMLAuthClient
          src/vstats/vchelper/client.go:131
  vstats/vchelper.newClient
          src/vstats/vchelper/client.go:103
  vstats/vchelper.New
          src/vstats/vchelper/inventoryHelper.go:268
  main.setupInventory
          src/vstats/main.go:591
  main.main
          src/vstats/main.go:238
  runtime.main
          GOROOT/src/runtime/proc.go:255
  

• Solution User VECS stores are not present when checking certificate expiration
  
  

  for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; sudo /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; done

  
  
  
  

Environment

vCenter 7.x
vCenter 8.x

Cause

• Solution User Stores are missing VECS Store
• After re-creating the VECS stores some services are missing permissions in vpxd-extensions
• The vstats service is missing from vpxd-extension store when executing
  
  /usr/lib/vmware-vmafd/bin/vecs-cli store get-permissions --name vpxd-extension



Resolution

• Add the missing service to the vpxd-extension store
  
  /usr/lib/vmware-vmafd/bin/vecs-cli store permission --name vpxd-extension --user vstatsuser --grant read
• Start the vstats service
  
  service-control --start vstats