Lifecycle UI connection fails in VCF Operations Fleet Manager

Products

• VCF Operations

Issue/Introduction

You see a discrepancy in status reporting between the VCF Operations UI and the Fleet Manager Lifecycle UI. The VCF Operations UI indicates that Fleet Management is connected, but the Fleet Manager Lifecycle view shows the status as:

VCF Operations Fleet Management is Not Ready

Environment

VCF Operations 9.0

VCF Operations 9.0.1

Cause

The secure token, which VCF Operations issues to the Fleet Manager component for connectivity, is not automatically rotating as expected before its 90-day expiration. When the token expires, the Lifecycle UI loses connection, resulting in the "VCF Operations Fleet Management is Not Ready" status.

Resolution

Restart the lifecycle service to temporarily re-establish the connection.

1. SSH to the Lifecycle Manager (LCM) node. 
2. Run the command:
   • systemctl restart vrlcm-server.service
3. Wait a few minutes for the service to settle, then refresh the Lifecycle UI in VCF Operations. 
4. If the connection fails again after a short period, proceed to the Workaround section below. 

Workaround

Use the following procedure to manually drop the expired key from the database and restart the lifecycle service.

1. Log out of the VCF Operations user interface (UI).
2. SSH to the VCF Operations Master or Replica node.
3. Open a connection to the PostgreSQL central database by running the following command:
   • su - postgres -c "/opt/vmware/vpostgres/current/bin/psql -d vcopsdb -p 5433"
4. Drop the expired key by running the following command in the database shell:
   • delete from kv_ops_signing_keys;
5. SSH to the Lifecycle Manager (LCM) node.
6. Restart the lifecycle service:
   • systemctl restart vrlcm-server.service
7. Wait a few minutes for the services to settle, then refresh the Lifecycle UI in VCF Operations.