This article addresses the issue where the `hbrsrv` service in vSphere Replication fails to start. This failure prevents connectivity to the vSphere Replication service on port 8123.
When `hbrsrv` fails to start, you may encounter the following symptoms:
xxx verbose hbrsrv[04839] [Originator@6876 sub=Crypto] Loading Private Key '/etc/vmware/ssl/hbrsrv_broker_private.pem'
xxx verbose hbrsrv[04839] [Originator@6876 sub=Crypto] Loading Public Key '/etc/vmware/ssl/hbrsrv_broker_public.pem'
xxx verbose hbrsrv[04839] [Originator@6876 sub=Crypto] Computing key expiration from private key file modification time.
xxx panic hbrsrv[04839] [Originator@6876 sub=Main] HbrError stack:
xxx panic hbrsrv[04839] [Originator@6876 sub=Main] [0] Could not compute a reasonable expiration time for private key '/etc/vmware/ssl/hbrsrv_broker_private.pem'
xxx panic hbrsrv[04839] [Originator@6876 sub=Main] [1] Now: 2023-12-04T08:11:52.24508Z Creation time: 2023-05-05T03:34:26Z
xxx panic hbrsrv[04839] [Originator@6876 sub=Default]
--> Panic: Couldn't initialize broker token manager crypto!
--> Backtrace:
--> [backtrace begin] product: VMware vSphere Replication Server, version: 8.0.1, build: build-21471995, tag: hbrsrv, cpu: x86_64, os: linux, buildType: release
--> backtrace[00] hbrsrv-bin[0x00BBE95F]
--> backtrace[01] hbrsrv-bin[0x00BB4CE2]
--> backtrace[02] hbrsrv-bin[0x00CEA297]
--> backtrace[03] hbrsrv-bin[0x00CEA3B1]
--> backtrace[04] hbrsrv-bin[0x00996FD5]
--> backtrace[05] hbrsrv-bin[0x00903EC3]
--> backtrace[06] hbrsrv-bin[0x007D3F99]
--> backtrace[07] libc.so.6[0x00022B87]
--> backtrace[08] hbrsrv-bin[0x008A2FC1]
--> [backtrace end]
VMware vSphere Replication 8.x
VMware vSphere Replication 9.x
The issue is caused by discrepancies in the timestamp of the certificate files, which prevents proper computation of the key expiration.
This results in the failure of the `hbrsrv` service to initialize the broker token manager cryptography.
To resolve this issue, follow these steps:
1. Snapshot VR
2. Remove Old PEM Files
rm /etc/vmware/ssl/hbrsrv_broker_private.pem
rm /etc/vmware/ssl/hbrsrv_broker_public.pem
3. Restart the hbrsrv Service
systemctl stop hbrsrv
systemctl start hbrsrv
During this procedure, VM replication will be temporarily unavailable. Ensure to perform these steps during a maintenance window or when replication services can be paused.