VMware vCenter Server HA replication fails due to expired VCHA user password

Products

• VMware vCenter Server

Issue/Introduction

Symptoms

• The VMware vCenter Server HA configure tab reports:
  
  A replication failure might be occurring at the moment. Automatic failover protection is disabled.
• The VMware vCenter Server HA monitor tab reports:
  
  Appliance configuration is out of sync.
  Appliance state is out of sync.
  Appliance sqlite db is out of sync.
• In the /var/log/vmware/vcha/vcha.log file on the current active node, you see entries similar to:
  
  error vcha[7FC6BE0E6700] [Originator@6876 sub=VchaUtil] Error executing command /usr/bin/rsync: exit status=[12], stdout=[], stderr=[
  --> VMware vCenter Server Appliance 6.5.0.5100
  --> Type: vCenter Server with an external Platform Services Controller
  --> WARNING: Your password has expired.
  --> Password change required but no TTY available.
  --> rsync: connection unexpectedly closed (0 bytes received so far) [sender]
  --> rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.2]
  warning vcha[7FC6BE0E6700] [Originator@6876 sub=RsyncRepl-largeFrp] Rsync failed for vmw, retrying in 8 secs
• Running the command chage --list vcha on the current active node shows that the password for the account VCHA has expired:
  
  [ ~ ]# chage --list vcha
  Last password change : Nov 14, 2016
  Password expires : Jan 13, 2017
  Password inactive : never
  Account expires : never
  Minimum number of days between password change : 1
  Maximum number of days between password change : 60
  Number of days of warning before password expires : 7

Cause

This issue occurs when VMware vCenter Server HA is enabled, a new local OS user VCHA is created, and used to perform the file replication between the current active node and the current passive node.

 

Note: You are also enabled to use RSA Key for SSH Authentication. However, even if the default user password expires, RSA Key for SSH Authentication is not possible.

 

Resolution

This issue is resolved in vCenter Server Appliance 6.5 Update 1, available at Support Documents and Downloads (broadcom.com).

 

To work around this issue, reset the VCHA user password on all three vCenter HA nodes (active, passive, and witness).

 

Important: Ensure that you perform these steps on all three vCenter HA nodes with the same password for the VCHA user.

1. Log in to each vCenter HA as root using SSH or VM Console.
2. Change to the BASH shell by running the shell command:
   
   
   Command> shell
3. Reset the password for the VCHA user using the password command:
   
   
   [ ~ ]# passwd vcha
   New password:
   Retype new password:
4. Set the VCHA user account to never expire by running this command:
   
   [ ~ ]# chage -m 0 -M 99999 vcha
5. Confirm that the VCHA user account is set to never expire by running this command:
   
   [ ~ ]# chage --list vcha
   Last password change : Jan 13, 2017
   Password expires : never
   Password inactive : never
   Account expires : never
   Minimum number of days between password change : 0
   Maximum number of days between password change : 99999
   Number of days of warning before password expires : 7

 

Notes:

• There should be no need to restart any services. Replication should now begin to succeed. It may take several minutes for all nodes to get synced.
• If the replication does not start or complete, restart the vCenter HA Passive node.