Deploying vCenter High Availability fails with error: A general system error occurred: Failed to run pre-setup

Products

• VMware vCenter Server

Issue/Introduction

• Deploying vCenter High Availability fails with error: A general system error occurred: Failed to run pre-setup
  
  Logs present in /var/log/vmware/vcha/prepare-vcha.log matches the below snippet
  • ####-##-#####:##:##Z  ERROR prepare-vcha Failed to create vcha user, stderr: You are required to change your password immediately (root enforced)
    useradd: PAM: Authentication token is no longer valid; new one required
     code: 1
    ####-##-#####:##:##Z INFO prepare-vcha *** START: Vcha pre-setup ***
####-##-#####:##:##Z  INFO prepare-vcha embedded DB Type can be vcha enabled.
####-##-#####:##:##Z  INFO prepare-vcha Found Eth1 interface with IP ##.###.##.##.###/##
####-##-#####:##:##Z INFO prepare-vcha Completed Vcha Network config saved to vchaeth.cfg.
####-##-#####:##:##Z  INFO prepare-vcha vcha.aware file created
####-##-#####:##:##Z  INFO prepare-vcha Opening firewall ports for VCHA...
####-##-#####:##:##Z  INFO prepare-vcha Reloading firewall...
####-##-#####:##:##Z  INFO prepare-vcha Saving clean pg_hba.conf...
####-##-#####:##:##Z  INFO prepare-vcha Saving clean pg-firewall..
####-##-#####:##:##Z  INFO prepare-vcha Completed setup of firewall for vcha.
####-##-#####:##:##Z  INFO prepare-vcha Installing the vcha-hacheck boot script...
####-##-#####:##:##Z  INFO prepare-vcha Setting interface eth0 to manual mode...
####-##-#####:##:##Z  INFO prepare-vcha Completed updating /etc/systemd/network/10-eth0.network to manual
####-##-#####:##:##Z INFO prepare-vcha Successfully set interface eth0 to manual mode.
####-##-#####:##:##Z ERROR prepare-vcha Failed to create vcha user, stderr: You are required to change your password immediately (root enforced)
useradd: PAM: Authentication token is no longer valid; new one required
 code: 1
  • root@hostname [ ~ ]# chage -l root
You are required to change your password immediately (root enforced)
chage: PAM: Authentication token is no longer valid; new one required
• vCenter High Availability configuration can fail due to incorrect value or has extra dot in IP address with same Error- A general system error occurred: Failed to run pre-setup

  • Logs present in /var/log/vmware/vpxd/vpxd.log matches the below snippet

    vpxd.log
    ####-##-#####:##:##Z info vpxd[07076] [Originator@6876 sub=vpxLro opId=150309f8-ccf3-439d-b107-8279c396faf3] com.vmware.vcenter.vcha.cluster.deploy: taskId task-######:######-####-####-####-############
####-##-#####:##:##Z info vpxd[06741] [Originator@6876 sub=vpxLro opId=150309f8-ccf3-439d-b107-8279c396faf3] [VpxLRO] -- BEGIN task-######-- Datacenters -- com.vmware.vcenter.vcha.cluster.deploy --
####-##-#####:##:##Z error vpxd[06741] [Originator@6876 sub=vpxUtil opId=150309f8-ccf3-439d-b107-8279c396faf3] /usr/bin/python failed with error [1] and output []
####-##-#####:##:##Z error vpxd[06741] [Originator@6876 sub=vpxUtil opId=150309f8-ccf3-439d-b107-8279c396faf3] /usr/bin/python failed with error [1] and output [Traceback (most recent call last):
-->   File "/usr/lib/vmware-vcha/scripts/ipSubnet.py", line 62, in <module>
-->     main()
-->   File "/usr/lib/vmware-vcha/scripts/ipSubnet.py", line 57, in main
-->     if matchSubnet(ip1, ip2):
-->   File "/usr/lib/vmware-vcha/scripts/ipSubnet.py", line 33, in matchSubnet
-->     sameSubnet = checkSameSubnet(ip1, ip2)
-->   File "/usr/lib/vmware-vcha/scripts/ipSubnet.py", line 22, in checkSameSubnet
-->     ipnet2 = IPNetwork(ip2, strict=False)
-->   File "/usr/lib/python3.7/ipaddress.py", line 84, in ip_network
-->     address)
--> ValueError: '##.###.###.### / ###.###..###.#' does not appear to be an IPv4 or IPv6 network
--> ]
####-##-#####:##:##Z info vpxd[06741] [Originator@6876 sub=FailoverClusterConfigurator opId=######:######-####-####-####-############] Pre-setup: vcha.aware, SSH keys and PG
####-##-#####:##:##Z error vpxd[06741] [Originator@6876 sub=vpxUtil opId=150309f8-ccf3-439d-b107-8279c396faf3] /usr/bin/python failed with error [1] and output [logs available at: /var/log/vmware/vcha
--> Traceback (most recent call last):
-->   File "/usr/lib/vmware-vcha/scripts/preSetup.py", line 479, in setupVchaEthCfg
-->     witnessIp, selfGatewayIp, peerGatewayIp, checkIps)
-->   File "/usr/lib/vmware-vcha/scripts/createVchaEthCfg.py", line 185, in createVchaEthCfg
-->     wtnsIp, node1GwIp, node2GwIp)
-->   File "/usr/lib/vmware-vcha/scripts/createVchaEthCfg.py", line 99, in checkAllIPFormat
-->     raise ValueError('node2Ip - %s: %s' % (err, IP_ERRS.get(err)))
--> ValueError: node2Ip - 6: Expected valid Netmask or CIDR prefix.
--> During handling of the above exception, another exception occurred:
####-##-#####:##:##Z info vpxd[06741] [Originator@6876 sub=FailoverClusterOperator opId=150309f8-ccf3-439d-b107-8279c396faf3] com.vmware.vcenter.vcha.cluster.deploy Request:
-->     },
-->     "deployment": "AUTO",
-->     "passive": {
-->         "failover_ip": null,
-->         "ha_ip": {
-->             "default_gateway": null,
-->             "dns_servers": null,
-->             "ip_family": "IPV4",
-->             "ipv4": {
-->                 "address": "'##.###.###.###",
-->                 "prefix": null,
-->                 "subnet_mask": "###.###..###.#"
-->             }

Environment

VMware vCenter Server Appliance 6.x

VMware vCenter Server Appliance 7.x

Cause

The deployment fails because of root password being expired.

The deployment fails when the Subnet IP address is incorrect or has additional extra dot or value (example below)

• Script at /usr/lib/vmware-vcha/scripts/createVchaEthCfg.py
  • The failure is triggered at line 99, where the subnet mask for the Passive node IP is being validated. The value passed (###.###..###.#) contains an extra dot, which causes the script to raise a ValueError: 99    raise ValueError('node2Ip - %s: %s' % (err, IP_ERRS.get(err)))
  • This corresponds with the deployment spec in the log snippet below:
    • "passive": {
  "ha_ip": {
    ...
    "ipv4": {
    "address": "##.###.###.###",
      "prefix": null,
      "subnet_mask": "###.###..###.#"
    }
  }
}

Resolution

Resolution 1: Authentication Issue Observed in prepare-vcha.log

If an authentication error is encountered during VCHA deployment and the log indicates the root password needs to be changed, follow the steps below:

1. Check the root password status:
   • root@hostname [ ~ ]# chage -l root
     You are required to change your password immediately (root enforced)
     chage: PAM: Authentication token is no longer valid; new one required
2. Reset the root password:
   • root@hostname [ ~ ]# passwd
3. Verify the password status again:
   • root@hostname [ ~ ]# chage -l root
     Last password change                                    : Oct ##, ####
     Password expires                                           : Jan ##, ####
     Password inactive                                          : never
     Account expires                                             : never
     Minimum number of days between password change          : 0
     Maximum number of days between password change         : 90
     Number of days of warning before password expires           : 7
4. Once the password is successfully updated, retry the VCHA deployment.

Resolution 2: Invalid or Malformed IP Address Observed in vpxd.log

If the vpxd.log indicates an invalid IP address (e.g., extra dots or incorrect value in IP or Subnet), restart the required services:

1. SSH to the vCenter Server and run the following commands:
   • service-control --restart --vsphere-ui
   • service-control --restart --vmware-vpxd
2. Once the services have restarted, retry the VCHA deployment with correct entries in IP Address/Subnet and follow through the steps in the official documentation: Configure vCenter HA With the vSphere Client

Additional Information