VCHA deployment fails with error: Verify if management interface (nic0) is configured correctly and is reachable

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• Deploying VMware vCenter Server VCHA might fail if DNS lookup fails
• In vpxd.log files you will see entries similar to:

YYYY-MM-DDTHH:MM:SS.322Z warning vpxd[7FCAC9CB9700] [Originator@6876 sub=VpxProfiler opID=EventManagerProcessJobs-5a8c4936] EventManagerProcessJobs [TotalTime] took 30000 ms
YYYY-MM-DDTHH:MM:SS.700Z error vpxd[7FCAC95AB700] [Originator@6876 sub=vpxUtil opID=ConnectionIpSettingsPageMediator-validate-59588-ngc:70004751-ad] /usr/bin/python failed with error [1] and output [Traceback (most recent call last):
--> File "/usr/lib/vmware-vcha/scripts/ifaces.py", line 235, in <module>
--> main()
--> File "/usr/lib/vmware-vcha/scripts/ifaces.py", line 223, in main
--> ifaceInfos = getIfaceInfo(iface, skipResolveOpt)
--> File "/usr/lib/vmware-vcha/scripts/ifaces.py", line 164, in getIfaceInfo
--> vchaIp, vchaFqdn = iputil.getIpFqdnForEth0(skipResolveOpt)
--> File "/usr/lib/vmware-vcha/scripts/iputil.py", line 200, in getIpFqdnForEth0
--> raise Exception('Unable to get hostname resolved IP address for eth0')
--> Exception: Unable to get hostname resolved IP address for eth0
--> ]
YYYY-MM-DDTHH:MM:SS.700Z info vpxd[7FCAC95AB700] [Originator@6876 sub=vpxLro opID=ConnectionIpSettingsPageMediator-validate-59588-ngc:70004751-ad] [VpxLRO] -- FINISH task-699
YYYY-MM-DDTHH:MM:SS.700Z info vpxd[7FCAC95AB700] [Originator@6876 sub=Default opID=ConnectionIpSettingsPageMediator-validate-59588-ngc:70004751-ad] [VpxLRO] -- ERROR task-699 -- FailoverClusterConfigurator -- vim.vcha.FailoverClusterConf
igurator.prepare: vim.fault.InvalidState:
--> Result:
--> (vim.fault.InvalidState) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = (vmodl.LocalizableMessage) [
--> (vmodl.LocalizableMessage) {
--> key = "com.vmware.vim.vcha.error.invalidFailoverNicConfig",
--> arg = <unset>,
--> message = <unset>
--> }
--> ]

Environment

VMware vCenter Server Appliance 6.5.x
VMware vCenter Server Appliance 6.7.x
VMware vCenter Server 7.0.x
VMware vCenter Server Appliance 6.0.x

Cause

vCenter HA is failing to resolve the vCenter hostname during the initial configuration of HA. 

Some reasons for this failure could be:

• TCP connections are blocked between vCenter and DNS. 
• The appliance has a mixed case hostname.
• DNS reverse lookup is not working properly.
• Hostname is set to localhost.

Resolution

This is a known issue. Currently, there is no resolution.

Workaround:
To work around this issue, ensure that DNS requests using TCP are allowed between the VCSA and its DNS servers.

1. Make sure forward and reverse lookup are resolving without timeout.
2. Verify network settings by running /opt/vmware/share/vami/vami_config_net.
3. Ensure that there no DNS Port block (Port 53) to DNS server. Run the following command on the vCenter Appliance:
   /usr/bin/host -v -W 30 -T <ip of DNS>
   
   
   Example:
   /usr/bin/host -v -W 30 -T 1xx.yy.zz.ww
   -T corresponds to TCP packets to DNS
   
4.  Verify if the initial HA startup script runs with TCP by running the following script:
   cd /usr/lib/vmware-vcha/scripts​
   python -c 'import iputil; print iputil.getIpFqdnForEth0()'
   
   
   Example output:
   ('1xx.yy.zz.ww', 'vc.mydomain.org')
   
   
   If your vCenter is a later version of 6.7, or 7.X (these builds use Python v3, earlier versions use Python v2)

The Python command syntax needs to be as follows:
python -c 'import iputil; print(iputil.getIpFqdnForEth0())'

If you get the following syntax error, you are using the incorrect older Python syntax for the command where the VCSA has Python v3.
 

File "<string>", line 1
    import iputil; print iputil.getIpFqdnForEth0()
                              ^
SyntaxError: invalid syntax

 

If you get error below, enable blocked ports on the firewall.

Traceback (most recent call last):

  File "<string>", line 1, in <module>
  File "iputil.py", line 200, in getIpFqdnForEth0
    raise Exception('Unable to get hostname resolved IP address for eth0')
Exception: Unable to get hostname resolved IP address for eth0.

After the connection succeeds to DNS, the HA deployment should work. 

Additional Information

This can be also an error for DNS reverse lookup is not working properly.

YYYY-MM-DDTHH:MM:SS.025+01:00 error vpxd[06414] [Originator@6876 sub=vpxLro opId=590b2054-####-####-####-########379] [VpxLRO] Unexpected Exception: N3Vim5Fault12InvalidState9ExceptionE(Fault cause: vim.fault.InvalidState
--> )
--> [context]zKq7AVECAQAAADx5MwEYdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TKByewHAWxpYnZpbS10eXBlcy5zbwCBc/0HAYGVcQgBAe9064LqayQBdnB4ZACCe2wkAYKrhCIBgq2UIgGCDMwiAYJHMiMBgmuVYAGCrJZgAYLapWABgj3TXwGCCH1gAQDnWiMAj7AjAFx1NwOHfwBsaWJwdGhyZ
WFkLnNvLjAABA82D2xpYmMuc28uNgA=[/context]
YYYY-MM-DDTHH:MM:SS.030+01:00 error vpxd[06414] [Originator@6876 sub=FailoverClusterOperator opId=590b2054-cfc2-4473-9b76-5dec9c8dc379] Exception: N5Vmomi11MethodFault9ExceptionE(Fault cause: vim.fault.InvalidState
--> )
--> [context]zKq7AVECAQAAADx5MwEPdnB4ZAAAnuk3bGlidm1hY29yZS5zbwAAV4ksANV8LQC7+TKBmRwlAXZweGQAgbeWYAGBbplgAYEcpmABgT3TXwGBCH1gAQDnWiMAj7AjAFx1NwKHfwBsaWJwdGhyZWFkLnNvLjAAAw82D2xpYmMuc28uNgA=[/context]
YYYY-MM-DDTHH:MM:SS.032+01:00 info vpxd[06414] [Originator@6876 sub=vpxLro opId=590b2054-####-####-####-########379] [VpxLRO] -- FINISH task-603408
YYYY-MM-DDTHH:MM:SS.032+01:00 info vpxd[06414] [Originator@6876 sub=Default opId=590b2054-####-####-####-########379] [VpxLRO] -- ERROR task-603408 -- Datacenters -- com.vmware.vcenter.vcha.cluster.deploy: vim.fault.InvalidState:
--> Result:
--> (vim.fault.InvalidState) {
-->    faultCause = (vmodl.MethodFault) null, 
-->    faultMessage = (vmodl.LocalizableMessage) [
-->       (vmodl.LocalizableMessage) {
-->          key = "com.vmware.vcha.error.unresolvedFailoverIp", 
-->          arg = <unset>, 
-->          message = <unset>
-->       }
-->    ]
-->    msg = ""
--> }
--> Args:
-->