Install & upgrade failures of VMware Aria Automation when using Windows Server 2012 Domain Controllers

Products

• VMware Aria Suite

Issue/Introduction

 

• The installation request fails in Aria Suite Lifecycle with message:

Error Code: LCMVRAVACONFIG590004
First boot check failed on VMware Aria Automation. Check if all the network details provided are valid.
VMware Aria Automation VA deployment failed at first boot check on host name < FQDN of VRA >. If first boot check failed due to NTP setting, ensure provided NTP server(s) - are valid. If invalid, retry the task with valid NTP server(s) in the retry parameter 'ntp' by setting 'deleteAlreadyCreatedVM' to 'true'. To configure more than one NTP server, provide comma(,) separated values for 'ntp'.

• The /var/log/bootstrap/firstboot.log contains errors similar to

Running check single-aptr
make: *** [/opt/health/Makefile:38: single-aptr] Error 1
make: Target 'firstboot' not remade because of errors.
Running check eth0-ip

Running check non-default-hostname

Running check single-aptr
make: *** [/opt/health/Makefile:38: single-aptr] Error 1
make: Target 'firstboot' not remade because of errors.
2023-07-19 21:43:32Z Script /etc/bootstrap/firstboot.d/02-setup-kubernetes failed, error status 124

Environment

VMware Aria Automation 8.x

Cause

• The error can occur on Windows Server 2012 Domain controllers with EDNS enabled or if there are duplicate DNS records for the Aria automation appliances.
• As part of installation & upgrades Aria Automation calls a health script located under /opt/health/Makefile.The relevant check for hostname resolution contained in this script is as follows:
  
  single-aptr: eth0-ip
          $(begin_check)
          echo Check the ip address if eth0 resolves only to a single hostname
          [ 1 -eq $$(/usr/bin/dig +noall +answer +noedns -x $$( iface-ip eth0 ) |  grep "PTR" | wc -l ) ]
          $(end_check)
  
  
• This health check will fail if it does not receive the expected response from the DNS server:
• The command /usr/bin/dig +noall +answer +noedns -x <Aria automation IpAddress> should return a single PTR record as per below:
  
  
  
• This error may also arise if the /etc/hosts file is missing the following details:
  127.0.0.1 localhost 
  127.0.0.1 vra-k8s.local

Resolution

To resolve the issue, ensure that the Domain Name System (DNS) contains only a single valid forward and reverse lookup entry for the Aria Automation appliances and that the /etc/hosts file is correctly configured. On older Windows Server 2012 domain controllers, either upgrading or disabling EDNS has helped the installation proceed. However, Before considering this change please contact Microsoft support to understand the full implications.

Additional Information

• This error can also be displayed when attempting to upgrade Aria Automation and FIPS mode isn't enabled on the Aria Suite lifecycle appliance. 
• If the upgrade fails with this error but you still have access to the Aria Automation appliance and all of the service pods are up and running, open an SSH to the Aria Automation appliance and run command vracli version to see what Aria Automation shows as the current installed code. 
• If Aria Automation shows the target code installed, this is most likely due to a break in communication between Aria suite lifecycle and Aria Automation.  
• Per KB, VMware Aria Suite Lifecycle 8.14 Patch 1 Day 2 operations fail for VMware Aria Automation with error code LCMVRAVACONFIG590024, since Aria Automation version 8.14.1 FIPS mode must be enabled on the Aria suite lifecycle to ensure proper connectivity and communication between Aria suite lifecycle and Aria Automation. 
  
  
• In the event of this error being triggered during an upgrade of Aria Automation, please check and ensure FIPS mode is enabled on the Aria suite lifecycle. Once enabled, trigger an inventory sync of the Aria Automation appliance.
• If the inventory sync completes successfully, the correct version of Aria Automation should now be displayed.