Deployment fails at ui-toggles when there is a DNS wildcard configured
search cancel

Deployment fails at ui-toggles when there is a DNS wildcard configured

book

Article ID: 326099

calendar_today

Updated On:

Products

VMware Aria Suite

Issue/Introduction

This article provides instructions for patching the environment so the deployment may succeed.

Symptoms:
  • After deployment failure, the log /var/log/deploy.log contains messages similar to
[2021-12-23 03:08:18] ERROR Failed to upgrade release 'ui-toggles' in namespace 'prelude'
Traceback (most recent call last):
File "/opt/scripts/helm-upstall", line 319, in main
helm_upgrade(namespace, release_name, args.chart_path, overrides=overrides, values_set=args.set, values_set_string=args.set_string, timeout=upgrade_timeout, args=shlex.split(args.helm_upgrade_args))
File "/opt/python-modules/vracli/decorators.py", line 181, in wrapper
raise err
File "/opt/scripts/helm-upstall", line 225, in helm_upgrade
run_cmd(cmd)
File "/opt/scripts/helm-upstall", line 47, in run_cmd
raise RuntimeError('command failed')
RuntimeError: command failed
++ result=1
++ set +x
Exit code of install/update of release ui-toggles is 1
++ return 1
++ on_exit
++ '[' 1 -ne 0 ']'
++ echo 'Deployment failed. Collecting log bundle ...'
Deployment failed. Collecting log bundle ...
...
  • The log /services-logs/prelude/ui-toggles/console-logs/ui-toggles-fixture.log contains messages similar to 
    curl: (7) Failed to connect to identity-service.prelude.svc.cluster.local port 8000 after 2 ms: Connection refused


Environment

VMware vRealize Automation 8.6.x
VMware vRealize Automation 8.4.x
VMware vRealize Automation 8.5.x

Cause

  • An issue has been identified within the helm chart of the ui-toggles-fixture job.
    • The container, which is running the ui-toggles job, ends up with the default for k8s ndots:5 option in its resolv.conf file. When the job tries to resolve identity-service.prelude.svc.cluster.local, the resolver in the container appends the domain name of vRA. If there is a wildcard entry configured in the DNS server used by vRA, the result of the DNS query is the IP address configured in that wildcard DNS entry. This results in a Connection refused or a similar error, causing ui-toggles-fixture to fail and thus a deployment failure.

Resolution

This issue is resolved in VMware vRealize Automation 8.7 and above.

Workaround:

Prerequisites

  • root user and password
  • SSH or console access to the virtual appliance(s)

Procedure

  1. SSH into one appliance in the cluster
  2. Run the following command 
    base64 -d <<< '/Td6WFoAAATm1rRGAgAhARwAAAAQz1jM4AUNAlNdADYbyRcPoXs3RneokXCaup584kklp/S+ZakqwL5n+BL8qNtU/9u9xoEWYmjHW8oQKdYlcITmJ9MpJIkRMGaj24wkrxvSu1UTBiwElwxf45SKq5OP6zdggC7azlhkJld6jwKcE17wueiHQWR+3Y0v/ycpbwtgbsW4oRdFhVRVtved6erWyP+uCtKDsMbAwTa6kJmo6l1xs8o+QoECQceqwwtrtRdQM1zBMLrb6lDeKV3/sjZeTExz3M6ngit68+uXBIYpCT8w4G9ddK9a16UkyRDvoHjn5hm2tPEFEE5F1FRGol5pStxo90vj1JSLqid+pe2gClyd4+g5nVWtpYBswd8id30m1XWgR75OKr9r2pui0SEf4APrE8xfGWXnjI8NCnJEmx3JPMaDNhEc5vd8P8o0jDXX0h2v1Qmwg6K+Qrzf/5qsybIqCeEmIgT9FARpruqGEkUBWxemMZOWiaYghRzzrGZ3CxXO4LazOGTsKwDFpfjMfo4D5jMPRiWa+4xlsAorAcP9TGYpSN+f3Wqyw+PkGo31LPy+8b3x/Ljn9rTGxLzil59stkyp5h5krYwXKqY2pvPpBNMJHuEXaGJAs3TRuFazOwWAX73nczIO//cZLKQombezqbgWti0GkEWzaZ+GirFF9azzy5Agyp8Q1CjHTXlIqQFSkfsUWcEhtVnSjr6YQUwpnHe6GPs4a233seLCHpn3uLTR53xqChSLEaysEQwVRsxnK6X9YNV0ki66Q5flTrP3MWsBPYBUuHqFz4xk2irpHuusJQMZteeHOCh2GuAAAGBSxz32gB5TAAHvBI4KAACUn9bRscRn+wIAAAAABFla' | xz -d | bash -
  3. Run 
    /opt/scripts/deploy.sh


Additional Information

A wildcard DNS record in a zone file looks similar to this example 
*.example.com. 3600 IN MX 10 host1.example.com.
This wildcard DNS record will cause DNS lookups on domain names ending in example.com that do not exist to have MX records synthesized for them. So, a lookup for the MX record for somerandomname.example.com would return an MX record pointing to host1.example.com.

Powered by Wolken Software