Adding a Secondary DNS Server Fails in VMware Cloud Foundation SDDC Manager due to Blocked ICMP and Immutable Network Files

Products

• VMware Cloud Foundation

Issue/Introduction

• Attempts to add an alternate or secondary DNS server via the SDDC Manager UI or per API workflow fail. The process generates validation and configuration errors.
• During the precheck phase, the following error is observed due to ICMP traffic restrictions:

  Failed to validate NTP configuration. Validate input specification as new DnsConfiguration to VMware Cloud Foundations

• When reviewing the API workflow tasks, the validation subtask fails with:

  Message: Error occurred while validating details of entities.
  Remediation Message: Please make sure all the entities are reachable and ACTIVE.
  Reference Token: ######

• When the validation task is bypassed, the configuration task (ConfigureDnsServersOnSddcManagerAction) subsequently fails with:

  Message: Error occurred while configuring DNS server(s) primary:##.##.##.##, secondary:##.##.##.## for SDDC Manager.
  Remediation Message: Please check the connectivity to DNS server(s) primary:##.##.##.##, secondary:##.##.##.## from SDDC Manager.
  Reference Token: ######

• When reviewing the /etc/ntp.confit diverges from the NTP configuration seen in the SDDC Manager UI (for example the UI shows an NTP server IP, but that IP is missing from the file).
• Curl is working fine for both primary and secondary DNS.
• The issue persists across SDDC Manager reboots

Environment

VMware Cloud Foundation 5.2.x

Cause

This issue is caused by two compounding factors within the environment and the SDDC Manager appliance:

1. ICMP traffic is blocked by network policy, which prevents the SDDC Manager from successfully validating entity reachability during the workflow.
2. The network configuration file on the SDDC Manager (/etc/systemd/network/10-eth0.network) has been assigned the immutable attribute (+i). This attribute prevents the automated workflow from writing the new DNS configuration to the file.
   The immutable flag can be verified by running the following command on the SDDC Manager appliance:

   root@<SDDC Manager Hostname> [ ~ ]# lsattr /etc/systemd/network/10-eth0.network
   ----i---------e------- /etc/systemd/network/10-eth0.network

Resolution

Contact Broadcom support for assistance with solving this issue.