VCF License Server unable to connect to VCF Operations with proxy configured
search cancel

VCF License Server unable to connect to VCF Operations with proxy configured

book

Article ID: 441747

calendar_today

Updated On:

Products

VCF Operations

Issue/Introduction

After deploying or upgrading to VMware Cloud Foundation 9.1, the VCF License Server appliance fails to connect to VCF Operations with below symptoms: 

    • The VCF License Server splash screen (VM Console) displays error: VCF Ops Hosts: Not Configured
    • The connectivity status in VCF Operations shows: Connectivity: DISCONNECTED
    • VCF Operations hosts may appear by FQDN in the console, but the synchronization fails.
    • License server is found to be configured with a proxy server.
    • Viewing the VM console screen of the license server shows: 




VCF License Server Status Dashboard
Last Updated: YYYY-MM-DD HH:MM:SS
Current LS Version : 9.1.#.#.##########
---------------------------------------------------------------
Service Status: Nominal
---------------------------------------------------------------
UCF Ops Hosts: Not configured
Connectivity: DISCONNECTED
IP Address: ###.###.###.###
Hostname: ####.####.####    (Resolvable)
DNS Servers:
Last Upgrade Check: YYYY-MM-DD HH:MM:SS
Upgrade Check Status: OPS_INFO_NOT_FOUND
Upgrade Status: NEVER_UPGRADED

Environment

VMware Cloud Foundation 9.1

Cause

VCF License Server is not required to communicate with the internet and has no external network connection requirements. It requires port 443 to communicate with vCenter and VCF Operations.

Resolution

Perform these checks to validate if the current configuration of License Server can be corrected or if a redeployment is necessary:

  1. Ensure the License Server has direct network access to vCenter and VCF Operations on port 443. 
  2. Validate that DNS forward and reverse lookups for the License Server FQDN are functioning correctly.
  3. Ensure that no proxy server is configured for internal traffic, as the License Server does not require internet connectivity and should not route internal management traffic through a proxy. 

Follow the sections below to verify connectivity, synchronize time, and resolve configuration blocks across the environment.

  1. Verify License Server Power Status: 
    1. Navigate to Inventory in the vSphere Client.
    2. Select the cluster where the license server virtual machine (VM) runs.
    3. Select the license server VM and verify the Power Status shows Powered On.
    4. If the VM is powered off, select Actions > Power On.
  2. Synchronize Component Time (NTP):
    1. Time drift exceeding 30 seconds between the license server, vCenter Server, and VCF Operations causes authentication and synchronization failures.
    2. Open SSH sessions to the ESXi host running the license server, the vCenter Server, and the VCF Operations appliance.
    3. Run the following command in quick succession on each component to check the current UTC time:

      date -u +"%Y-%m-%dT%H:%M:%SZ"

    4. If timestamps differ by more than 30 seconds, configure an identical NTP server across all components:
      1. Configure NTP on the vCenter Server.
      2. Configure NTP on the ESXi host hosting the license server VM.
      3. Reconfigure NTP on the VCF Operations cluster nodes.
  3. Verify vCenter Server Connectivity to the License Server:
    1. Check the status of current license assignments:
      1. If a license was previously assigned, verify if the vCenter Server user interface displays an alert banner indicating the connection to the license server has been lost.
      2. If a license has not yet been assigned, proceed to validate direct network layer communication from the command line.
    2. Access the vCenter Server appliance via SSH.
    3. Validate TCP port 443 connectivity to the license server by running:

      curl -v -k https://<license-server-IP>

      Alternatively, run:

      openssl s_client -connect <license-server-IP>:443

    4. If the connection fails or blocks, add an explicit firewall rule to allow TCP port 443 traffic from the vCenter Server IP address to the license server IP address.

    5. Test forward and reverse DNS resolution from the vCenter Server using nslookup:

      nslookup <license-server-hostname> <vCenter-DNS-IP>
      nslookup <license-server-IPAddress> <vCenter-DNS-IP>

      1. To identify the vCenter Server DNS IP address: Log in to the vCenter Server Appliance Management Interface (VAMI) at https://<vcenter_IP_or_FQDN>:5480. Navigate to Networking, and under Networking Settings, click Edit > Next, then expand Hostname and DNS to locate the configured DNS server entries.

      2. To identify the license server hostname and IP address: Locate the license server VM in the vCenter Inventory, launch the Remote Console, and retrieve the values listed under Hostname and IP Address.

    6. If the lookups fail, create an A record for the license server hostname in the primary DNS server utilized by vCenter Server.

  4. Verify License Server Connectivity to VCF Operations:

    1. In VCF Operations, navigate to Manage > Licensing > Licenses & Registration.

    2. Under Registration and License Server Status, click Manage > License Servers to inspect the Connectivity to VCF Operations column.

      Note: The status may falsely display as connected in this user interface if the Licensing Server vApp was deployed with a valid IP address but possesses missing or invalid DNS records.

    3. If the status shows Not Connected or if the server does not appear after enrollment, open an SSH session to a host on the same management subnet as the license server and test port 443 access to VCF Operations:

      curl -v -k https://<VCF-Operations-IP>

    4. If blocked, configure an allow rule on the network firewall for TCP port 443 from the license server subnet to the VCF Operations IP address.

    5. Verify DNS resolution for the VCF Operations FQDN from the license server's designated DNS server:

      nslookup <VCF-Operations-hostname> <license-server-DNS-IP>

    6. If the returned IP address is incorrect or returns NXDOMAIN, update the A record in the DNS server, or correct the vApp DNS options by powering off the license server VM and adjusting Configure > vApp Options > Domain Name Servers.

  5. Redeploy the VCF License Server without a proxy configured via the following steps when above validations are done:

    1. Download the licenser server appliance OVA from the Broadcom download portal.
    2. Power off the license server VM from the vCenter UI.
    3. Remove from disk.
    4. Deploy a new license server with the OVA. Refer: Add and Register an Additional License Server

Additional Information


To receive updates on this issue, please Subscribe to this article using the instructions in KB#275360.

Powered by Wolken Software