vCenter Unable to Communicate with VCF Operations License Server (Port 443) due to Duplicate IP
search cancel

vCenter Unable to Communicate with VCF Operations License Server (Port 443) due to Duplicate IP

book

Article ID: 453800

calendar_today

Updated On:

Products

VCF Operations

Issue/Introduction

 

  • Attempts to attach a VCF Operations license to a vCenter Server fail.

  • The License Server reports a "nominal" state and shows as disconnected within the VCF Operations (VCF-O) UI.

  • Basic network connectivity from the vCenter Server shell to the License Server appears functional (ping and nslookup succeed).

  • Stateful TCP connections from the vCenter Server to the License Server fail. Running curl -kv https://<license_server_ip> or attempting an openssl connection on port 443 results in dropped packets, connection timeouts, or connection refused errors.

  • No intermediate network firewalls, local OS firewalls, or proxies are blocking traffic between the appliances.

 

Environment

VCF 9.x

vCenter Server 9.x

VCF License Server Appliance 9.1

 

Cause

This issue is caused by a duplicate IP address assignment on the network. Another device or virtual machine has inadvertently been assigned the same IP address as the VCF-O License Server.

When a duplicate IP exists on the same subnet, it causes ARP (Address Resolution Protocol) cache instability, commonly referred to as MAC address flapping, on the upstream network switches.

Why Ping Works:

Stateless or single-packet traffic, such as an ICMP echo request (ping), may intermittently succeed depending on which device's MAC address is currently active in the switch's ARP table.

Why Port 443 Fails:

A TLS handshake over port 443 requires a sustained, stateful TCP connection. During the multi-packet handshake, if the network switch updates its ARP table to the conflicting device's MAC address, the TCP packets are routed to the wrong destination mid-stream. This immediately breaks the TCP sequence, causing the connection to hang, drop, or be refused.

Resolution

To resolve this issue, the IP conflict must be eliminated so that stateful TCP traffic can route correctly to the License Server:

1. Identify the Conflict: Use network switch ARP tables, IPAM software, or perform an arp -a check from a device on the same subnet to identify the conflicting MAC address currently claiming the License Server's IP.

2. Re-IP the Conflicting Device: Assign the conflicting device to a new, unused IP address, or shut down the conflicting device if it was deployed in error.

3. Clear ARP Cache: Depending on the environment, you may need to clear the ARP cache on the local router/switch, or simply wait a few minutes for the MAC address flap to age out.

4. Verify Application Connectivity: Re-run curl -kv https://<license_server_ip> from the vCenter shell to verify the TLS handshake completes successfully.

5. Re-attempt Licensing: Proceed with attaching the VCF-O license to the vCenter Server.

Additional Information

Additional resources to check:

kb 445428 - VCF Operations 9.1 fails to detect License Server due to DNS Resolution

kb 441251 - adding-license-server-in-vcf-operations

kb 440982 - Licenses could not be assigned to vCenters