Cisco ACI based environment conflicting with VIP / ILB access
search cancel

Cisco ACI based environment conflicting with VIP / ILB access

book

Article ID: 315988

calendar_today

Updated On:

Products

VCF Operations VMware Cloud Foundation

Issue/Introduction

  • The Administration > Management > System Monitor page intermittently shows Failed to load resources and agents are reporting Disconnected in a Cisco ACI based environment.

  • The cluster VIP (Virtual IP) / ILB (Integrated Load Balancer) is not reachable for ingestion, however ingestion sent directly to a node succeeds.

  • The /var/log/loginsight-agent/liagent_latest-date.log contains the following error

    • Transport error while trying to ingest SSL connect error

  • The application web page loads only sometimes and other times it does not load when accessing it using the VIP / ILB. However, you can access the cluster using the node IPs.

  • Issues logging in and getting data from the system when using the VIP / ILB



Environment

Aria Ops for Logs / vRLI / Log insight: 4.x, 8.x, 9.0.2

VCF: 3.x, 4.x, 5.2, 9.0

Cause

  • The Aria Operations for Logs load balancer uses a Direct Server Return (DSR) configuration. By default, DSR does not work in Cisco ACI because of data-plane IP learning.

  • The L4-L7 Virtual IPs option was introduced in Cisco Application Policy Infrastructure Controller (APIC) Release 1.2(1m).

  • This option disables data-plane IP learning for the specific DSR virtual IP address.

  • Failure to disable IP learning for the DSR virtual IP address will result in IP endpoint flapping between different locations in the Cisco ACI fabric.

Resolution

  1. Navigate to Tenant > Application Profiles > Application EPGs / uSeg EPGs.

  2. Ensure that GARP is set to Enabled for the segment on CISCO ACI.

    1. The default GARP setting is "disabled."

    2. If applicable, set IP Data-plane Learning to no, as there is no GARP option.

  3. Log into the Aria for Logs Admin UI and identify the node that has the VIP / ILB 

  4. Restart Guest OS for all cluster nodes via vCenter

    • Note: The node holding the VIP / ILB must be the last node to be rebooted.

    • Note: Wait for each node to come online before rebooting the next node.

    • ​​​​​​Note: A service restart is not sufficient, an OS reboot is required.

Additional Information

  • For more information, see ACI Fabric Endpoint Learning White Paper.

  • In the event that this Resolution is not working for Aria Operations for Logs in Cisco based environments, the issue could be due to a rather complex ACI design with intra-EPG isolation enabled and micro-segmented (uSeg) EPGs.

  • With IP data-plane learning disabled, the fabric depends on ARP/GARP and COOP to move the VIP. In an intra-EPG isolated and micro-segmented design, those ARP/GARP signals can be proxied, suppressed, or scoped by policy (e.g., per-EPG contracts, ARP suppression on the Bridge Domain). If the new ILB owner's GARP doesn't qualify as a valid "move" in the same policy context, the leaf keeps the stale EPM binding, so the VIP appears stuck on the old node.

  • Also, because uSeg EPGs split endpoints into multiple policy scopes, the "old owner" and "new owner" can be in different uSeg EPGs. ACI may hold separate endpoint state per scope; without the right GARP handling and Bridge Domain settings, the old record doesn't get replaced, even though traffic is flowing.

  • In such Cisco network configurations, the standard load balancer / VIP functionality in Aria Operations for Logs will not work. The only workaround is to use an external load balancer such as F5 or DNS-based balancing.

  • NOTE: The configuration of external load balancing options is out of the scope of Broadcom support.

  • Please see Microsegmentation with Cisco ACI for more details and limitations.

    "Configuring a Layer 4 to Layer 7 virtual IP (VIP) address under microsegmented EPGs or their corresponding base EPGs is not supported."



Powered by Wolken Software