VCF Automation Appliance cluster health shows as degraded due to duplicate network configuration

Products

• VCF Automation
• VCF Operations

Issue/Introduction

In VCF Operations 9.0.x (formerly Aria Operations), the VCF Automation Appliance cluster may intermittently report a Critical alert for Degraded Health.

Symptoms include:

• Alerts occurring at inconsistent intervals (every 10 minutes to 3 hours).
• Alert Details:
        Type: Administrative Alerts
        Sub-Type: Availability
        State: Critical
        Object Type: AutomationAdapter Instance
        Name: <VCF Automation_instance_name>
        Info: <VCF Automation_instance_name> AutomationAdapter Instance is acting abnormally since...
        Alert Definition Name: Adapter instance Object is down
        Alert Definition Description: Adapter instance Object is reported as down. This may be caused by adapter configuration or connectivity issues.
• Cluster health flapping between Healthy and Degraded.
• Logs in the automation-adapter.log show HTTP 400 Bad Request exceptions:

ERROR AutomationAdapter ... Exception while quering /vcf/virtualDatacenterStoragePolicies?page=1&pageSize=128 javax.ws.rs.BadRequestException: HTTP 400 Bad Request

Environment

• VMware Cloud Foundation (VCF)
• VCF Operations 9.0.x (formerly Aria Operations)
• VCF Automation Appliances (Aria Automation)

Cause

This issue occurs when a vestigial or "stray" Automation Appliance exists in the vCenter inventory that is not registered in the VCF Ops cluster but shares the same network configuration (IP address/MAC address) as an active node. This results in intermittent network contention.

Resolution

To resolve this issue, you must identify and decommission the duplicate appliance:

1. Log in to the vSphere Client.
2. Navigate to the inventory and locate the VCF Automation Appliances.
3. Compare the list of VMs in vCenter against the nodes registered in your VCF Ops/Aria Operations cluster configuration.
4. Identify any "stray" appliance that is powered on in vCenter but not listed as a member of the cluster.
5. Verify the network settings of the stray appliance to confirm it matches an existing cluster node.
6. Power off the duplicate appliance.
7. Monitor the cluster health in VCF Ops for 24–48 hours to ensure the alerts have stopped.
8. Once stability is confirmed, delete or reconcile the duplicate VM from the vCenter inventory.