Error: "General error while polling audit events. Details: Bad request: 503" on the load balancer API used for VMware Cloud Director
search cancel

Error: "General error while polling audit events. Details: Bad request: 503" on the load balancer API used for VMware Cloud Director

book

Article ID: 437975

calendar_today

Updated On:

Products

VMware Cloud Director

Issue/Introduction

  •  You will notice that VMware Cloud Director(VCD) sometimes will be responding with a HTML randomly. For example:

    </html>
    </body>
    <h3 class=""error-message""></h3>
    <h2 class=""heading"">VMware Cloud Director</h2>
    <body cds-theme=""light"">
    </head>
    <link rel=""icon"" href=""/####.ico"" type=""image/png""/>
    <link href=""/css/error.css?t="" rel=""stylesheet"">
    <link href=""/css/vcd-clarity-base.css?t="" rel=""stylesheet"" id=""vcd-clarity-base"">
    <title>VMware Cloud Director</title>
    <head>
    <html xmlns=""http://www.example.org/1999/xhtml"" xml:lang=""mul"">
    <!DOCTYPE html PUBLIC ""-//W3C//DTD XHTML 1.0 Strict//EN"" ""http://www.example.org/TR/xhtml1/DTD/xhtml1-strict.dtd"">
    </html>

  • The VCD services should not interrupted, but you will see the following error on the load balancer API used for VCD:

    General error while polling audit events. Details: Bad request: 503

  • You can find the below errors in the /opt/vmware/vcloud-director/logs/vmware-vcd-watchdog.log:

    2026-04-07 17:18:09 | INFO  | vmware-vcd-watchdog started, will perform a status check every 60 seconds
    2026-04-07 17:18:19 | INFO  | Started vmware-vcd-cell (pid=#######)
    2026-04-07 17:19:19 | INFO  | vmware-vcd-cell running [This will log at INFO every 5 status checks while vmware-vcd-cell is running]
    2026-04-07 17:19:19 | WARN  | wget failed, return code: 4, will retry in 30 seconds
    2026-04-07 17:19:49 | WARN  | wget failed, return code: 4, will retry in 30 seconds
    2026-04-07 17:20:19 | WARN  | wget failed, return code: 4, will retry in 30 seconds
    2026-04-07 17:21:49 | WARN  | wget failed, return code: 4, will retry in 30 seconds
    2026-04-07 17:22:19 | WARN  | wget failed, return code: 4, will retry in 30 seconds
    ...
    2026-04-07 17:31:50 | INFO  | vmware-vcd-cell running

Environment

VMware Cloud DIrector 10.6.1.2

Cause

The issue is caused by using the wildcard character * in the NO_PROXY list as this is not widely supported. Many libraries will fail to parse this correctly and might ignore the rule entirely for example wget and curl.
This is the reason wget is giving the API error after fetching from the vmware-vcd-watchdog.log and so are the API giving Bad request: 503. Whereas VCD services are running successfully as VCD can parse the * mentioned in the proxy.

Resolution

To resolve the issue please follow the below steps:

  1. Take a backup of the Cloud Director database before making any changes. To take the backup from the VCD VAMI, see Backup and Restore of VMware Cloud Director Appliance.
  2. ssh to the primary cell.
  3. vi /etc/sysconfig/proxy

    change this line: NO_PROXY="localhost, 127.0.0.1, *.example1.net, *.example2.net"
    to this: NO_PROXY="localhost, 127.0.0.1, .example1.net, .example2.net

  4. You need to repeat Step.3 for each VCD cells. ssh to each cell and make the changes in each of the proxy file as mentioned above.
Powered by Wolken Software