HCX NE/IX appliances are not sending logs to remote syslog server

Products

• VMware HCX

Issue/Introduction

• The HCX Manager acts as a relay server, facilitating the transmission of logs from the Fleet Appliances (IX & NE) to the designated remote syslog server.
• Despite proper configuration on the HCX Admin UI (port 9443), syslog logs from the Fleet Appliances are not being sent to the remote syslog server.
• When reviewing HCX-NE or IX logs you would observe something similar in /var/log/messages. Below log message indicates that the HCX-NE or IX appliance is attempting to send logs to the remote syslog server, but the connection is being blocked, due to the iptables rules on the HCX Manager preventing traffic on port 5140.
  

  <43>1 2024-10-08T14:30:17+00:00 <Appliance-name-NE-I#> syslog-ng 541 - - Syslog connection failed; fd='13', server='AF_INET(###.###.#.##:5140)', error='Connection timed out (110)', time_reopen='10'
  <43>1 2025-05-01T14:08:14+00:00 <Appliance-name-IX-I#> syslog-ng 535 - - Syslog connection failed; fd='19', server='AF_INET(###.###.#.##:5140)', error='Connection timed out (110)', time_reopen='10'
  

   

Environment

VMware HCX (versions earlier to 4.11)

Cause

HCX Manager currently has iptables rules in place that block TCP port 5140, which is required for the HCX Manager to receive logs from the Fleet appliances for remote syslog.

Resolution

• This issue is resolved in VMware HCX 4.11, available at Broadcom downloads.
• If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.

Additional Information

VMware HCX 4.11 Release Notes

• To check HCX-NE VM logs : Login to HCX Manager admin console >> ccli >> list >> go [NE_Appliance] >> ssh
• To check HCX-IX VM logs : Login to HCX Manager admin console >> ccli >> list >> go [IX_Appliance] >> ssh