SSL Certificate Error: Syslog client disconnected due to a SSL handshake problem

Products

• VCF Operations

Issue/Introduction

In a VCF Operations 9.x environment, the VCF Operations for Networks platform appliance intermittently or daily fails to forward syslog messages to VCF Operations for Logs.

The following symptoms are observed:

• Daily email notifications indicating a syslog SSL handshake problem. The email notification states: Syslog client [Component_FQDN] disconnected due to a SSL handshake problem. This may be a problem with the SSL Certificate or with the Network Time Service. In order for VMware Cloud Foundation Operations for Logs to accept syslog messages over SSL, a certificate that is validated by the client is required and the clocks of the systems must be in sync.
• In the VCF Operations for Logs server-side runtime.log, the following warning appears: [Syslog ssltcp disconnect by [HOSTNAME] from port [PORT] : Remote host terminated the handshake] Syslog client [HOSTNAME] disconnected due to a SSL handshake problem. This may be a problem with the SSL Certificate or with the Network Time Service.
• In the VCF Operations for Networks client-side /var/log/loginsight-agent/liagent.log, the following errors are present: SSL fatal alert: unknown CA CertX509:65 | Cannot open certificate file /var/lib/loginsight-agent/cert/[HOSTNAME].crt for read: Permission denied
• When attempting to transition to the CFAPI protocol, the agent may report: Transport error while trying to connect to '[HOSTNAME]': Problem with the SSL CA cert (path? access rights?) : Error code:77

Environment

VCF Operations 9.0.x
VCF Operations for Networks 9.0.x
VCF Operations for Logs 9.0.x
VCF Operations for Logs Agent 9.0.x

Cause

This issue can be caused by two primary factors:

1. File System Permissions: The Log Insight agent daemon (liagentd) on the VCF Operations for Networks appliance lacks sufficient read permissions for its locally cached certificate files located in /var/lib/loginsight-agent/cert/.
2. Configuration Conflict: A duplicate or stale integration instance exists in the VCF Operations UI, configured with the legacy Syslog protocol (Port 1514) instead of the modern CFAPI protocol (Port 9543). This prevents the correct synchronization of agent settings.

Resolution

To resolve this issue, perform a configuration cleanup and correct the log forwarding protocol in the VCF Operations UI.

1. Log in to the VCF Operations UI as local admin.
2. Navigate to Infrastructure Operations > Configurations > Log Collection.
3. Identify the VCF Operations for Networks instance(s).
4. If unexpected duplicate instances exist, Delete the inactive or "Stopped" adapter from Administration > Integrations.
   
   NOTE: The Networks Adapter is created automatically when VCF Operations for Networks component is installed and is named 'VCF OPS NI_vrni-platform'. Any manual modifications or deletion of this original instance may trigger a duplicate / new instance and prevent Log Collection status to show “Log Collection is not enabled” and you are unable to edit. If you experience this issue, review  VCF Operations, Log Collection Configuration Page Fails to Load for VCF Operations for Networks Adapter for more information before proceeding with the remaining steps.
   
   
5. Select the Active adapter and click Edit.
6. Change the Protocol from Syslog (Port 1514) to CFAPI (Port 9543).
7. To address the "Error code: 77" and local path resolution issues, use the ssl_accept_any=yes flag within the agent's /var/lib/loginsight-agent/liagent.ini config file for SSL communication
   
   
   • See VCF Operations for Logs Agent is unable to connect securely to Cloud Proxy Log Forwarder (KB 415041)
8. Save the configuration.
9. Log in to the VCF Operations for Networks CLI via console or SSH and restart the agent daemon to force a configuration sync: systemctl restart liagentd
10. Verify the connection by monitoring the /var/log/loginsight-agent/liagent.log for error messages and confirm in VCF Operations: Infrastructure Operations > Configurations > Log Collection shows 'Active'

Additional Information

For further details regarding agent transport errors when connecting to a Cloud Proxy or Log Forwarder, see:

• VCF Operations for Logs Agent is unable to connect securely to Cloud Proxy Log Forwarder (KB 415041)

For further details about the duplicate adapter instance and inability to edit from Log Collection page, see:

• VCF Operations, Log Collection Configuration Page Fails to Load for VCF Operations for Networks Adapter