Numerous “SSL Handshake failed” errors and SSH authentication failures are recorded in the vCenter logs.

Products

• VMware vCenter Server
• VMware vCenter Server 8.0

Issue/Introduction

SSL errors related to rsyslogd or nsd_ossl may be recorded in the vCenter Server logs. Additionally, SSH authentication failures may be recorded during the same timeframe.

Example logs:

vc1 rsyslogd netstream session 0x7f39#####10 from ##.##.##.## will be closed due to error [v8.2306.0 try https://www.rsyslog.com/e/2089 ]
vc1 rsyslogd SSL_ERROR_SSL Error in 'osslHandshakeCheck Server': 'error:00000001:lib(0)::reason(1)(1)'
vc1 rsyslogd nsd_ossl:TLS session terminated with remote client 'xx.xx.xx.xx': Handshake failed with error code: 1
vc1 rsyslogd nsd_ossl:OpenSSL Error Stack: error:0A000197:SSL routines::shutdown while in init
vc1 sshd[xxxx]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=xx.xx.xx.xx user=admin
vc1 sshd[xxxx]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=xx.xx.xx.xx user=user

These logs are recorded originating from a specific IP address.

Environment

 

• VMware vCenter Server 7.0
• VMware vCenter Server 8.0

 

Cause

This issue may occur when external security audit tools or vulnerability scanners are performing port scans or connection tests against the vCenter Server.

• rsyslogd and OpenSSL errors are recorded when a scanner disconnects without successfully completing the SSL handshake or attempts to connect using an invalid protocol.

• Numerous SSH authentication failure logs are recorded because scanners attempt to log in using default accounts (e.g., admin, user, factory) or random usernames.

Resolution

If these errors are not causing operational issues (such as vCenter service interruptions), verify the source using the following steps:

1. Identify the IP address recorded in the "remote client" or "rhost" fields within the logs.

2. Confirm if this IP address belongs to an internal security audit server or a monitoring server.

3. Check if the time the errors occurred matches the audit schedule (e.g., scheduled monthly execution).

If the investigation confirms that the access is from a legitimate audit tool, these errors can be ignored.

If the source IP address is unknown or is not a legitimate audit server, this may indicate a security risk. In such cases, consider blocking access from that IP address using a firewall or similar security measure.