Threat defense failing with Error: "casoop: curl_easy_perform() failed: SSL peer certificate or SSH remote key was not OK"

Products

• Messaging Gateway

Issue/Introduction

Following the update to Messaging Gateway (SMG) 10.9.1, the Threat Defense scanning begins failing with the following errors:

2024-08-12T22:49:31+05:30 (ERROR:257817.266323520): [85641] casoop: curl_easy_perform() failed: SSL peer certificate or SSH remote key was not OK.
2024-08-12T22:49:31+05:30 (ERROR:257817.266323520): [85643] casoop: HTTP POST to CAS server "CAS IP" received response error for sha256 <KEY>: SSL peer certificate or SSH remote key was not OK.

Additionally, the messages sent to the CAS server will display the verdict as "unscannable".

Cause

Messaging Gateway 10.9.1 uses stricter certificate validation than previous releases, which is causing validation of the TLS certificate presented by the CAS server to fail.

This is likely due to a mismatch between the IP address of the CAS server and the hostname or IP address configured in the certificate presented by the CAS server. This mismatch between the hostname or IP that SMG is configured to connect to and the hostname or IP in the certificate's Subject Alternative Name (SAN) list is causing validation of the CAS server TLS certificate to fail.

Resolution

This issue may be addressed through several means:

• Ensure that the SMG CAS server configuration in Threat Defense > CAS Connect is using an IP address or hostname present in the CAS server TLS certificate Subject Alternative Name list
• Generate and install a TLS certificate on the CAS server with a SAN which matches the hostname or IP address of the CAS server

Note: This is not a software defect, and the improvement to SMG TLS certificate validation security will not be changed in a later release.