ALERT: Some images may not load properly within the Knowledge Base Article. If you see a broken image, please right-click and select 'Open image in a new tab'. We apologize for this inconvenience.
SSL sessions rejected with error "OpenSSL RSA operation failure" when decrypting on SSL Visibility
Article ID: 169325
Updated On:
Products
SSL Visibility Appliance Software
Issue/Introduction
When using the SSL Visibility Appliance product you may notice that SSL sessions to various Google domains (and possibly others) are being rejected with the session log indicating that there is an OpenSSL RSA operation failure. This has been observed when using Chrome (56.0.2924.76) and Firefox (51.0.1), and only if the policy action is to inspect the flow.
The error is caused by a signature algorithm being used that was pulled from the TLS1.3 spec called RSA-PSS.
The error is caused by a signature algorithm being used that was pulled from the TLS1.3 spec called RSA-PSS.
Resolution
Release 3.11.2.1 addresses the issue and is available for download on BTO as of January 25th, 2017.
There is a mention of the new signature support in the release notes as follows: SSL Visibility 3.11.2.1 allows clients to authenticate servers using the RSA-PSS signature scheme.
The most efficient way to do this is by creating a custom Subject/Domain Name list for Google sites and applying it to a cut-through rule within the ruleset. The Subject/Domain Name List should contain, at the very least, the following entries:
CN=*google.com
CN=*googlevideo.com
CN=*googleapis.com
More entries can be added to the custom list as needed.
Instructions on how to create a Subject/Domain Names List can be found in the SSL Visibility Appliance Administration & Deployment Guide.
There is a mention of the new signature support in the release notes as follows: SSL Visibility 3.11.2.1 allows clients to authenticate servers using the RSA-PSS signature scheme.
Workaround
If an upgrade is not feasible at this time, a workaround for the issue is to cut-through the domains being rejected.The most efficient way to do this is by creating a custom Subject/Domain Name list for Google sites and applying it to a cut-through rule within the ruleset. The Subject/Domain Name List should contain, at the very least, the following entries:
CN=*google.com
CN=*googlevideo.com
CN=*googleapis.com
More entries can be added to the custom list as needed.
Instructions on how to create a Subject/Domain Names List can be found in the SSL Visibility Appliance Administration & Deployment Guide.
Feedback
Yes
No
Powered by
