F5 Load Balanced Workflow Server returns error "Padding is invalid and cannot be removed" when running reports

Products

• IT Management Suite

Issue/Introduction

Two Workflow servers (version 8.6.1040.0) reside behind a F5 load balancer with HTTPS balance. Both servers have the "ProcessManager Front End" role, both use the same domain account for the "Symantec Workflow windows service" and for the "ProcessManagerPool application pool". Only one of the servers is configured as a "Background server" and is the only one allowed to process Timeouts and Escalations. A ConnectionContext was created to connect this background server to an external database for generating reports in ProcessManager.

When reports are run from the Workflow server with the ConnectionContext it works fine and the reports run as needed.  But when the reports are run from the the Workflow server we see the error: "Padding is invalid and cannot be removed"

As a workaround and be able to see the reports correctly, two ConnectionContexts data sources were created and enable in the reports with the option to manually change the "Data Source", so the user could change it depending on which ProcessManager they were logged in on. Both ConnectionContext strings have the same information of the external DB allowing the users to connect to it, the only difference is that one was created in a ProcessManager portal and the other connection was created in the other portal.

The same issue occurs with the AD Connection. If we search for a AD user from the ProcessManager (server) where the ADConnection was configured, we are able to find users, but, if we try the search from the other ProcessManager, we get the same error mentioned before. 

 

Environment

ITMS 8.x

Workflow 8.x

Cause

The problem is related to the fact that sometimes each Workflow server uses its OWN encryption key for encryption of passwords/etc (ConnectionContext table holds encrypted ConnectionString).

Due to the lack of KMS exchange between the servers, server B cannot decrypt data encrypted by server A.

Resolution

Resolved in WF 8.6.2032

• Previously created connection strings will not work (expected & OK)
• New created connection string works fine and users can successfully use connection string from 1st WF on 2nd WF and vice versa to create their own custom report using for example: SELECT * FROM AuditHistory query.