Workflow hub API sessions fail with "error_code": "WFW-REST403" Forbidden post replacing CA certificates for Telco cloud automation

Products

• VMware Telco Cloud Automation

Issue/Introduction

TCA Web certificate or Appliance Management has been replaced with a CA-signed certificate as per the document or Replace appliance certificates

Post CA replacement, executing the 'Caas Multi Version Upgrade Workflow' via Workflow hub (WFH) fails with 
Error : 

                "workflow_run": "55dd21cc-6cbe-461d-b348-bd3c7b63d064"
            },
            "entity_name": "[\"ci-met-s2r-ccdm\"]",
            "event": "ended",
            "logs": "{\"message\":\"{\\\"error\\\":\\\"Forbidden\\\",\\\"path\\\":\\\"/tca/global/api/v1/sessions\\\",\\\"status\\\":403,\\\"timestamp\\\":\\\"2025-07-16T04:43:53.725+00:00\\\",\\\"x-hm-authorization\\\":\\\"\\\"}\",\"operation\":\"Create TCA session\"}",
            "runId": "dbd382aa-5a46-4153-bfd4-be88b2431842",
            "task": "preprocess",
            "task_status": "Failed"
        },
        "error_code": "WFW-REST403",
        "locusId": null,
        "state": "Preprocess",
        "status": "Ended",
        "timestamp": "2025-07-16 04:43:53.789378",
        "url": null

WFW-REST403    indicates that access to the requested resource is forbidden

/logs/pods/tca-cp-cn_tca-api-xxx/tca/0.log report 
2025-07-16T05:31:53.381618457Z stdout F 2025-07-16 05:31:53.380 UTC [http-nio-8443-exec-7, , , TxId: ] INFO  c.v.vchs.hybridity.audit.AuditTrail- {"internal":{"threadEnterprise":"No Thread Context","threadUser":"No Thread Context","thread":"http-nio-8443-exec-7","lineNumber":165,"classname":"com.vmware.vchs.hybridity.api.LoginUtil","method":"logAuth"},"userIdentities":[{"username":"[email protected]","tenantId":"default","enterprise":"DEFAULT","organization":"DEFAULT","userRoles":[],"endpointId":"20231030152301215-fcdb9209-27e5-4264-8aea-a208014639e2"}],"tenantIds":["default"],"severity":"CRITICAL","userIdentity":{"username":"[email protected] "},"eventId":"0730fa4b-91b0-4c2d-a65a-32ee6cd20774","eventTime":1752643913380,"message":"Access Denied","eventName":"Login Failed","service":{"name":"Login Failed"},"restEndpoint":{"uri":"\/tca\/global\/api\/v1\/sessions","method":"POST","sourceIPAddress":"100.100.X.X"},"requestParameters":{"query":[]},"responseElements":{"isAuthenticated":"false"}}
2025-07-16T05:31:53.383429738Z stdout F 2025-07-16 05:31:53.383 UTC [http-nio-8443-exec-7, , , TxId: ] ERROR c.v.v.h.a.HybridityAccessDeniedHandlerImpl- Sending Response Error 403 for /tca/global/api/v1/sessions
2025-07-16T05:32:00.334333176Z stdout F 2025-07-16 05:32:00.334 UTC [vimstats-single-scheduler1, , , TxId: ] INFO  c.v.hybridity.nfvm.VimStatsCache- CAche update in progress
2025-07-16T05:32:00.420905566Z stdout F 2025-07-16 05:32:00.420 UTC [vimstats-update-pool4, , , TxId: ] WARN  c.v.vca.hybridity.util.NSPRestClient- Login to cloud https://<TCA-CP> failed, with status 401: Unauthorized

 

Environment

Telco cloud automation 3.2

Cause

Post replacing CA certificates for Telco cloud automation, WFH secrets needs to be reconfigured for the integration to work 

Resolution

Execute WFH secret manager shell script 'create-lcm-cluster-esxi-secrets.sh' from TCA control plane (tca-cp) node. Follow the steps from Secret Management

/opt/vmware/scripts ]$ ./create-lcm-cluster-esxi-secrets.sh
Enter the TCA host:

Enter the TCA username:

Enter the TCA password:
Enter the TCA organization: (default)

Proceeding...
Enter the secret namespace: (default)

Enter vCenter details as input:
Enter the vCenter FQDN:

Enter the vCenter username:

Enter the vCenter password:
Do you have an additional vCenter to input? (Y/N)

tee: output.txt: Permission denied
Connecting to the Workflow Hub