Error "[E_INTERNAL_ERROR] Error registering a VIM account : Cannot create tenant" is received while adding virtual infrastructure (VI) vCloud director endpoint on Telco cloud automation

Products

• VMware Telco Cloud Automation
• VMware Telco Cloud Platform

Issue/Introduction

• Unable to add a virtual infrastructure (VI) vCloud director endpoint on Telco cloud automation, it fails with the error message
  " [E_INTERNAL_ERROR] Error registering a VIM account: Cannot create tenant"
  
  
• Manually registering a new vCloud Org and deleting the newly created vCloud org works fine

• Identify the endpointId from TCA log
  /logs/pods/tca-mgr_tca-api-xxxx/tca/1.log
  
  [{"username":"[email protected]","tenantId":"default","enterprise":"DEFAULT","organization":"DEFAULT","userRoles":["8bb14097-f062-40ab-a60b-aed654c1c05c"],"endpointId":"20241218055144355-6a8565f7-88ed-413a-a14f-3e785ccde8d4"}],"tenantIds":["default"],"severity":"INFO","userIdentity":{"username":"[email protected] "},"eventId":"d49e3f59-f39c-4d27-817e-559004702037 ","eventTime":1743755269594,"message":"Authentication success","restEndpoint":{"uri":"\/tca\/vim\/api\/v1\/orgs\/default\/vims","method":"POST","sourceIPAddress":"10.191.xx.xx"},"requestParameters":{"query":[],"sessionId":"964cb1ee2e1158c3119fe052c0d6052f1ae400e17c9686102928eae36deac2c1"}}

Environment

TCA: 3.2.x

TCP: 5.0.1

Cause

This issue occurs when there are stale entries from the past unsuccessful or incomplete registrations present in the TCA database. 

 

Resolution

Remove the remnant entry from an earlier unsuccessful/incomplete registration job from the TCA database. Once removed, proceed and continue with the VIM registration flow.

1. Please ensure to back up the nodes before making the changes below to the database. Backing Up and Restoring TCA Manager & Control Plane.  Take a snapshot of the TCA Manager and Control plane VMs.
   
   admin@tca-cp [ ~ ]$ debug-tca-cluster
   
   2025-04-08 06:44:04,101 - INFO - Found no running debug pod
   2025-04-08 06:44:12,841 - INFO - Creating new debug pod...
   2025-04-08 06:44:58,530 - INFO - Attaching to debug pod tca-debug-ayvobo-5t7k2
   
   debug-pod [ ~ ]$
   
   debug-pod [ ~ ]$ debug-tca


2. Main Menu: What do you want to do? (Use shortcuts or arrow keys)
      1) Connect to Postgres
      2) Get logs of a service
      3) Debug service
      4) Exit
   
   Answer: 1) Connect to Postgres
   
   
3. An alternate way of getting into PostgreSQL is by executing the following command directly on the TCA-M SSH session:
   kubectl exec -it postgres-0 -n tca-mgr -- psql -d tca -U tca_admin -h localhost
   
   
   
4. Using the UUID from the error message:
   select * from "endpointLink" where "sourceUuid" = '#################-########-####-####-####-############';
   
   

   id  |
                                                         val
                                                                                                             |        creationDate        |        lastUpdated         |     type     | rowType | uuid |           dest
   inationUuid            |                       sourceUuid                       | destinationType | sourceType | destinationUrl | sourceUrl | manageBy
   ----+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
   --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
   --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
   -------------------------------------------------------------------------------------------------------------------------------------
    216 | {"type": "REALM_PAIRED", "source":
   
   {"uuid": "20241218055144355-6a8565f7-88ed-413a-a14f-3e785ccde8d4"}
   , "status": "ok", "rowType": "link", "isDeleted": false, "destination":
   
   {"uuid": "ad607805-20d9-4726-96f 8-056e1bbd5b30"}
   , "creationUser": "HybridityAdmin", "linkVerified": 1743555011391, "lastUpdateUser": "HybridityAdmin", "creationTenantId": "default", "creationEnterprise": "HybridityAdmin", "creationOrganization"
   : "HybridityAdmin", "lastUpdateEnterprise": "HybridityAdmin", "lastUpdateOrganization": "HybridityAdmin"} | 2025-04-02 00:50:11.392364 | 2025-04-02 00:50:11.392364 | REALM_PAIRED | link    |      | ad607805-20d9-
   4726-96f8-056e1bbd5b30 | 20241218055144355-6a8565f7-88ed-413a-a14f-3e785ccde8d4 |                 |            |                |           |
   (1 row)

   
   
   
5. Delete the endpoint link based on the ID
   delete from "endpointLink" where id = 216;