CDC/RDC Deployment Failure in Telco Cloud Automation with vRO 8.3

Products

• VMware
• VMware Telco Cloud Automation

Issue/Introduction

Symptoms:
In Infrastructure Automation, during deployment of Central / Regional Site, at the step "Deploy appliances for domain <domain-name>", there is a failure at the sub-step "Configure VRO for domain <domain-name>" with error "Location details not present in authResponse.headers" and domain provisioning fails. Even on re-sync, there is a failure at this sub-step and the user is not able to proceed past this.

 

Environment

VMware Telco Cloud Automation 1.9.5
VMware Telco Cloud Automation 1.x
VMware Telco Cloud Automation 1.8
VMware Telco Cloud Automation 2.0
VMware Telco Cloud Automation 1.9

Cause

This occurs due to the appliance password for vRO provided by the user in the input specification is not able to be set during deployment of the vRO appliance. After deployment of the appliance, during configuration, when the provisioning workflow step tries to login to the deployed vRO, login fails, causing a failure in configuring the appliance, and failing the domain provisioning as a result.

Resolution

This is a known issue in vRO 8.3, which is addressed in 8.4.2 and above.

Workaround:

Approach 1

Use vRO 8.4.2 for new CDC or RDC domain deployment. If you have run into CDC/RDC domain deployment failure with vRO 8.3 -

• Login to the corresponding domain vCenter
• Identify the vRO appliance, power it OFF and delete it
• Login to TCA manager and replace vRO 8.3 with vRO 8.4.2 in the images section under Infrastructure automation → Configuration
• Trigger a resync of the failed domain

Follow these steps:

Step 1

Ensure that the domain in question is in a FAILED state and no active tasks are running on it.

Step 2

Login to the vCenter UI for your domain in a web-browser of your choice. vCenter appliance will have been deployed as part of the domain provisioning. Its URL will be present in the appliances list for the domain in question.

Follow the path TCA UI > Infrastructure Automation > Domains > |Domain Type (Central / Regional)| > |Domain Name| > Appliances. The vCenter appliance URL will be available here.

Step 3

In vCenter, in the Hosts and Clusters view, you'll be able to find the vro VM at the path: |VC Name| > |Domain Name|-dc01 > |Domain Name|-cl01 > rp-management > vro, then Power-off the vro VM.

Step 4

Once the vro VM is powered-off, you may choose to delete it or rename it to a different name. This is required because when we re-sync the domain from Infrastructure UI at the end of this process with a new vRO version, it will create a new vRO VM with the same name (vro) and presence of another VM with the same name will cause a conflict and cause a failure again during domain provisioning.

Step 5

Update the image URL for vRO appliance in TCA UI > Infrastructure Automation > Configuration > Images. Click on Edit, update the URL for VMware vRealize Orchestrator and save the changes.

Step 6

Re-sync the domain in question. Follow the path TCA UI > Infrastructure Automation > Domains > |Domain Type (Central / Regional)|. Select the required domain, click on RESYNC and confirm RESYNC in the pop-up that opens.

Step 7

Wait for the domain provisioning to finish. It should be successful now.
 

Approach 2

CDC/RDC domain deployment failure with vRO 8.3 -

• Login to vRO appliance via SSH (or from the console) as root using the default password "vmware"
• The login will indicate the expiration of root password and will prompt to set a new password
• Use the same password as provided in input specification for vRO appliance during domain provisioning.
• Update the secret (follow step #3 in https://kb.vmware.com/s/article/78183)
• Login to TCA manager and trigger a resync of the failed domain

Follow these steps :

Step 1

Ensure that the domain in question is in a FAILED state and no active tasks are running on it.

Step 2

Make a note of the IP assigned to the vRO appliance VM.

Follow the path TCA UI > Infrastructure Automation > Domains > |Domain Type (Central / Regional)| > |Domain Name| > Appliances. The vRO appliance IP & FQDN will be available here.

Step 3

SSH into the vro VM, using the root account with "ssh root@<vRO-IP>". For password, enter "vmware" which is the default password. It will display a message that the current password is expired and ask to update the password. Use the same password as provided in the input specification for vRO appliance during domain provisioning. The password will now have been set and we would have logged in as the root user.

Step 4

Now, we need to reflect this SSH password update to control center UI of the vRO appliance as well. Follow the process from step 3 - Update the secret in the solution mentioned in this KB article: https://kb.vmware.com/s/article/78183.

Step 5

After the pods are re-deployed and are in running state, you can try to verify that the password has been updated. Try logging in with user "root" and the new password set in the above step to the control-center UI of the vRO appliance.

Open the vRO appliance via the vRO IP/FQDN from Step 2 above and open it in a web-browser. Click on "Start the Control Center" and login with the credentials. You should be able to login successfully. Now logout of the control center as nothing is to be changed here.

Note: In the FQDN, if the port says 8281, please use 443 instead. Port 443 is the correct one for vRO 8.x and above.

Step 6

Re-sync the domain in question. Follow the path TCA UI > Infrastructure Automation > Domains > |Domain Type (Central / Regional)|. Select the required domain, click on RESYNC and confirm RESYNC in the pop-up that opens.

Step 7

Wait for the domain provisioning to finish. It should be successful now.