Migration from vRealize Automation 6.2.x to 7.2 - 7.3 leaves a large number of mismatched and empty resources in Deployments or fails with "504 Gateway time-out" or "413 Request Entity Too Large" errors

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• Migration from vRA 6.2.x to vRA 7.2 or 7.3 fails. You see any of these errors:
  • 504 Gateway Time-out.
  • 413 Request Entity Too Large.
  • Cannot access a disposed object.
    Object name: 'System.Net.Http.HttpClient'.
• If the migration from 6.2.x to 7.3 is successful, you see any of these results:
  • Empty deployments.
  • VMs belonging to more than 1 deployment.
  • VMs not associated with blueprints.

Environment

VMware vRealize Automation 7.3.x
VMware vRealize Automation 6.2.x
VMware vRealize Automation 7.2.x

Cause

This issue occurs because, migration of environments that contain a large number of machines or blueprints can result in a timeout between vRA services resulting in incomplete migration of machines and blueprints.

Resolution

This is a known issue affecting VMware vRealize Automation 7.2 and 7.3.0.

This issue is resolved in VMware vRealize Automation 7.3.1 and VMware vRealize Automation 7.4, available at VMware Downloads. 

Workaround:
To work around this issue, before migration deploy the attached version specific hotfix on the target vRA 7.2 or 7.3 environment.

Note: If you have already run migration, you will need to revert the target vRA 7.2 / vRA 7.3 environment, deploy the fix and re-run migration.

Procedure:

1. Take a backup of these existing files:
   1. Connect to the IaaS machine of the target 7.2/7.3 environment that hosts the Model Manager Data component.
   2. Navigate to <vRA Install Folder>\Server\Model Manager Data\Cafe and rename these files:
      1. VMware.Cafe.dll to VMware.Cafe.dll.bak
      2. VMware.Cafe.Client.dll to VMware.Cafe.Client.dll.bak
      3. Vcac-config.exe to Vcac-config.exe.bak
2. Download and unzip the attached version specific zip file ( 000051096_vRA 7.3 for vRA 7.3 and 000051096_vRA 7.2 for vRA 7.2)
3. Copy the extracted VMware.Cafe.dll, VMware.Cafe.Client.dll and Vcac-config.exe files to this <vRA Install Folder>\Server\Model Manager Data\Cafe folder.
4. To resolve the 504 Gateway Time-out error, increase the client and server timeouts of the HAProxy service:
   1. SSH into the vRA virtual appliance.
   2. Open /etc/haproxy/haproxy.cfg file.
   3. Edit and change the following settings:
      1. timeout client 500s to timeout client 3600s
      2. timeout server 500s to timeout server 3600s
   4. Save the file.
   5. Restart the HAProxy service by running the command:
      service haproxy restart
   6. Repeat steps a to e for each virtual appliance in the system.
5. Open the <vRA Install Folder>\vCAC\Server\Model Manager Data\Cafe\Vcac-Config.exe.config file, and add the CafeClientCacheDuration appSetting and set it to 5 hours:
    

<appSettings>
<add key="EventLogSource" value="vCAC" />
<add key="repositoryAddress" value="" />
<add key="repositoryTimeout" value= "200" />
<add key="repositoryIgnoreServerCertificateWarnings" value="true"/>
<add key="CafeClientCacheDuration" value="05:00:00"/>
</appSettings>
 

6. Run migration.
7. When migration is successfully completed:
   1. Repeat Step 4 to reset the client and server timeout values of the HAProxy service back to 500s. Ensure to restart the HAProxy service when the settings are changed.
   2. Remove the CafeClientCacheDuration setting that was added in Step 5, or set it to 1 hour:
      
      <add key="CafeClientCacheDuration" value="01:00:00"/>

Note: This affects both vRA 7.2 and 7.3.