Aria Orchestrator workflow to rescan HBAs and VMFS fails

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

While trying to rescan HBAs/VMFS for ESXi hosts using an orchestrator workflow, it fails with below error -

Error when rescanning VMFS. Reason: InternalError: https://esxi:443/sdk invocation failed with 'java.net.SocketTimeoutException: Read timed out'.

Environment

VMware Aria Automation 8.x

Cause

This issue happens when the socket read timeout reaches early within Orchestrator and the connection closes before the task completion on vSphere. This is usually seen when the rescan tasks take longer time to complete on vSphere level.

Resolution

1. Take snapshots(without memory and with quiesce option enabled) of all three Aria Automation nodes from vCenter server [Snapshots of the nodes should not have a difference of more than 40 seconds]
2. Edit the /data/vco/usr/lib/vco/app-server/conf/vmo.properties file and add below line to it (Timeout value needs to be set as per the task completion time taken on vSphere)-
   

   com.vmware.vmo.plugin.vi4.socketReadTimeout = 360000

3. Restart the Orchestrator pods (This will take the Orchestrator UI down for a few minutes till the pods redeploy)
   

   kubectl rollout restart deployment vco-app  -n prelude

4. Re-run the workflow from Orchestrator