Unknown CPI error 'Unknown' with message 'Unable to communicate with the remote host, since it is disconnected.'

Products

• VMware vCenter Operations Manager
• VMware Tanzu Application Service for VMs
• VMware vCenter Server
• VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

Unknown CPI error 'Unknown' with message 'Unable to communicate with the remote host, since it is disconnected.' in '<create|delete|...>_vm' CPI method

Environment

• VMware Tanzu Kubernetes Grid Integrated Edition
• VMware vCenter Operations Manager
• VMware Tanzu Application Service for VMs
• VMware vCenter Server 

Cause

• The message "Unable to communicate with the remote host" does not appear anywhere in the bosh-vsphere-cpi codebase.
• The CPI tried to tell vCenter to create, delete, etc, the VM and vCenter returned the error. 
• The ESXi host that the VM is on or being placed on is in a bad state. 

Resolution

• Check vCenter for more details.
• Once vSphere is working correctly the CPI operation should succeed.