Error: Unknown CPI error 'Unknown' with message 'execution expired' in 'create_vm' CPI method

Products

• Operations Manager

Issue/Introduction

While you do any of the following:

• Create PKS cluster
• Upgrade stemcell
• BOSH create/recreate VMs
• Upgrade PAS or PKS
• Any similar activity which requires bosh to create new instance and update it

You may observe failure reported with error message: 

Error: Unknown CPI error 'Unknown' with message 'execution expired' in 'create_vm' CPI method

This is due to Bosh director VM not able to talk over port 443 to one or more ESXi hosts. This applies to any ESXi host in the environment that is part of the specified clusters in the Availability Zones or using the Specified Datastore in Bosh director config.

More on the issue in: https://github.com/cloudfoundry/bosh-vsphere-cpi-release/pull/233

Note: This issue can affect PCF 2.4, 2.5, 2.6 & 2.7.

Environment

Product Version: 2.4

Resolution

Workaround

Open port 443 between bosh director and all ESXi hosts in the environment that are part of the specified clusters in the Availability Zones or are using the Specified Datastore in BOSH director config.

Permanent solution

A fix has been released https://github.com/cloudfoundry/bosh-vsphere-cpi-release/releases/tag/v52.1.6

Upgrade Ops manager to one of the following version: 2.5.23 , 2.6.15, or 2.7.4.