How to recreate BOSH Director VM

Products

• Operations Manager

Issue/Introduction

Symptoms:

You see the following issues in Ops Manager if the Director VM is missing:

• The Status page for the Ops Manager Director Tile will not show any data
• A bosh login will fail with an error:

`System call error while talking to director: No route to host - connect(2) for "10.xx.xxx.xx" port 25555 (10.xx.xxx.xx:25555) 

• In the Changelog file, the following error can be seen:

E, [2017-05-26 18:27:59 #6554] [task:5616] ERROR -- DirectorJobRunner: 419: unexpected token at '"consul.a'
Deploying: 
Creating instance 'bosh/0': 
Creating VM: 
Creating vm with stemcell cid 'sc-c174b9f9-fccd-4ee0-ad39-57bac11a0922': 
CPI 'create_vm' method responded with error: CmdError{"type":"Unknown","message":"A general system error occurred: Invalid response code: 503 Service Unavailable","ok_to_retry":false} 
Exit code 1 
===== 2018-05-15 08:55:30 UTC Finished "bosh2 --no-color --non-interactive --tty create-env /var/tempest/workspaces/default/deployments/bosh.yml"; Duration: 79s; Exit Status: 1 
Exited with 1.```

Environment

Cause

Director VM is in a bad state.

Director VM was moved or deleted by an external cause. This should not happen during normal operation. This KB can be used to recover a missing Director VM.

Resolution

Execute the steps:

1. SSH into Ops Manager VM as ubuntu
2. Locate the file at /var/tempest/workspaces/default/deployments/bosh-state.json
3. Edit the file as a sudoer
4. Locate the line with name current_manifest_sha and delete the value of it
5. In Ops Manager GUI page, click “Apply Changes”

Alternatively, you can also trigger recreate via UI by steps:

1.  Increase the persistent disk size of the Director VM under OpsMan / Director / Resources
2.  Run Apply changes

Note if stemcell is deleted from vsphere then you may need to consider steps in this KB article How to recreate BOSH Director VM when the Stemcell is deleted from vSphere