BOSH ssh fails on a particular VM

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

BOSH ssh fails with error as in the screenshot.



BOSH ssh works on other VMs in the same deployment. The error is only from one VM.

Environment

Product Version: 2.10

Resolution

Behind "bosh ssh", bosh and the sshd service on the target VM would negotiate a temporary user for the SSH session. The above error may indicates problem with the temporary user or with the sshd service itself. To resolve the issue, use "bosh recreate" to recreate this VM. If you want to avoid recreating the VM and if vcap user exists for the VM, there is an alternative resolution.

Option 1

Recreate the VM with "bosh recreate".
 

Option 2 (if vcap user exists)

Take diego_cell as an example.

1. Identify the IP of the problematic Diego cell:

• bosh -d cf-GUID vms | grep cell_GUID

2. Retrieve SSH password for user "vcap":

• Access OpsManager UI -> TAS tile -> Credentials tab -> Navigate to the section of Diego Cell - VM Credentials, click the link -> search for "diego_cell" to locate the password. See screenshot as below.
• 
• 

3. SSH to the Diego cell with user vcap and password. Note that all Diego cells share the same vcap password.

• ssh vcap@<DIEGO_CELL_IP>

4. Restart sshd service. Make sure it is running after restart.

• sudo su
• systemctl restart sshd
• systemctl status sshd

5. Logout from the Diego cell and try "bosh ssh" with it again.

Note: For VMs in bosh deployments "service-instance-GUID", there is no predefined vcap user. So Option-2 does not work and you have to recreate the VM.