Recover from BOSH Instance in "stopped" or "detached" State.
Article ID: 373202
Updated On:
Products
Issue/Introduction
A failure occurs during a bosh upgrade or deployment leaving on or more instances in a "stopped" or "detached" state, preventing further attempts for a successful deployment.
$ bosh is --details
Instance Process State AZ IPs Deployment State VM CID VM Type Disk CIDs Agent ID Index Bootstrap Ignore <instance 1> - az2 172.51.129.4 <deployment 1> stopped <vm 1> medium.disk <disk 1> <agent 1> 1 false false <instance 2> - az1 172.51.129.3 <deployment 1> detached <vm 2> medium.disk <disk 2> <agent 2> 0 true false
Environment
- Tanzu Kubernetes Grid Integrated Edition
- BOSH
- TAS
Resolution
As a workaround, the following steps can be executed. This applies to situations wherein there is 2 or more count of the same instance.
- Ignore the other non-problematic & running instances by running `bosh -d $CF ignore $instance-name`. This is to effectively control which specific instances should not be updated or deleted in the next steps. If an instance is ignored, then it will not be updated or deleted.
- In the Resource Configuration of the TAS tile, decrease the number of instances from 2 to 1.
- Run "Apply Changes". This should delete the problematic instance. Monitor to successful completion.
- Un-ignore the ignored instances by running `bosh -d $CF unignore $instance-name` on each instance that was ignored in step 1.
- In the Resource Configuration of the TAS tile, revert the number of instances to the original setting, from 1 to 2. This should recreate the 2nd instance successfully.
- Run "Apply Changes" again and monitor to completion.
If the above workaround doesn't work, please contact Broadcom support for assistance in resolving this condition.
Additional Information
Feedback
