Recover ALB controllers and service engine VMs from Read-Only partitions
Article ID: 317234
Updated On:
Products
VMware
Issue/Introduction
Symptoms:
- Nodes with read-only filesystem will go into an inactive state.
- Cluster will not be converged/stable.
- Multiple processes on the controller will fail to come up.
- SEs in this state will have degraded traffic processing since some of the processes will be in a bad state.
- Nodes with read-only filesystem will go into an inactive state.
- Cluster will not be converged/stable.
- Multiple processes on the controller will fail to come up.
- SEs in this state will have degraded traffic processing since some of the processes will be in a bad state.
Environment
VMware NSX Advanced Load Balancer 20.1.x
Resolution
- Launch the console of the VM that is affected.
- If the VM has failed to boot up, you should see an error like "The root filesystem on /dev/sda3 requires a manual fsck"
- Please note that the root filesystem could be mounted on a different path than "/dev/sda3". Make a note of the error and it will highlight the path that needs to be repaired.
- Once you have access to the console, run the following command:
- After the command has been run, you should see that the file system has been modified:
- After this has been run, reboot the VM and this time it should boot up successfully.
- If the issue is not resolved after these steps, please open a case with technical support.
- If the VM has failed to boot up, you should see an error like "The root filesystem on /dev/sda3 requires a manual fsck"
- Please note that the root filesystem could be mounted on a different path than "/dev/sda3". Make a note of the error and it will highlight the path that needs to be repaired.
- Once you have access to the console, run the following command:
fsck.ext4 -f /dev/sda3- The path after -f will be the path mentioned in the error.
- After the command has been run, you should see that the file system has been modified:
- After this has been run, reboot the VM and this time it should boot up successfully.
- If the issue is not resolved after these steps, please open a case with technical support.
Additional Information
Impact/Risks:
- When the disk goes into this state, the VMs are not properly booted up leading to multiple issues.
- This can affect the user's ability to access GUI and make changes and it can also affect the SE's ability to process traffic correctly.
- When the disk goes into this state, the VMs are not properly booted up leading to multiple issues.
- This can affect the user's ability to access GUI and make changes and it can also affect the SE's ability to process traffic correctly.
Feedback
Yes
No
Powered by
