vCenter Server Inaccessible and Unresponsive Due to Operating System crash or Filesystem Corruption

Products

• VMware vCenter Server

Issue/Introduction

• vCenter Server becomes inaccessible through the network and fails to respond to pings
• Users experience issues such as the inability to manage vSphere environment due loss of access to the vCenter Server user interface, and disruption of critical services running on the vCenter Server.

Environment

VMware vCenter Server

Cause

Due to the power failure or network outage, the vCenter server gets into the state of inaccessibility and unresponsiveness because of the corruption in the vCenter Server file system or Operation system.

Resolution

Follow the steps below to address the vCenter inaccessibility issue.

1. Check vCenter Server disk space:
   - Ensure that the virtual disks assigned to the vCenter Server VM have sufficient free space. A lack of disk space can cause performance issues and lead to corruption.
   - If disk space is low, consider expanding the virtual disks or deleting unnecessary files and logs. Follow the KB here to know more details vCenter Server Appliance disk space is full

2. Verify network connectivity:
   - Check if the vCenter Server VM's network adapter is connected and properly configured.
   - Ensure that the VM is assigned to the correct port group or VLAN.
   - Verify that the network switches and routers are functioning correctly and that there are no firewall rules blocking communication to and from the vCenter Server. 
   - Follow the KB here to more details Using Curl to test port connectivity in VMware vCenter Server Appliance

3. Check vCenter Server services:
   - Connect to the vCenter Server VM using SSH or console access.
   - Verify that all critical vCenter Server services are running using the command `service-control --status --all`.
   - If any services are not running, attempt to start them using the command `service-control --start <service_name>`.
   - Refer to the KB for more details Check vCenter Server essential service status and dependencies

4. Review vCenter Server logs:
   - Access the vCenter Server logs and check for any error messages or indications of corruption, such as file system errors or database inconsistencies.
   - Refer to the KB to know more about log file location of vCenter server Location and Contents of vCenter Server log files

5. Run file system check to fix the file system corruption.
   - If there are file system errors found, run the file system check to fix the corruption.
   - Refer to the following KB for more details Manually running a filesystem check

6. Restart the vCenter Server:
   - If the previous steps do not resolve the issue, try restarting the vCenter Server VM.
   - After the restart, check if the vCenter Server is accessible and functioning correctly.

For further help, open a Broadcom support case. Refer to the KB Creating and managing Broadcom cases

Additional Information

Best Practices to follow for vCenter.

• Regularly create and test backups of vCenter Server to minimize data loss and downtime in case of issues. File-Based Backup and Restore of vCenter Server
• Ensure vCenter Server and ESXi hosts are updated with the latest version. 
• If using virtual distributed switches (DVS), make sure to implement ephemeral port groups so that vCenter Server, storage, and other management resources can still be reconnected to the DVS even in the event vCenter Server becomes disconnected from the DVS. Refer to the KBs, Moving vCenter Server to an ephemeral port group on a vSphere Distributed Switch  and Static (non-ephemeral) or ephemeral port binding on a vSphere Distributed Switch for more details.