ESXi host web client shows 404 error due to misconfiguration of endpoint.conf

Products

VMware vSphere ESXi

Issue/Introduction

Symptoms:
Attempting to access the VMware ESXi Host Client (UI) provides a "404 error".

Environment

VMware vSphere ESXi 7.x
VMware vSphere ESXi 6.7.x
VMware vSphere ESXi 8.x

Cause

A common cause of inaccessible host web clients in VMware ESXi environments stems from an incorrect configuration within the /etc/vmware/rhttpproxy/endpoint.conf file. This critical file governs traffic routing within the ESXi environment, dictating how web requests are handled and directed to various internal services. The issue specifically manifests when the redirection port for the user interface (UI) is misconfigured.

The root cause lies in the incorrect specification of the port to which web client requests are redirected. Instead of accurately pointing to port "8309," the default port on which the hostd service listens for web management traffic, the configuration erroneously redirects requests to port "8303." This misconfiguration effectively blocks access to the host web client because requests are being sent to a non-existent endpoint. Consequently, users attempting to access the web UI are unable to connect.

It's worth noting that this problem can sometimes occur when administrators are attempting to resolve a separate issue related to the /cgi-bin redirection port. This is particularly relevant when upgrading from older ESXi versions (5.x or 6.x) to ESXi 7.0.x or later. In such upgrade scenarios, the configuration relating to /cgi-bin may need adjustment, and an error during this process can inadvertently lead to the misconfiguration of the UI redirection port, resulting in the described web client access issues. This particular /cgi-bin issue is highlighted when exporting a support bundle in the Web Client, and the ESXi host is not populated correctly, related KB: ESXi 6.5 not populated when exporting a support bundle in the Web Client

Resolution

A software patch is not available to automatically remediate the specific misconfiguration detailed in the /etc/vmware/rhttpproxy/endpoint.conf file. Standard software patching procedures involve the deployment of updates and fixes by VMware to resolve identified bugs or vulnerabilities within the system. However, the present issue arises from an erroneous modification of the existing configuration file, rather than a systemic software defect.

Workaround Procedure:

To temporarily restore access to the system without the implementation of a software patch, the following manual procedure is to be executed with precision:

Establish a secure connection (SSH or ESXi Shell) to the affected ESXi host using root credentials to proceed with configuration changes.
Configuration File Backup:

Prior to any modification of the /etc/vmware/rhttpproxy/endpoint.conf file, a backup must be created to safeguard against potential data loss or system instability. Employ the following command to create a backup copy:
```
cp /etc/vmware/rhttpproxy/endpoint.conf /etc/vmware/rhttpproxy/endpoint.conf.bak
```
This command duplicates the original configuration file, ensuring a restoration point in case of unintended consequences.
Configuration File Modification:

Access the /etc/vmware/rhttpproxy/endpoint.conf file using a text editor, such as vi. Identify the line within the file that redirects the root path ("/"). This line likely contains an incorrect port assignment, directing traffic to an invalid internal port.

Modify the port number from "8303" to "8309" to accurately reflect the correct listening port for the web client. The line, after modification, should appear as follows:
```
/ local 8309 redirect allow
```
This modification ensures that all web client requests directed to the root path are routed to the appropriate internal port.
rhttpproxy Service Restart:

After making the necessary changes to the configuration file, the rhttpproxy service must be restarted to apply the modifications. Execute the following command to restart the service:
```
/etc/init.d/rhttpproxy restart
```
This command terminates the existing rhttpproxy service instance and initiates a new instance, incorporating the updated configuration.

This manual procedure temporarily rectifies the misconfiguration by ensuring that web client requests are directed to the correct internal port, thereby restoring access to the host web client. It is imperative to adhere to these instructions meticulously. Subsequent to the execution of this procedure, vigilant monitoring of the system is required to identify and address any emergent issues. It is also recommended to document this manual intervention for future reference and auditing purposes.

Additional Information

Impact/Risks:

The recommended workaround requires manual modification of the /etc/vmware/rhttpproxy/endpoint.conf file. It is crucial to exercise extreme caution during this process, as errors introduced during the manual editing can exacerbate the existing issue or potentially lead to further complications. In the event of any unintended consequences or erroneous modifications, immediately restore the configuration from the backup file previously created. After restoration, carefully review the steps and begin the modification process anew. Thorough understanding and meticulous execution are paramount to a successful resolution.

For assistance please reach out to Broadcom Support