RHTTPPROXY Services Fail to Start on vCenter
search cancel

RHTTPPROXY Services Fail to Start on vCenter

book

Article ID: 375424

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

vmware-rhttpproxy services fail to start on vCenter server due to config.xml file corruption and/or incorrect permissions assigned. 

  • File-based backups can fail 
  • Update/patch of VCSA also fails when services try to start up.
  • If you reboot/restart services on the VC, these will fail to start also.

 

In the VC logs can see below errors: 

 

 /var/log/vmware/rhttpproxy/rproxy_init.log.stdout: 

 

2024-06-19T12:23:47.815+02:00 error -[1786012] [Originator@6876 sub=RhttpProxy] Unable to reload SSL credentials; error: N7Vmacore24InvalidArgumentExceptionE(In-memory creation of SSLcontext via vecs cert data doesn't work yet.Specify ssl certificateFile andprivateKeyFile. Stopping rhttpproxy.)
--> [context]zKq7AVECAQAAADAhbQENLQAAydlacmh0dHBwcm94eQAAUCzSACD3zQBzitkAkhZWAOYYVgDwqF8AnC5XAGDAWgD0y1UApt1VAQp+AmxpYmMuc28uNgAAKZBV[/context]
2024-06-19T12:23:47.816+02:00 error -[1786012] [Originator@6876 sub=Default] Application error: In-memory creation of SSLcontext via vecs cert data doesn't work yet.Specify ssl certificateFile andprivateKeyFile. Stopping rhttpproxy.
2024-06-19T12:23:47.816+02:00 error -[1786012] [Originator@6876 sub=Default] Backtrace:
--> [backtrace begin] product: rhttpproxy, version: 8.0.2, build: build-23929136, tag: -, cpu: x86_64, os: linux, buildType: release
--> backtrace[00] rhttpproxy[0x005AD9C9]
--> backtrace[01] rhttpproxy[0x00D22C50]
--> backtrace[02] rhttpproxy[0x00CDF720]
--> backtrace[03] rhttpproxy[0x00D98A73]
--> backtrace[04] rhttpproxy[0x00561692]
--> backtrace[05] rhttpproxy[0x005618E6]
--> backtrace[06] rhttpproxy[0x005FA8F0]
--> backtrace[07] rhttpproxy[0x00572E9C]
--> backtrace[08] rhttpproxy[0x005AC060]
--> backtrace[09] rhttpproxy[0x0055CBF4]
--> backtrace[10] rhttpproxy[0x0055DDA6]
--> backtrace[11] libc.so.6[0x00027E0A]
--> backtrace[12] rhttpproxy[0x00559029]
--> [backtrace end]

 

/var/log/vmware/rhttpproxy/rproxy_init.log.stderr

Starting service process with pid: 1786012.
Application error: In-memory creation of SSLcontext via vecs cert data doesn't work yet.Specify ssl certificateFile andprivateKeyFile. Stopping rhttpproxy.
Backtrace:
[backtrace begin] product: rhttpproxy, version: 8.0.2, build: build-23929136, tag: -, cpu: x86_64, os: linux, buildType: release
backtrace[00] rhttpproxy[0x005AD9C9]
backtrace[01] rhttpproxy[0x00D22C50]
backtrace[02] rhttpproxy[0x00CDF720]
backtrace[03] rhttpproxy[0x00D98A73]
backtrace[04] rhttpproxy[0x00561692]
backtrace[05] rhttpproxy[0x005618E6]
backtrace[06] rhttpproxy[0x005FA8F0]
backtrace[07] rhttpproxy[0x00572E9C]
backtrace[08] rhttpproxy[0x005AC060]
backtrace[09] rhttpproxy[0x0055CBF4]
backtrace[10] rhttpproxy[0x0055DDA6]
backtrace[11] libc.so.6[0x00027E0A]
backtrace[12] rhttpproxy[0x00559029]
[backtrace end]
rproxy_init.log.stderr (END)

 

/var/log/vmware/vmon/vmon.log may show similar entries to below when trying to start the service:

<rhttpproxy> Service pre-start command's stderr: /usr/lib/tmpfiles.d/envoy.conf:14: Line references path below legacy directory /var/run/, updating /var/run/envoy รข /run/envoy; please update the tmpfiles.d/ drop-in file accordingly.

 

If patch/update failed, will see similar errors as below in /var/log/vmware/applmgmt/Patchrunner.log: 

 

2024-06-19T10:23:49.131Z ERROR vmware_b2b.patching.phases.patcher Patch hook Patch got unhandled exception.
Traceback (most recent call last):
  File "/storage/seat/software-updateedty8f6i/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 208, in patch
    _patchComponents(ctx, userData, statusAggregator.reportingQueue)
  File "/storage/seat/software-updateedty8f6i/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 89, in _patchComponents
    _startDependentServices(c)
  File "/storage/seat/software-updateedty8f6i/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 56, in _startDependentServices
    serviceManager.start(depService)
  File "/storage/seat/software-updateedty8f6i/stage/scripts/patches/libs/sdk/service_manager.py", line 909, in wrapper
    return getattr(controller, attr)(*args, **kwargs)
  File "/storage/seat/software-updateedty8f6i/stage/scripts/patches/libs/sdk/service_manager.py", line 799, in start
    super(VMwareServiceController, self).start(serviceName)
  File "/storage/seat/software-updateedty8f6i/stage/scripts/patches/libs/sdk/service_manager.py", line 665, in start
    raise IllegalServiceOperation(errorText)
service_manager.IllegalServiceOperation: Service cannot be started. Error: Error executing start on service rhttpproxy. Details {
    "detail": [
        {
            "id": "install.ciscommon.service.failstart",
            "translatable": "An error occurred while starting service '%(0)s'",
            "args": [
                "rhttpproxy"
            ],
            "localized": "An error occurred while starting service 'rhttpproxy'"
        }
    ],
    "componentKey": null,
    "problemId": null,
    "resolution": null

Environment

vCenter Server 8.0

Cause

The /etc/vmware-rhttpproxy/config.xml file is corrupted and incorrect permissions are set to this file

 

  • The config.xml file will not look normal - check contents and compare with a working VC to confirm any corruption here 
  • The config.xml file also doesn't have correct permissions assigned when running "ls -l" in the /etc/vmware-rhttpproxy/ directory, permissions are not set to rhttpproxy:root (see screenshot below on how should look)

Resolution

To resolve the issue follow below steps: 

**Ensure to always have valid Snapshots (offline for Enchanced Linked Mode environments) or file-based backups completed prior to making any changes. 

1. Review /etc/vmware-rhttpproxy/config.xml file to ensure no corruption

2. If the config file is corrupted, copy contents from a VC on similar version in their environment and complete below: 

  • make a copy of the config.xml file

cp config.xml config.xml.bkp 

  • Edit the config.xml and remove the current contents

vi config.xml 

-> to remove you can run something similar to below to remove 100/1000 lines etc. 

1000dd

  • Paste the contents from working VC config.xml file to the affected VCs 

press "i" and right click to paste it in

  • Save the changes

press "ESC" and type ":wq!" and enter

 

3. Check the permissions on the config.xml, can run "ls -l" and permissions should look like below: 

  • If they do not, then we need to update these to match above - to complete this, run below command: 

chown rhttpproxy:root config.xml 

  • Once complete, start the rhttproxy services: 

service-control --start vmware-rhttpproxy

 

Services should now be up and running. (restarting all service may be required): service-control --stop --all && service-control --start --all  

 

Powered by Wolken Software