vpxd Crashes with a "Panic: Memory exceeds hard limit."
search cancel

vpxd Crashes with a "Panic: Memory exceeds hard limit."

book

Article ID: 379179

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

 

  • One or more vRealize Operations Manager (vROps) instances are accessing the vCenter Server to collect data.

  • One or more of these vROps instances have the NSX-vSphere Adapter installed and running.

  • The vpxd process crashes intermittently or frequently.

  • vCenter UI becomes unusable with code 503 Service unavailable or "no healthy upstream" until the service is restarted.

  • In the vpxd logs (/var/log/vmware/vpxd/vpxd-<number>.log), you observe a disproportionately high number of vim.view.ViewManager.createContainerView tasks compared to vim.view.View.destroy tasks.

  • The logs show memory exhaustion errors, with messages similar to:

     YYYY-MM-DDThh:mm:ss panic vpxd[08492] [Originator@6876 sub=Default]
         -->
         --> Panic: Memory exceeds hard limit. Panic
         --> Backtrace:
         --> [backtrace begin] product: VMware VirtualCenter, version: 7.0.3, build: build-24026615, tag: vpxd, cpu: x86_64, os: linux, buildType: release
         --> backtrace[00] libvmacore.so[0x0037DB8B]
         --> backtrace[01] libvmacore.so[0x002C79C5]: Vmacore::System::Stacktrace::CaptureFullWork(unsigned int)
         --> backtrace[02] libvmacore.so[0x002D6C5B]: Vmacore::System::SystemFactory::CreateBacktrace(Vmacore::Ref<Vmacore::System::Backtrace>&)
         --> backtrace[03] libvmacore.so[0x00370CD7]
         --> backtrace[04] libvmacore.so[0x00370DF3]: Vmacore::PanicExit(char const*)
         --> backtrace[05] libvmacore.so[0x002C7827]: Vmacore::System::ResourceChecker::DoCheck()
         --> backtrace[06] libvmacore.so[0x0023B390]
         --> backtrace[07] libvmacore.so[0x002349E7]
         --> backtrace[08] libvmacore.so[0x00239F75]
         --> backtrace[09] libvmacore.so[0x003765C0]
         --> backtrace[10] libpthread.so.0[0x00007F87]
         --> backtrace[11] libc.so.6[0x000F36BF]
         --> backtrace[12] (no module)
         --> [backtrace end]

 

Environment

vCenter Server 7.0.x

Cause

  • The adapter continuously creates a high volume of ContainerView-based filters in the vpxd process over a short period of time but fails to destroy them once they are no longer needed.

  • Over time, this leads to a buildup of memory usage within the vpxd service. As memory is exhausted, vpxd eventually crashes with a panic due to exceeding its memory limits.

Resolution

There is no permanent resolution for this issue due to the underlying design of the NSX-vSphere Adapter.

Workaround:- 

To mitigate this issue and prevent further crashes of the vpxd service:

  • Stop or remove the NSX-vSphere Adapter from all vROps instances connected to the affected vCenter Server.

Note: The NSX-vSphere Adapter was designed for integration with NSX-V, which is no longer officially supported. For details, refer to Broadcom’s official notice:

Broadcom Knowledge Base Article 314283

  • For environments running NSX-T, use the NSX-T specific adapter provided for vROps, as the NSX-vSphere Adapter is not compatible with NSX-T.

 

 


 

Additional Information

If NSX-V is no longer in use, ensure all associated adapters and integrations are fully removed to prevent service impact.

 

Powered by Wolken Software