Proxy crashed due to JRE out-of-memory. Proxy version not compatible with existing metric filter rules.
The proxy is one of the key mechanisms to ingest data into Wavefront & DX OpenExplore. When upgrading your proxy, it is best to have a rollback plan to deal with unexpected issues after upgrade.
While we advised to reach out to support for assistance proxy issues, downgrading the Proxy to the previous known good version, is many times the fastest way to resume ingestion while a cause and solution can be identified and released.
Command used to downgrade depends on the proxy version installed. Here are three common OS command examples. Reach out to you OS support for additional assistance.
In this example we are going to downgrade using Debian based distro commands. Quick install instructions for Debian or RPM can be found here, Wavefront proxy packages.
sudo apt-get install wavefront-proxy=<desiredVersion>
service wavefront-proxy restart
service wavefront-proxy status
wavefront-proxy.service - LSB: Wavefront Proxy
Loaded: loaded (/etc/init.d/wavefront-proxy; generated)
Active: active (running) since <Date/Time> UTC; 2h 14min ago
Docs: man:systemd-sysv-generator(8)
Process: 10747 ExecStart=/etc/init.d/wavefront-proxy start (code=exited, status=0/SUCCESS)
observability-for-kubernetes utilizes an Operator deployment type for its proxies. Each version of the Operator deploys only a specific version of the Proxy.
Please work with your Kubernetes Admin regarding the underlying Kubernetes environment. Questions about your underlying Kubernetes environment are out of scope of this document.
kubectl apply -f https://github.com/wavefrontHQ/observability-for-kubernetes/releases/download/PREVIOUS_VERSION/wavefront-operator.yaml
kubectl get pods -A
Proxy Troubleshooting
Wavefront Proxy Packages - Quick install instructions for:
Observability for Kubernetes Operator - Downgrade
wavefrontHQ / observability-for-kubernetes, Release Versions