DX AIOps - Unable to open Openshift console

book

Article ID: 198091

calendar_today

Updated On:

Products

DX Operational Intelligence DX Application Performance Management CA App Experience Analytics CA Application Performance Management (APM / Wily / Introscope)

Issue/Introduction

Symptoms:

-"oc get pods" returns many pods in Evicted status
-Unable to access to openshift console, "This page isn't working"

Cause

In this case, the issue was caused by a capacity issue, openshift-web-console pod couldn't be created so access to openshift is not possible.

Here is a summary of the steps taken to identify the root cause:

oc login -u system:admin

oc project openshift-web-console

oc get pods

oc describe pod <webconsole-.. podname>

 

 

Environment

DX OPERATIONAL INTELLIGENCE - 1.3.2, 20.x

DX APPLICATION PERFORMANCE MANAGEMENT 20.x

 

 

 

Resolution

1. Login as admin

oc login -u system:admin

2.Temporally allow pods creation in Master - as scheduling of pods is disabled

oc get nodes

oc adm manage-node <master-name> --schedulable=true

3. Wait for a couple of minutes, then check the pod is in running state

oc project "openshift-web-console"
oc get pods

4. In parallel, switch to "default" project and verify that "router" pod is running:

oc project default
oc get pods

5. Once router and web-console pods are running, disable schedulable

oc adm manage-node <master> --schedulable=false

6. Verify access to openshift console

 

7. Additionally, if for example, you find many OI pods in Evicted status, you can safely remove those to better monitor the health of your system:

DX OI/APM - many pods in Evicted status - how can I delete these unwanted pods?
https://knowledge.broadcom.com/external/article/206249

Additional Information

See also

DX OI - How to reduce data retention to prevent a high disk usage severely impacting OI performance?
https://knowledge.broadcom.com/external/article/188786

DX OI - nfs directory almost full, many pods in Evicted status and lost access to Openshift console
https://knowledge.broadcom.com/external/article/198040

DX OI - Troubleshooting, Common Issues and Best Practices
https://knowledge.broadcom.com/external/article/190815/dx-oi-troubleshooting-common-issues-and.html

Attachments