AIOps - Unable to login to DX Platform - jarvis-elasticsearch pods evicted

Products

• DX Operational Intelligence
• CA App Experience Analytics
• DX Application Performance Management

Issue/Introduction

"Failed to login. Verify that user ID and password are correct"

Cause

DX Authorization is controlled by AXA services, the message indicates and there is some issue with either axa services, postgres data or elastic nodes.

In this case, the problem was caused by a lack of disk space in one of the Elastic Node, the below message was found when running: kubectl describe <pod>

kubectl describe po jarvis-elasticsearch-3-7984bc48f9-zzp2j -n dxi
Name:           jarvis-elasticsearch-3-7984bc48f9-zzp2j
Namespace:      dxi
Node:           node5/
Start Time:     Fri, 28 Feb 2020 05:17:36 +0000
Labels:         app=jarvis-elasticsearch-3
                app-svc=jarvis-elasticsearch
                pod-template-hash=3540670495
                product_id=jarvis
                type=core
Annotations:    <none>
Status:         Failed
Reason:         Evicted
Message:        Pod The node was low on resource: [DiskPressure].
IP:
Controlled By:  ReplicaSet/jarvis-elasticsearch-3-7984bc48f9
..


Elastic node was configured with 200G only

As per documentation
https://techdocs.broadcom.com/content/broadcom/techdocs/us/en/ca-enterprise-software/it-operations-management/dx-platform-on-premise/1-0/installing/sizing-recommendations.html#concept.dita_9b35cd55dbc2557dd8739a25e94219ccf1842563_SizingProcess1

Elastic Nodes requires 500 GB SSD

Environment

DX Operational Intelligence 1.3.x, 20.2
DX Application Performance Management 11.x, 20.2
DX AXA 20.2

Resolution

Increase disk space in the Elastic nodes

Additional Information

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/dx-platform-on-premise/20-2/installing/software-requirements.html