DX 20.2 - doi adminui and axaservices-notify-filter pods not starting

Products

• DX Operational Intelligence
• DX Application Performance Management
• CA App Experience Analytics

Issue/Introduction

After the installation of DXAPM 20.2 there are 2 pods that are not starting 

# kubectl get pods -ndxi -o wide | grep Init
axaservices-notify-filter-689cb64cb9-n2l9f             0/1     Init:0/1           0          3h43m   10.10.10.32    myserver.mydomain.com        <none>           <none>
doi-adminui-679477dc76-gmc6m                           0/1     Init:3/4           0          3h43m   10.10.10.32    myserver.mydomain.com        <none>           <none>

Restarting everything does not help

Environment

Release : 20.2

Component : CA DOI Foundations

Cause

Both the doi-adminui and axaservices-notify-filter pods check for the existence of a file dxi-platform-init-pod/complete in a loop during their initialisation as can be seen in the yaml extract

 

 "initContainers": [
      {
        "name": "init-notify-dxi-platform-init-pod",
        "image": "myserver.mydomain.com:5000/dxi/acc/busybox:1",
        "command": [
          "/bin/sh",
          "-c",
          "while [ ! -f /log/dxi-platform-init-pod/complete ]; do echo \"dxi-platform-init-pod/complete - File doesn't Exist\"; sleep 5; done; echo \"dxi-platform-init-pod/complete - File Exists\" "
        ],

 

This file is created when the pod dxi-platform-init-pod completes - this is called during installation.

 

In this case that file was missing and checking the dxi-platform-init-pod.log (available in the nfs share under folder dxi-platform-init-pod) showed that this pod had not completed and so had not created the file

[ RUNNING 1.reset.masteradmin.password.sh]
Waiting for adminui to be up...............................................................................................................................................................................................................................................................................................................
Resetting password for masteradmin [Cohort : masteradmin]

Obtaining security token from ESS...
STATUS:..........................................................................................
Failed to obtain authorization token... Exiting.

[ END 1.reset.masteradmin.password.sh ]


[ RUNNING 2.create.ga.account.sh]
Waiting for adminui to be up............................................................................................

Resolution

Re-executing the pod dxi-platform-init-pod allowed the initialisation to complete and then the 'complete' file was created. The doi-admin and axaservices-notify-filter  pods were then able to start.