High response time accessing endpoints exposed by signin service (e.g. /signin/token)

Products

• VIP Authentication Hub

Issue/Introduction

Looking at the log of ingress controller pod, the response time of accessing endpoints exposed by signin service (e.g. /signin/token) is high. For example, in below environment, the 'GET /default/ui/v1/signin/token' request's response time is 4.766 seconds (the 3rd last field in the log)

[ssp1@w-manage ~]$ kubectl get pods -n ingress
NAME                                        READY   STATUS    RESTARTS   AGE
ingress-nginx-controller-54d8b558d4-9m96m   1/1     Running   0          12d

[ssp1@w-manage ~]$ kubectl logs ingress-nginx-controller-54d8b558d4-9m96m -n ingress | grep 'GET /default/ui/v1/signin/token' | grep -v notice | grep "[1-9].... 200"
xxx.xxx.xxx.xxx - - [28/Feb/2023:00:10:11 +0000] "GET /default/ui/v1/signin/token HTTP/2.0" 200 1053 "https://wsssp.demo-broadcom.com/default/ui/v1/signin/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36" 189 4.766 [ssp-ssp-ssp-signin-443] [] xx.xx.x.x:3000 1070 4.766 200 274848324b8d82717771492b9decfb10

Entropy is not an issue. Running below command shows entropy value is greater than 3000.

kubectl run busybox --image=busybox:1.28 --rm -ti --restart=Never --command -- cat /proc/sys/kernel/random/entropy_avail

Environment

Release : VIP Authentication Hub 2022.Jun.04,  ssp-1.0.2946

Cause

CPU Requests/Limits for the signin pods is set lower to recommended values, e.g. 100m/200m

Resolution

Increase CPU Requests/Limits to 600m/2000m for the signin pod.

We can run
   kubectl edit deployment ssp-ssp-signin -n ssp

search for "cpu" and edit the resources section as shown below

to confirm you can run below command and check the modification is done.

  kubectl describe deploy/ssp-ssp-signin -n ssp