The objective is to deploy the API portal version 5.3.1.
The process is automated by Argo CD which pulls the chart from a GIT repo. The first two jobs go well, until the yyyyy-tl-manager job runs which creates the secrets :
$ kubectl logs yyyyy-tls-manager-dx555
This is an install, proceeding..
Generating certificate: apim-ssl
Generating certificate: apim-datalake
Generating certificate: dispatcher-ssl
Generating certificate: apim-dssg
Generating certificate: pssg-ssl
Generating certificate: apim-solr
Generating certificate: tps
Generating certificate: apim-tps
writing RSA key
Purging temporary folder ..
Updating Kubernetes Secrets
E0410 08:59:35.294853 134 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \https://xxxxxxxxxxx1:443/api?timeout=32s\: EOF" E0410 08:59:45.353674 134 memcac
he.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \https://xxxxxxxxxxx1:443/api?timeout=32s\: EOF" E0410 08:59:55.412513 134 memcache.go:265] "Unhandled Error" err
="couldn't get current server API group list: Get \https://xxxxxxxxxxx1:443/api?timeout=32s\: EOF" E0410 09:00:05.469377 134 memcache.go:265] "Unhandled Error" err="couldn't get current server AP
I group list: Get \https://xxxxxxxxxxx1:443/api?timeout=32s\: EOF" E0410 09:00:15.528036 134 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \https://xxxxxxxxxxx1:443/api?timeout=32s\: EOF" Unable to connect to the server: EOF
error: error validating "STDIN": error validating data: failed to download openapi: Get https://xxxxxxxxxxx1:443/openapi/v2?timeout=32s: EOF; if you choose to ignore these errors, turn validation off with --validate=false
Purging temporary folder..
E0410 09:00:45.894397 175 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \https://xxxxxxxxxxx1:443/api?timeout=32s\: EOF" E0410 09:00:55.975517 175 memcac
he.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \https://xxxxxxxxxxx1:443/api?timeout=32s\: EOF" E0410 09:01:06.041580 175 memcache.go:265] "Unhandled Error" err
="couldn't get current server API group list: Get \https://xxxxxxxxxxx1:443/api?timeout=32s\: EOF" E0410 09:01:16.099591 175 memcache.go:265] "Unhandled Error" err="couldn't get current server AP
I group list: Get \https://xxxxxxxxxxx1:443/api?timeout=32s\: EOF" E0410 09:01:26.154108 175 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \https://xxxxxxxxxxx1:443/api?timeout=32s\: EOF" Unable to connect to the server: EOF
error: error validating "STDIN": error validating data: failed to download openapi: Get https://xxxxxxxxxxx1:443/openapi/v2?timeout=32s: EOF; if you choose to ignore these errors, turn validation off with --validate=false
Purging temporary folder..
When this job finishes, the next job "yyyyy-ingress-nginx-admission-create" keeps failing:
0410 09:41:01.904026 1 client_config.go:667] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
{"err":"Get \https://xxxxxxxxxxx1:443/api/v1/namespaces/apiportal-gaa-test-01/secrets/yyyyy-ingress-nginx-admission\: EOF","level":"fatal","msg":"error getting secret","source":"k8s/k8s.go:232","time":"2025-04-10T09:41:11Z"}
I can't see that this secret ever being made but we got a moment where a cloud engineer saw this error:
Error: couldn't find key apim-ssl.p12 in Secret apiportal-gaa-test-01/portal-external-secret
The created portal-external-secret and portal-internal-secret that are empty and every time the yyyyy-tl-manager job runs, it appears a secret is being created in the AKS cluster "yyyyy-job-secret". This secret contains the certificates and keys.
API Portal 5.3.1
The ClusterRole yyyyy-ingress-nginx-admission currently has permissions for validating webhook configurations and pod security policies but it doesn’t have permissions to create pods or secrets.
Manually create the yyyyy-ingress-nginx-admission secret