Standalone Orchestrator with custom tls certificate is inaccessible after upgrading to 8.18.1

Products

• VMware Aria Suite

Issue/Introduction

• The Orchestrator client UI fails to load after upgrading to version 8.18.1
• Upgrade was reported as succesfully
• The /opt/scripts/deploy.sh script completes with the expected service initialization completed message logged in the /services-logs/prelude/vco-app/file-logs/vco-server-app.log:
  org.springframework.web.servlet.DispatcherServlet - Completed initialization in XXXX ms

• the contour logs located under /services-logs/prelude/contour/console-logs/contour.log contain an invalid TLS private key messasge similar to:
  
  

  time="" level=info msg="performing delayed update" context=contourEventHandler last_update=3m43.704036126s outstanding=1
  time="" level=error msg="unresolved secret reference" context=IngressProcessor error="invalid TLS private key: x509: failed to parse private key (use ParsePKCS8PrivateKey instead for this key format)" name=dispatch namespace=prelude secret=prelude/contour-ingress-cert-and-key 

Environment

VMware Aria Automation Orchestrator 8.18.1

Cause

The issue occurs when a PKCS#8 key (should be encapsulated between -{}{}{}BEGIN PRIVATE KEY{}{}{}- END PRIVATE KEY) has been encapsulated with the PKCS#1 format (-{}{}{}BEGIN RSA PRIVATE KEY{}{}  {}{}-{}{}END RSA PRIVATE KEY{}{}-- )


New ingress controller counter performs a more strict check of certificate formats. Earlier versions warned against such a scenario but allowed it to be applied using the -force option of vracli certificate ingress --set command.

Resolution

To resolve the issue:

You can remove the "RSA" work form the private key's begin and end sections:

A) This can be done before setting the certificate with the vracli command, directly in the chain text file.

Or 

B) directly by updating it in kubernetes on the upgraded system:

1) SSH to appliance as root user and run command:

kubectl -n prelude edit vaconfigs.prelude.vmware.com prelude-vaconfig 

2) Locate section

certificates:
  ingress:

3) Rename 

----BEGIN RSA PRIVATE KEY----
to 

----BEGIN PRIVATE KEY----
and 

----END RSA PRIVATE KEY----
to 

----END PRIVATE KEY----

4) Exit & Save with:

:wq

5) restart services:

/opt/scripts/deploy.sh