Unable to access harbor UI deployed as Supervisor Service. "invalid TLS certificate" unexpected block type 'PRIVATE KEY'

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

• Users are unable to access the Harbor UI despite the package being deployed successfully on Supervisor. Further verification shows that the Virtual Service VIP in AVI for the Contour is down, indicating a potential issue with the service.

• Virtual Service VIP status shows down in AVI :
  

• Cannot access the contour/envoy containers from Supervisor:

 

Environment

VMware vSphere Kubernetes Service

Cause

The failure to access the Harbor UI and the "Down" status of the Virtual Service VIP in AVI is caused by a TLS configuration mismatch within the Harbor Supervisor configuration.

This typically occurs due to one of the following factors:

• Malformed Private Key: The tls.key provided in the configuration is not in a valid PEM-encoded format or contained non-standard block types that prevented the ingress controller (Contour) from successfully performing a TLS handshake.

• Certificate/Key Mismatch: The existing certificate pair is out of sync, meaning the public certificate (tls.crt) did not mathematically correspond to the private key being used.

• Incomplete Trust Chain: The configuration lacked the necessary intermediate or Root CA certificates in the ca.crt or tls.crt fields, causing the AVI Load Balancer's health checks to fail when attempting to verify the backend's identity.

 

Please see: Install Harbor with a Custom Certificate for more details.

Resolution

The issue is resolved by regenerating the certificate signing request (CSR) and ensuring the private key is in the correct PEM format before updating the Harbor configuration.

1. Regenerate Certificate:
   • Regenerate a new Certificate Signing Request (CSR) for Harbor (e.g., using openssl utility).
   • Have the new CSR signed by your internal Certificate Authority (CA).
2. Verify and Update Configuration: (See: Install Harbor with a Custom Certificate for more details.)
   
   • Ensure the private key (tls.key) obtained from the signing process is in the correct PEM format (e.g., beginning with -----BEGIN PRIVATE KEY----- or -----BEGIN RSA PRIVATE KEY-----) and does not contain any unexpected block types.
   • Update the Harbor configuration on the Supervisor with the complete and correctly formatted certificate chain. The imported certificate chain structure in tlsCertificate: should look something like
     
tlsCertificate:
  tlsSecretLabels: {"managed-by": "vmware-vRegistry"}
  tls.crt: |
    -----BEGIN CERTIFICATE-----
    MIIByTCCAW6gAwIBAgIRAP19vYR/8UXAOv6MNUAKNE8wCgYIKoZIzj0EAwIwJDEQ
    ...
    -----END CERTIFICATE-----
  tls.key: |
    -----BEGIN PRIVATE KEY-----
    MHcCAQEEIC2gYq8nfu3tbmhoqsBNvU5Jp/kK0dQa45797QjcLOF9oAoGCCqGSM49
    ...
    -----END PRIVATE KEY-----        
  ca.crt: |
    -----BEGIN CERTIFICATE-----
    MIIBnTCCAUOgAwIBAgIQbnzh5NbwWN6E9xaJhrv4yTAKBggqhkjOPQQDAjAkMRAw
    -----END CERTIFICATE-----

   
   
3. Final Verification:
   Once the configuration is updated with the valid certificate and deployed, the Harbor UI access should be restored, and the Virtual Service VIP in AVI for the Contour should show as up. 

 

Additional Information

Reference: Install Harbor with a Custom Certificate