Error: "Failed to update certificate authorities" when configuring Microsoft CA - VMware Cloud Foundation 9.1

Products

• VCF Operations

Issue/Introduction

When attempting to configure a Microsoft Certificate Authority (CA) within the VCF Operations console for certificate management, the operation fails.

Symptom:

• The UI displays the error: Failed to update certificate authorities. Please verify the provided input values and try again.
• Users may observe that manual certificate imports into the SDDC Manager truststore do not resolve the issue in the VCF Operations interface.
• The operationsmanager.log on the SDDC Manager typically shows a java.net.SocketException: Connection reset or SSLPeerUnverifiedException when calling the MicrosoftCaPlugin.

Environment

VCF Operations 9.1.0.x

Microsoft Certificate Authority (IIS-based Web Enrollment)

Cause

The configuration failure is often caused by a combination of the following factors in the VCF 9.1 architecture:

1. IIS Configuration: The Microsoft CA Web Enrollment site (/certsrv) is not configured to allow Basic Authentication, which is required for the automated API calls from VCF Operations.
2. SSL Handshake Failure: The IIS server is configured to "Require" or "Accept" client certificates, causing a connection reset when VCF Operations attempts to connect without a client-side certificate.
3. Patch Level: Early releases of VCF 9.1.0.0 have known limitations in the Microsoft CA integration plugin that are addressed in subsequent maintenance patches.

Resolution

To resolve this issue, perform the following steps:

1. Prepare Microsoft CA for Basic Authentication:

   1. Log in to the Microsoft CA server.
   2. Open IIS Manager and navigate to Default Web Site > CertSrv.
   3. In the Authentication section, ensure Basic Authentication is Enabled.
   4. In the SSL Settings section, ensure Client Certificates is set to Ignore.
   5. Restart the IIS service by running iisreset from an administrative command prompt.

2. Verify Service Account and Template:

   1. Confirm the service account has Read and Enroll permissions on the Certificate Template (default friendly name: "VMware").
   2. Ensure the service account can manually log in to the https://<ca_fqdn>/certsrv URL from a browser.

3. Upgrade to VCF Operations 9.1.0.0400:

   • Broadcom recommends upgrading to VCF Operations 9.1.0.0400 or higher, as this release contains specific improvements for Microsoft CA (MSCA) integration and truststore handling.
   • Refer to the VCF Operations 9.1.0.0400 Release Notes for detailed fix information.

4. Re-attempt Configuration:

   1. Log in to VCF Operations.
   2. Navigate to Fleet Management > Certificates.
   3. Click Configure CA and re-enter the Microsoft CA details.

Workaround

If an upgrade is not immediately possible, ensure the root and subordinate CA certificates are manually trusted by the SDDC Manager:

1. SSH to the SDDC Manager as vcf and switch to root.
2. Import the CA certificate to the Java keystore:
   echo off | openssl s_client -connect <ca_fqdn>:443 2>&1 | openssl x509 -outform PEM 1> /tmp/ca.crt && keytool -importcert -alias msca_root -file /tmp/ca.crt -keystore /etc/alternatives/jre/lib/security/cacerts --storepass changeit
3. Restart SDDC Manager services:
   echo 'y' | /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh

Additional Information

Configure a Certificate Authority for VMware Cloud Foundation

Troubleshooting Microsoft CA configuration failures in VCF Operations 9.0.x