'Unavailable for Deployment' Errors Following vSphere Cloud Account certificate change

Products

• VCF Automation

Issue/Introduction

• You have recently changed the vSphere, NSX and SDDC manager certificate that is used as a Cloud Account in VCF Automation 9.1
• Data collection is failing with below error
  

  PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

• You notice errors similar to "Unavailable for Deployment or Failed to validate" when attempting to update the Cloud Account and see error message
• Error message similar to the one shown below are displayed on the Cloud Account view:
  

Environment

VCF Automation 9.1

Cause

The Cloud Account endpoints are retaining the old certificate information. The system is unable to establish a valid certification path to the requested targets because the certificates on the vSphere and NSX endpoints do not match the newly applied certificates.

Resolution

NOTE : Make sure to take a valid backup of VCFA appliance

Prerequisite 

1. Obtain a Bearer Token:

   • From the browser's Developer Tools > Network tab, copy the access bearer token.

Variable Mapping (Replace in calls below):

• ID3: VCF Automation FQDN

• ID7: Bearer token

To resolve this issue, you must manually patch the Cloud Account endpoints with the updated certificates via the API.

Follow the steps below to update the certificates for the vSphere and NSX endpoints, and then update the main VCF Cloud Account.

Step 1: Get the necessary cloud account details

Run the following API call to fetch the endpoint IDs for your Cloud Account:

curl -jk 'https://ID3/provisioning/uerp/provisioning/mgmt/endpoints?expand' \
--header 'Authorization: Bearer ID7' | \
jq ".documents | to_entries | map(select(.value.name == \"<Cloud Account Name>\")) | .[0] | {vcf: .key, nsx: .value.customProperties.nsx, vsphere: .value.customProperties.vsphere}"

Example Output:

{
  "vcf": "/resources/endpoints/########-####-####-####-############",
  "nsx": "/resources/endpoints/########-####-####-####-############",
  "vsphere": "/resources/endpoints/########-####-####-####-############"
}

Step 2: Capture the certificates in PEM format

Run the following commands to capture the required certificates.

Note: Save these outputs exactly as generated. The \n characters must remain inline.

For vCenter/vSphere (Leaf Certificate only):

openssl s_client -connect <vCenter_FQDN>:443 2> /dev/null | openssl x509 | awk 'NF {sub(/\r/, "");  printf "%s\\n",$0;}'

For NSX (Full Certificate Chain):

openssl s_client -connect <NSXEndpointFQDN>:443 -showcerts 2>/dev/null | awk '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/{sub(/\r/, ""); printf "%s\\n",$0;}'

Step 3: Check the endpoints

Verify the certificates for the NSX and vSphere links copied from Step 1.

curl -jk 'https://ID3/provisioning/uerp/resources/endpoints/<vc-endpoint-id>' --header 'Authorization: Bearer ID7' | jq . > <vc-endpoint-id>.json

curl -jk 'https://ID3/provisioning/uerp/resources/endpoints/<nsx-endpoint-id>' --header 'Authorization: Bearer ID7' | jq . > <nsx-endpoint-id>.json

Step 4: Patch endpoints with updated certificates

If the certificates do not match the valid certs fetched in Step 2, patch the individual cloud accounts.

Patch vCenter:

curl -jk --location --request PATCH 'https://ID3/provisioning/uerp/resources/endpoints/<vc-endpoint-id>' \
--header 'Authorization: Bearer ID7' \
--header 'Content-Type: application/json' \
--data-raw '{
     "endpointProperties": {
         "certificate": "<vsphere_leaf_certificate>"
     },
     "customProperties": {
         "certificate": "<vsphere_leaf_certificate>"
     }
}'

Patch NSX:

curl -jk --location --request PATCH 'https://ID3/provisioning/uerp/resources/endpoints/<nsx-endpoint-id>' \
--header 'Authorization: Bearer ID7' \
--header 'Content-Type: application/json' \
--data-raw '{
     "endpointProperties": {
         "certificate": "<nsx_full_chain_certificate>"
     },
     "customProperties": {
         "certificate": "<nsx_full_chain_certificate>"
     }
}'

Step 5: Update the VCF Cloud Account

Finally, update the main VCF endpoint itself, applying the leaf certificate to vSphere and the full chain to NSX.

curl -jk --location --request PATCH 'https://ID3/provisioning/uerp/resources/endpoints/<vcf-endpoint-id>' \
--header 'Authorization: Bearer ID7' \
--header 'Content-Type: application/json' \
--data-raw '{
     "endpointProperties": {
         "certificate": "<vsphere_leaf_certificate>",
         "nsxCertificate": "<nsx_full_chain_certificate>"
     },
     "customProperties": {
         "certificate": "<vsphere_leaf_certificate>",
         "nsxCertificate": "<nsx_full_chain_certificate>"
     }
}'