VCFA: Errors seen when viewing services or using the catalog "Services are not available for this namespace" / "Error reading virtualMachineImage list: VCFA: Failed to fetch client certificate"

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

All of the following are seen:

• On the Services -> Overview page within the tenant portal, the following is shown:

  • Services are not available for this namespace, try again later or check with your administrator.

• Trying to request an item from the catalog, this banner error is shown:

  • Some data cannot be retrieved. If the problem persists, contact your system administrator. Failed request: /data/namespaces.; Some data cannot be retrieved. If the problem persists, contact your system administrator. Failed request: projects.; [1][namespace.valueList][/data/namespaces] Error getting external data: Internal Server Error [Error Reference ID: <UUID>]; [2][project.valueList][projects] Error getting external data: Internal Server Error [Error Reference ID: <UUID>];

• At this time, the catalog service shows this text in an ERROR in its logfile /var/log/services-logs/prelude/catalog-service-app/file-logs/catalog-service-app.log
  • c.v.s.webmvc.error.RestExceptionHandler - API error 500 with response {"message":"500 : \"{\"apiVersion\":\"v1\",\"code\":500,\"details\":null,\"kind\":\"Status\",\"message\":\"Error reading virtualMachineImage list: VCFA: Failed to fetch client certificate, status message=authentication failed [Error Reference ID: <UUID>]\",\"metadata\": null,\"reason\":\"InternalServerError\",\"status\":\"Failure\"}\"

• If we check the logs within the vCenter Supervisor Control Plane VM (CPVM), we see an error similar to the following:
  • {"level":"error","timestamp":"########","logger":"oidc-upstream-observer","caller":"/work/internal/controller/supervisorconfig/oidcupstreamwatcher/oidc_upstream_watcher.go:374$oidcupstreamwatcher.(*oidcWatcherController).validateIssuer","message":"failed to perform OIDC discovery","namespace":"vmware-system-pinniped","name":"oidc-idp","issuer":"https://<VCFA_FQDN>/oidc","error":"Get \"https://<VCFA_FQDN>/oidc/.well-known/openid-configuration\": proxyconnect tcp: dial tcp <IP_address>:80: i/o timeout"} 

Environment

VCF Automation 9.x

Cause

There is a proxy set on the Supervisor CPVM. This may have been inherited from VCFA on creation

Resolution

Check and remove proxy configuration for the supervisor:

1. Go to vCenter UI: > Supervisor Management > (select supervisor) > Network > Proxy Configuration
2. Remove these proxy configuration settings, if any, and retry.

 

This could also be due to proxy settings set at VC level:

1. Come to the VAMI at https://<vcenter>:5480 
2. In the menu, choose Networking > Proxy Settings
3. Remove this proxy, if any, and retry.