After logging into the vRO web client a blue screen is displayed when using a vSphere Authentication provider
search cancel

After logging into the vRO web client a blue screen is displayed when using a vSphere Authentication provider

book

Article ID: 345167

calendar_today

Updated On:

Products

VMware Aria Suite

Issue/Introduction

This article has been published to document a known issue and the workaround which can be used to allow vRO to utilize the desired vCenter as the authentication provider.

Symptoms:

1. Opening up the vRO Web Client and attempting to login results in a blue screen in the web browser.


2. vCenter web sso logs show that there is a successful login from the vRO server:


2021-06-11T17:56:12.774Z INFO websso[68:tomcat-http--31] [CorId=0571xxxx-xxxx-xxxx-xxxx-xxxxxxxx7b8f] [com.vmware.identity.samlservice.impl.SAMLAuthnResponseSender] Posting successful authentication response to: https://vcoserver/vco/saml/websso/metadata
3. The vco-server-app.log indicates an I/O error when doing a GET for the SAML2 token because it is unable to find a valid certificate in the PKIX path:
2021-06-11T17:56:13.127Z WARN vco [host='vco-app-c7xxxxx99-pzlfq' thread='http-nio-8XX0-exec-4' user='-' org='-' trace='-'] {} com.vmware.o11n.web.SamlLogoutRequestor - Cannot logout principal: [[email protected]] from SSO Server.org.springframework.web.client.ResourceAccessException: I/O error on GET request for "https://vcenter/websso/SAML2/SLO/vsphere.local": PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target; nested exception is javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
...
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
...

4.  This has been observed when using a vCenter 7.x server as the authentication provider.


5.  The vCenter shows different casing for the endpoints in the lookup service, which is apparent by running the lstool.py list command or by viewing the lstool.txt in the commands folder on the vCenter log bundle.


Example:
/usr/lib/vmware-lookupsvc/tools/lstool.py list | grep "URL:"
...
URL: https://vcenter.fq.dn:443/sms/sdk
URL: https://VCENTER.fq.dn/websso/SAML2/Metadata/vsphere.local
...
The websso URL in particular may have issues with the casing.

Environment

VMware vRealize Orchestrator 8.4.x
VMware vRealize Orchestrator 8.1.x
VMware vRealize Orchestrator 8.x
VMware vRealize Orchestrator 8.0.x
VMware vRealize Orchestrator 8.3.x
VMware vRealize Orchestrator 8.2.x

Cause

This happens because vRO tries to match the lowercased name of the SAML2 token provider URI to the certificate name in the PKIX path, but is unable to find it because of the mixed or uppercase casing of the letters in the certificate.  The certificate is added to the vRO keystore when the authentication provider is registered.

Resolution

This is a known issue and will be fixed in a future release.  Please subscribe to the article to be notified of updates.

Workaround:

To work around this issue, the PNID of the vCenter must be lowercase, and all the endpoints in the lookup service must be lowercase.  This will produce a certificate which is lowercase for the SAML2 authentication provider and allow vRO to find the certificate in the PKIX path as a result.

Note: Before proceeding, please make sure there are full and valid backups of the vCenter Server and the vRO server.  Also be aware that other solutions such as vCD, NSX, SRM which utilize the lookup service will need to be re-registered/re-connected after this procedure.

1. Unregister the vCenter as the Authentication provider.


2. Check what the PNID is by navigating to the vCenter's VAMI and going to the network tab and examining the Hostname, and by running the following command:

/usr/lib/vmware-vmafd/bin/vmafd-cli get-pnid --server-name localhost

If the command and the Network tab in the VAMI displays a lowercased PNID, please skip to step 3.


3. If the name has uppercase characters, please resolve this by utilizing the documented method of changing the PNID.  The process will not work for just changing the case and will fail to set the CNs, so a temporary A Record FQDN pointing to the vCenter's same IP address will be required.  First change the hostname to the temporary FQDN name, and allow services to start in the vCenter.  Then change it back to the original FQDN, but with all lowercase letters, and allow all of the vCenter Services to start. Run the get-pnid command above again to verify the procedure was successful.


4. Use the lsdoctor.py -r, option 2 command to re-register the endpoints utilizing the latest PNID, documentation about the lsdoctor tool can be found in this KB article.  After the process successfully completes, verify that the vCenter related endpoints show lowercase casing: /usr/lib/vmware-lookupsvc/tools/lstool.py list | grep "URL:". 


5. Re-register the vCenter as the vRO Authentication Provider. Retry authentication to the vRO Web Client.  If it is still unsuccessful, please unregister the authentication provider and run through the following procedure to reset the lookup service and sso certificate in an SSH session to vRO:

a. kubectl -n prelude exec -it <vRO pod> -c vco-server-app -- bash (get <VRO pod> from 'kubectl -n prelude get pods')
b. rpm -hiv --nodeps /vco-cfg-cli.rpm
c. cd /usr/lib/vco-cli/bin/
d. ./vro-configure-inner.sh untrust --alias vco.vsphere.lookup-service.ssl.certificate
e. ./vro-configure-inner.sh untrust --alias vco.sso.ssl.certificate
f. Now re-register the authentication provider and it should ask to update the certificate.

After this, please try logging into the vRO Web Client again.  It should successfully authenticate.

Additional Information

Powered by Wolken Software