Windows session authentication doesn't load in Firefox version 54 and later when using the Enhanced Authentication Plugin

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• Windows session authentication doesn't load in Firefox, but is working as expected in IE and Chrome.
• You get prompt to launch the Enhanced Authentication Plugin, however nothing happens after the "Open Link" action.
• You are connecting to vCenter version 6.5 or higher.
• Reinstalling the Enhanced Authentication Plugin does not resolve the issue.
• In the C:\ProgramData\VMware\CIP\cipMsgProxy\sessions\session_xxxx\logs\wss_8094_0.log file, you see entries similar to:

[<YYYY-MM-DD> <TIME>] [info] asio async_read_at_least error: system:10053 (An established connection was aborted by the software in your host machine)
[<YYYY-MM-DD> <TIME>] [error] handle_read_handshake error: websocketpp.transport:2 (Underlying Transport Error)
[<YYYY-MM-DD> <TIME>] [fail] WebSocket Connection [::1]:64064 - "" - 0 websocketpp.transport:2 Underlying Transport Error
[<YYYY-MM-DD> <TIME>] [info] asio async_shutdown error: system:10053 (An established connection was aborted by the software in your host machine)

Environment

VMware vCenter Server Appliance 6.7.x
VMware vCenter Server 6.7.x
VMware vCenter Server 7.0.x
VMware vCenter Server Appliance 6.5.x
VMware vCenter Server 6.5.x

Cause

Firefox works with secure websockets (wss://). For this the certificate of the site needs to be trusted. Firefox does not allow the import of self-signed certificates as Certificate Authorities.

This issue occurs only in Firefox version 54 and later due to security changes.

For more information, see Mozilla Bug 1373791.

Resolution

To resolve the issue, follow the steps below:

1. Open Mozilla Firefox.
2. In the address bar, navigate to about:preferences#privacy
3. Under Certificates, select View Certificates...
4. Select Servers > Add Exception... and enter the URL below into Location:

https://vmware-plugin:8094

5. Click Get Certificate
   
6. Select Confirm Security Exception.

 

Alternatively, you may also add a security exception using the steps below:

1. Open Mozilla Firefox.
2. In the address bar, navigate to https://vmware-plugin:8094
3. Select Add Security Exception.

If the above doesn't resolve the issue, try importing vCenter Server Root certificates into the Firefox local Certificate store. For more information, see Mozilla KB Setting Up Certificate Authorities (CAs) in Firefox.