Unable to retrieve vSAN related information in HTML 5 client after updating to VC 6.7 U3
search cancel

Unable to retrieve vSAN related information in HTML 5 client after updating to VC 6.7 U3

book

Article ID: 326678

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

Symptoms:

Error reported in the GUI 
In Cluster -> Monitor -> Virtual Objects: "Failed to extract requested data. Check vSphere Client logs for details."
In Cluster -> Monitor -> Performance: "Unable to query data. Try again later."
In Cluster -> Monitor -> Capacity: "Failed to extract requested data. Check vSphere Client logs for details."

Below error in GUI 

Log extracts:

Please note the UUID will be unique to each environment

/var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log

[2019-09-06T12:25:05.512+02:00] [ERROR] http-nio-5090-exec-7 com.vmware.vsan.client.services.ProxygenController service method failed to invoke java.lang.reflect.InvocationTargetException: null
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.vmware.vsan.client.services.ProxygenController.invokeService(ProxygenController.java:130)
        at com.vmware.vsan.client.services.ProxygenController.invokeServiceWithJson(ProxygenController.java:64)
        at sun.reflect.GeneratedMethodAccessor544.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)

Caused by: com.vmware.vsphere.client.vsandp.core.sessionmanager.common.NotAuthenticatedException: Authentication information for the specified site is missing. Authenticate with the site and try again.
        at com.vmware.vsphere.client.vsandp.core.sessionmanager.common.VcClient.getConnection(VcClient.java:91)
        at com.vmware.vsphere.client.vsandp.core.sessionmanager.common.VcClient.getConnection(VcClient.java:67)
        at com.vmware.vsphere.client.vsandp.core.sessionmanager.common.VcClient.getConnection(VcClient.java:63)
        at com.vmware.vsan.client.services.common.CeipService.getCeipServiceEnabled(CeipService.java:42)
        ... 119 common frames omitted
Caused by: com.vmware.vsphere.client.vsandp.core.sessionmanager.vlsi.client.sso.tokenstore.NoTokenException: No token for site: 8AD6D1A1-6861-4EF2-AC79-CD2DB0B3C5E1

Environment

VMware vSAN 6.7.x

Cause

The root cause of this issue is a mismatch in the vCenter Server Service ID registration.

Before upgrading, the Service ID values were lowercase. After upgrading, it becomes UPPERCASE.
On the other hand in vSAN session management code (responsible to acquire connection to VC, vSAN, etc.), there is a bug when checking whether the site is authenticated. It first converts the ID to UUID which seems to turn it to lowercase. After that when actually getting the connection, it doesn't do this conversion causing the site to always remain unauthenticated.

During the upgrade, the letters in the Service ID changes from lowercase to uppercase which is causing the issue in combination with the incorrect way the Service ID is being searched for by the vSAN session management code.

Resolution

At this time, there is no resolution. However, we can identify the issue using the following steps on the PSC/VC (if embedded):

Live Troubleshooting to verify if Service Id is Uppercase

1. SSH to the PSC whether external or embedded and log in as root

2. Create the file listing all Service Registrations

# /usr/lib/vmidentity/tools/scripts/lstool.py list --url http://localhost:7080/lookupservice/sdk --no-check-cert > /tmp/listservices.txt

3. Output all “Service Type: vcenterserver” IDs with their associated vCenter URL

# cat /tmp/listservices.txt |grep "Service Type: vcenterserver" -A9 |egrep "Service ID|URL"

4. Determine if the letters in Service ID are UPPERCASE or lowercase


Workaround:

Use the vSphere Flash-based Web Client instead to manage vSAN related functions until fix is verified.

Additional Information

vmware-vsan-health-service.log

vmware-vsan-health-service-5148.log:2019-08-24T08:19:52.57Z ERROR vsan-mgmt[EventMonitor] [VsanHealthHelpers::convertObjectHealthState opID=noOpId] Cannot find the object health health state VsanObjectHealthState_Unknown
vmware-vsan-health-service-5148.log:2019-08-24T08:19:54.568Z ERROR vsan-mgmt[EventMonitor] [VsanHttpRequestWrapper::urlopen opID=noOpId] Exception while sending request : The read operation timed out
vmware-vsan-health-service-5148.log:2019-08-24T08:19:57.845Z ERROR vsan-mgmt[EventMonitor] [VsanVcClusterHealthSystemImpl::_GetConsistentConfigTest opID=noOpId] Failed to fetch unicast info on host xxxxxxxx-xxx.xxx.xx
vmware-vsan-health-service-5148.log:2019-08-24T08:19:58.882Z ERROR vsan-mgmt[EventMonitor] [VsanHealthHelpers::convertObjectHealthState opID=noOpId] Cannot find the object health health state VsanObjectHealthState_Unknown
vmware-vsan-health-service-5148.log:2019-08-24T08:25:27.753Z ERROR vsan-mgmt[Thread-308794] [VsanHttpRequestWrapper::urlopen opID=noOpId] Exception while sending request : The read operation timed out
vmware-vsan-health-service-5148.log:2019-08-24T08:26:10.968Z ERROR vsan-mgmt[EventMonitor] [VsanVcClusterHealthSystemImpl::_GetConsistentConfigTest opID=noOpId] Failed to fetch unicast info on host xxxxxxxx-xxx.xxx.xx
vmware-vsan-health-service-5148.log:2019-08-24T08:26:12.5Z ERROR vsan-mgmt[EventMonitor] [VsanHealthHelpers::convertObjectHealthState opID=noOpId] Cannot find the object health health state VsanObjectHealthState_Unknown
vmware-vsan-health-service-5148.log:2019-08-24T08:26:14.518Z ERROR vsan-mgmt[EventMonitor] [VsanHttpRequestWrapper::urlopen opID=noOpId] Exception while sending request : The read operation timed out
vmware-vsan-health-service-5148.log:2019-08-24T08:28:48.718Z ERROR vsan-mgmt[EventMonitor] [VsanHealthHelpers::convertObjectHealthState opID=noOpId] Cannot find the object health health state VsanObjectHealthState_Unknown
vmware-vsan-health-service-5148.log:2019-08-24T08:28:51.366Z ERROR vsan-mgmt[EventMonitor] [VsanHttpRequestWrapper::urlopen opID=noOpId] Exception while sending request : The read operation timed out

Powered by Wolken Software