After replacing the witness appliance on a 2 node robo cluster objects are reported as reduced no rebuild

Products

• VMware vSAN
• VMware vSAN 8.x

Issue/Introduction

• A 2-node ROBO cluster experienced vSAN objects in a "Reduced availability with no rebuild" state after the witness appliance was replaced.
• Even with a stretched storage policy, all data objects are in a "reduced availability with no rebuild" health state.

Environment

VMware vSAN 8

Cause

Shortly after the changing the witness appliance the data nodes started logging the following in clomd.log:

ESXi: /var/log/clomd.log

Post witness change the data nodes start logging the below:
####-##-##T##:##:##.###Z No(29) clomd[4310283]: [Originator@6876] CLOM_ReadParamsFromVsi: Setting CLOMGetAppCxt()->isMetroCluster to FALSE
####-##-##T##:##:##.###Z No(29) clomd[4310283]: [Originator@6876] CLOM_ReadParamsFromVsi: Setting CLOMGetAppCxt()->isMetroCluster to FALSE
####-##-##T##:##:##.###Z No(29) clomd[4310283]: [Originator@6876] CLOM_ReadParamsFromVsi: Setting CLOMGetAppCxt()->isMetroCluster to FALSE

Here the data node has logged that it was in a stretched cluster previously then post witness change it no longer is seen as part of a stretched cluster.

####-##-##T##:##:##.###Z Cr(26) clomd[2099762]: [Originator@6876 opID=1804292291] CLOM_CheckClusterResourcesForPolicy: in stretched cluster, 0 witness disks available, needed at least 1 witness disks
####-##-##T##:##:##.###Z No(29) clomd[4309796]: [Originator@6876] main: Is in stretched cluster mode? No
####-##-##T##:##:##.###Z No(29) clomd[4310283]: [Originator@6876] main: Is in stretched cluster mode? No

 

To check the stretchedClusterMode value run the below command on only the 2 data nodes in the robo cluster. 

vsish -e get /vmkModules/vsanutil/stretchedClusterMode

Resolution

In a 2-node ROBO cluster, both data nodes must have the stretchedClusterMode value set to 1.

vsish -e set /vmkModules/vsanutil/stretchedClusterMode 1