Enabling or change vSAN encryption from Native KMS to External fails with error: NCERRInsufficientPermissions

Products

• VMware vSAN 8.x

Issue/Introduction

Issue is presented when trying to enable vSAN encryption or it can be reported when trying to change vSAN Encryption from Native KMS to External KMS Provider

From /var/log/vmware/vsan-health/vsanvcmgmtd.log user might see bellow entries.

2025-09-15T11:19:39.590+01:00 ERROR vsan-mgmt[09677] [VsanVcClusterConfigSystemImpl::_Run opID=agw-0004067-52bb] Failed to reconfigure vSAN cluster, (vmodl.RuntimeFault) {
  msg = "Received SOAP response fault from [<<io_obj p:0x00007f0cdc4232d8, h:172, <UNIX ''>, <UNIX '/var/run/envoy-hgw/hgw-pipe'>>, /hgw/host-xxxx/vsan>]: generateClusterKeys\nN7HostCtl3Lib4Vsan16VsanCtlExceptionE: Failed to create KEK from KMS cluster: Provider xxxxxx KMS"
}
Traceback (most recent call last):
  File "bora/vsan/clusterconfig/vpxd/pyMoVsan/VsanVcClusterConfigSystemImpl.py", line 11718, in _Run
  File "bora/vsan/clusterconfig/vpxd/pyMoVsan/VsanVcClusterConfigSystemImpl.py", line 11702, in _ExecuteInternal
  File "bora/vsan/clusterconfig/vpxd/pyMoVsan/VsanVcClusterConfigSystemImpl.py", line 9349, in Execute
  File "bora/vsan/clusterconfig/vpxd/pyMoVsan/VsanVcClusterConfigSystemImpl.py", line 9549, in _ExecuteOperations
  File "bora/vsan/clusterconfig/vpxd/pyMoVsan/VsanVcClusterConfigSystemImpl.py", line 9603, in _ExecuteCriticalOperation
  File "bora/vsan/clusterconfig/vpxd/pyMoVsan/VsanVcClusterConfigSystemImpl.py", line 9679, in _PreTaskOperation
  File "bora/vsan/clusterconfig/vpxd/pyMoVsan/VsanVcClusterConfigSystemImpl.py", line 3208, in PreTaskAction
  File "bora/vsan/clusterconfig/vpxd/pyMoVsan/VsanVcClusterConfigSystemImpl.py", line 3095, in _SetClusterKeys
  File "bora/vsan/clusterconfig/vpxd/pyMoVsan/VsanVcClusterConfigSystemImpl.py", line 2999, in _GenerateClusterKeys
  File "bora/vsan/clusterconfig/vpxd/pyMoVsan/VsanVcClusterConfigSystemImpl.py", line 2956, in _SelectOneHostToGenerateKeys
  File "bora/vsan/clusterconfig/vpxd/pyMoVsan/VsanVcClusterConfigSystemImpl.py", line 2943, in _GenerateClusterKeysFromHost
  File "bora/vsan/vsanmgmtd/vpxd/VsanSystemProxy.py", line 163, in _method
  File "bora/vsan/vsanmgmtd/vpxd/VsanSystemProxy.py", line 109, in _CallToVsanmgmtd
  File "/usr/lib/vmware/site-packages/pyVmomi/VmomiSupport.py", line 618, in <lambda>
  File "/usr/lib/vmware/site-packages/pyVmomi/VmomiSupport.py", line 391, in _InvokeMethod
pyVmomi.VmomiSupport.vmodl.RuntimeFault: (vmodl.RuntimeFault) {
  msg = "Received SOAP response fault from [<<io_obj p:0x00007f0cdc4232d8, h:172, <UNIX ''>, <UNIX '/var/run/envoy-hgw/hgw-pipe'>>, /hgw/host-xxxx/vsan>]: generateClusterKeys\nN7HostCtl3Lib4Vsan16VsanCtlExceptionE: Failed to create KEK from KMS cluster: Provider xxxxx KMS"
}

 

From /var/log/vsansystem.log user might notice the following error:

025-09-11T10:22:55.871Z Er(163) vsansystem[2101698]: [vSAN@6876 sub=Default opId=agw-0002979-1c62-74bd] {2101698} :0 qlc_eh_err() - Server Error:General Failure, Explanation:[NCERRInsufficientPermissions]:

Environment

vSAN 8.0.x

Cause

When enabling vSAN Encryption on vSAN cluster or changing from Native to External KMS, all users created for vCenter assigned on the KMS server might not have enough privileges. 

Resolution

Contact technical team from KMS provider for further assistance.