"Failed to update port persistence location" error reported when connecting VMs to distributed switch port groups.

Products

• VMware vSphere ESX 7.x
• VMware vSphere ESX 8.x

Issue/Introduction

• When the VMs are powered on with vNICs connected to DVS port group, the task fails with error,

"A general system error occurred: vDS Host error: Cannot create DVPort of vDS on the host.An error occurred during host configuration: Failed to update port persistence location."

• Connecting the VM to a standard switch port group is successful.
• Migrating the VM to another ESXi host and then powering on the VM still reports same error.
• Deleted the network adapter and powered ON the VM. Then recreating the vNIC and connecting still doesn't help with VM connectivity.

At the time when VM is connected to the port group, following events are recorded in Hostd logs:

[YYYY-MM-DDTHH:MM:SS] warning hostd[2191144] [Originator@6876 sub=Hostsvc.NetworkProvider opID=lw1rh3ck-11168345-auto-6ndju-h5:71205339-a1-01-10-08e7 user=vpxuser] Error saving  dvport 5488 to /vmfs/volumes/64a008f7-72****18-93ea-b85****f417c/.dvsData/XX 13 7b XX 2e 03 f5 **-** ** ** 5b 83 f5 42 88/5488: N7HostCtl3Lib16HostCtlExceptionE(Failed to write DV Port state to file /vmfs/volumes/64a008f7-72****18-93ea-b85****f417c/.dvsData/XX 13 7b XX 2e 03 f5 **-** ** ** 5b 83 f5 42 88/5488 Status (bad000a)= I/O error)
[YYYY-MM-DDTHH:MM:SS] info hostd[2191144] [Originator@6876 sub=Hostsvc.NetworkProvider opID=lw1rh3ck-11168345-auto-6ndju-h5:71205339-a1-01-10-08e7 user=vpxuser] deleted port file /vmfs/volumes/64a008f7-72****18-93ea-b85****f417c/.dvsData/XX 13 7b XX 2e 03 f5 **-** ** ** 5b 83 f5 42 88/5488 : 5488
...
2024-09-05T01:36:06.133Z info hostd[2191144] [Originator@6876 sub=Solo.Vmomi opID=lwXXh3ck-1XX68XX5-auto-6ndju-h5:XX205339-XX-01-10-XXXX user=vpxuser:<user_id>] Result:
--> (vim.fault.DvsApplyOperationFault) {
-->    objectFault = (vim.fault.DvsApplyOperationFault.FaultOnObject) [
-->       (vim.fault.DvsApplyOperationFault.FaultOnObject) {
-->          objectId = "XXXX",
-->          type = "vim.dvs.DistributedVirtualPort",
-->          fault = (vim.fault.PlatformConfigFault) {
-->             text = "Failed to update port persistence location",
-->             msg = "",
-->          }
-->       }
-->    ],
-->    msg = "",
--> }
…
…
[YYYY-MM-DDTHH:MM:SS] info hostd[2481720] [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 44089 : At least one corrupt resource metadata region was detected on volume 64a008f7-72****18-93ea-b85****f417c (<Datastore_name>). Other regions of the volume might be damaged too.
[YYYY-MM-DDTHH:MM:SS] info hostd[2481720] [Originator@6876 sub=Hostsvc.VmkVprobSource] VmkVprobSource::Post event: (vim.event.EventEx) {
-->    key = 90,
...
-->
[YYYY-MM-DDTHH:MM:SS] info hostd[2481720] [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 44090 : At least one corrupt resource metadata region was detected on volume 64a008f7-****9518-93ea-b85****417c (Datastore_name). Other regions of the volume might be damaged too.

 

At the timestamp, vmkernel logs also recorded:

[YYYY-MM-DDTHH:MM:SS] cpu2:4776106)WARNING: Res3: 7066: Volume 64a008f7-7****518-93ea-b85****f417c ("<Datastore_name>") might be damaged on the disk. Resource cluster metadata corruption has been detected.
[YYYY-MM-DDTHH:MM:SS] cpu2:4776106)WARNING: FS3: 608: VMFS volume <Datastore_name>/64a008f7-72af****-93ea-b859****417c on naa.600a09803831*******b55XX685a53:1 has been detected corrupted
[YYYY-MM-DDTHH:MM:SS] cpu2:4776106)FS3: 610: While filing a PR, please report the names of all hosts that attach to this LUN, tests that were running on them,
[YYYY-MM-DDTHH:MM:SS] cpu2:4776106)FS3: 634: and upload the dump by `voma -m vmfs -f dump -d /vmfs/devices/disks/naa.600a098038**********557865585a53:1 -D X`

Cause

• The datastore volume where the VM was deployed has corruption issue. Due to this, dvPort details cannot be modified on the ".dvsData" file for the VM.
• Until the datastore corruption issue is resolved, VMs configured on the reported datastore will continue to report the error when connecting to DVS port groups.

Resolution

Workaround:

• Perform storage vMotion of the VM to another datastore and then re-connect to DVS port group.

For any further assistance open a case with Broadcom support.

For more information refer to VOMA article.