Converting a VMware virtual machine snapshot to a memory dump with vmss2core.exe fails with the following error :
Unrecognized .vmss file (magic 4f434e45)
Example:
#vmss2core.exe -W8 snapshot.vmsn snapshot.vmem
vmss2core version 8456865 Copyright (C) 1998-2017 VMware, Inc. All rights reserved.
Unrecognized .vmss file (magic 4f434e45).
VMware vSphere 8.x
This issue occurs because the virtual machine is encrypted.
When a snaphot with memory is taken, vmem file is also encrypted.
Decrypt an Encrypted Virtual Machine or Virtual Disk
https://knowledge.broadcom.com/external/article?articleNumber=181598
Encrypt an Existing Virtual Machine or Virtual Disk