Any operation on ESXi host fails with an error "License key has expired. License Expired, Check system date or install a new license"

Products

• VMware vCenter Server 7.0
• VMware vCenter Server 8.0
• VMware vSphere ESXi 7.0
• VMware vSphere ESXi 8.0

Issue/Introduction

• Licenses applied to the ESXi hosts are valid.
• While powering ON a VM or performing a vMotion, might receive the below error:
  "License key has expired. License Expired, Check system date or install a new license"
• In vCenter /var/log/vmware/vpxd

  YYYY-MM-DDTHH:MM:SS.844Z error vpxd[76891] [Originator@6876 sub=vpxLro opID=lx2ovegl-33747159-auto-k3bh4-h5:72504236-4f-01-01-01-01] [VpxLRO] Unexpected Exception: N3Vim5Fault14LicenseExpired9ExceptionE(Fault cause: vim.fault.LicenseExpired
  --> )
  --> [context]zKq7AVECAQAAAIUcWQEidnB4ZAAAxbVTbGlidm1hY29yZS5zbwAAUglDAIwxRACaSEuBD2I3AWxpYnZpbS10eXBlcy5zbwCBgmI3AYFkZDcBgWhnNwGCoGMOAXZweGQAgpXPYQKCrdNhAoI49yACgn+wGwKC670kAoJI3SQCgkr9JAKCfQhiAoLPGGICgvkvYQKCUothAoLJH9QBgk0M1QGChw3VAYJTDtUBgqaLVgKCfQhiAoLPGGICgvkvYQKClAJiAgDmyzcA+SQ4AJPAUQOujgBsaWJwdGhyZWFkLnNvLjAABC/eD2xpYmMuc28uNgA=[/context]
  YYYY-MM-DDTHH:MM:SS.851Z info vpxd[76891] [Originator@6876 sub=vpxLro opID=lx2ovegl-33747159-auto-k3bh4-h5:72504236-4f-01-01-01-01] [VpxLRO] -- FINISH lro--1312975396
  YYYY-MM-DDTHH:MM:SS.851Z error vpxd[76891] [Originator@6876 sub=Default opID=lx2ovegl-33747159-auto-k3bh4-h5:72504236-4f-01-01-01-01] [VpxLRO] -- ERROR lro--1312975396 --  --  -- VmprovWorkflow: :vim.fault.LicenseExpired
  --> Result:
  --> (vim.fault.LicenseExpired) {
  -->    faultCause = (vmodl.MethodFault) null,
  -->    faultMessage = (vmodl.LocalizableMessage) [
  -->       (vmodl.LocalizableMessage) {
  -->          key = "com.vmware.vim.license.error.LicenseExpired",
  -->          arg = (vmodl.KeyAnyValue) [
  -->             (vmodl.KeyAnyValue) {
  -->                key = "info",
  -->                value = "expired"
  -->             }
  -->          ],
  -->          message = "License Expired, Check system date or install a new license."
  -->       }
  -->    ],
  -->    licenseKey = "#####-#####-#####-#####-#####"
  -->    msg = "License key has expired."
  --> }
  --> Args:
  -->
  YYYY-MM-DDTHH:MM:SS.851Z warning vpxd[76891] [Originator@6876 sub=drsExec opID=lx2ovegl-33747159-auto-k3bh4-h5:72504236-4f-01-01-01] Caught MethodFault Fault cause: vim.fault.LicenseExpired
  --> . Failing powerOn
  

• In ESXi host /var/run/log/hostd.log

  YYYY-MM-DDTHH:MM:SS.674Z In(166) Hostd[2098847]: [Originator@6876 sub=Vimsvc.TaskManager opID=91777944 sid=52276e61 user=vpxuser] Task Completed : haTask--vim.dvs.HostDistributedVirtualSwitchManager.retrieveDVPortgroupConfigSpec-1963280953 Status success
  YYYY-MM-DDTHH:MM:SS.673Z In(166) Hostd[2098830]: [Originator@6876 sub=AdapterServer opID=CdrsLoadBalancer-ca7dc64-16957bc7-01-01-36-7930 sid=527f2db5 user=vpxuser:<no user>] AdapterServer caught exception; <<527f2db5-e0bd-9a46-ab80-62c15c871994, <TCP '127.0.0.1 : 8307'>, <TCP '127.0.0.1 : 36101'>>, ha-vmotionmgr, vim.host.VMotionManager.initiateDestination, <vim.version.v8_0_2_0, internal, 8.0.2.0>, [N11HostdCommon18VmomiAdapterServer19ActivationResponderE:0x000000acc7e9e478]>, N3Vim5Fault14LicenseExpired9ExceptionE(Fault cause: vim.fault.LicenseExpired
  YYYY-MM-DDTHH:MM:SS.680Z Db(167) Hostd[2098830]: [Originator@6876 sub=Solo.Vmomi opID=CdrsLoadBalancer-ca7dc64-16957bc7-01-01-36-7930 sid=527f2db5 user=vpxuser:<no user>] Arg migrationId:
  YYYY-MM-DDTHH:MM:SS.680Z Db(167) Hostd[2098788]: --> 7313599505281300864
  YYYY-MM-DDTHH:MM:SS.680Z Db(167) Hostd[2098788]: --> "/vmfs/volumes/######-#####-####-########/VM_name/VM_name.vmx"
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098830]: [Originator@6876 sub=Solo.Vmomi opID=CdrsLoadBalancer-ca7dc64-16957bc7-01-01-36-7930 sid=527f2db5 user=vpxuser:<no user>] Throw vim.fault.LicenseExpired
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098830]: [Originator@6876 sub=Solo.Vmomi opID=CdrsLoadBalancer-ca7dc64-16957bc7-01-01-36-7930 sid=527f2db5 user=vpxuser:<no user>] Result:
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: --> (vim.fault.LicenseExpired) {
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->    faultMessage = (vmodl.LocalizableMessage) [
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->       (vmodl.LocalizableMessage) {
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->          key = "com.vmware.vim.license.error.LicenseExpired",
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->          arg = (vmodl.KeyAnyValue) [
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->             (vmodl.KeyAnyValue) {
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->                key = "info",
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->                value = "expired"
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->             }
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->          ],
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->       }
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->    ],
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->    licenseKey = "#####-#####-#####-#####-#####",
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: -->    msg = ""
  YYYY-MM-DDTHH:MM:SS.680Z In(166) Hostd[2098788]: --> }
  YYYY-MM-DDTHH:MM:SS.717Z Er(163) Hostd[2098819]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/######-#####-####-########/VM_name/VM_name.vmx opID=CdrsLoadBalancer-ca7dc64-2ff15166-01-01-ff-794f sid=527f2db5 user=vpxuser:<no user>] CheckLicenses: ha-license-manager failed power check

Cause

hostd service does not have the correct information regarding the License assigned to the ESXi host.

Resolution

Contact VMware Support for additional assistance. 

Additional Information

Rebooting the affected ESXi host or restarting the hostd service on ESXi host may fix the issue. 

/etc/init.d/hostd restart