Symptoms:
- vCenter upgrade to 7.0 fails during vmidentity firstboot with the error:
Encountered an internal error.Traceback (most recent call last): File \"/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py\", line 1449, in main vmidentityFB.boot() File \"/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py\", line 343, in boot self.registerTokenServiceWithLookupService() File \"/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py\", line 567, in registerTokenServiceWithLookupService raise e File \"/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py\", line 563, in registerTokenServiceWithLookupService dynVars=dynVars) File \"/usr/lib/vmware-cm/bin/cloudvmcisreg.py\", line 714, in cloudvm_sso_cm_register serviceId = do_lsauthz_operation(cisreg_opts_dict) File \"/usr/lib/vmware/site-packages/cis/cisreglib.py\", line 1118, in do_lsauthz_operation ls_obj.register_service(svc_id, svc_create_spec) File \"/usr/lib/vmware/site-packages/cis/cisreglib.py\", line 348, in add_securityctx_to_requests return req_method(self, *args, **kargs) File \"/usr/lib/vmware/site-packages/cis/cisreglib.py\", line 360, in register_service svc_create_spec) File \"/usr/lib/vmware/site-packages/pyVmomi/VmomiSupport.py\", line 595, in <lambda> self.f(*(self.args + (obj,) + args), **kwargs) File \"/usr/lib/vmware/site-packages/pyVmomi/VmomiSupport.py\", line 385, in _InvokeMethod return self._stub.InvokeMethod(self, info, args) File \"/usr/lib/vmware/site-packages/pyVmomi/SoapAdapter.py\", line 1570, in InvokeMethod raise obj # pylint: disable-msg=E0702pyVmomi.VmomiSupport.vmodl.fault.SystemError: (vmodl.fault.SystemError) { dynamicType = <unset>, dynamicProperty = (vmodl.DynamicProperty) [], msg = 'LookupFaultServiceFault', faultCause = <unset>, faultMessage = (vmodl.LocalizableMessage) [], reason = 'Invalid fault'}
- In the /var/log/firstboot/vmidentity-firstboot.py_11661_stdout.log you may find entries similar to:
2022-11-05T02:42:45.849Z VMware Identity Service bootstrap failed.
2022-11-05T02:42:45.850Z Exception: Traceback (most recent call last):
File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 1449, in main
raise obj # pylint: disable-msg=E0702
pyVmomi.VmomiSupport.vmodl.fault.SystemError: (vmodl.fault.SystemError) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
msg = 'LookupFaultServiceFault',
faultCause = <unset>,
faultMessage = (vmodl.LocalizableMessage) [],
reason = 'Invalid fault'
- In the var/log/vmware/lookupsvc/lookupserver-default.log you may find entries similar to:
[2022-11-05T02:42:45.839Z pool-2-thread-5 ERROR com.vmware.vim.lookup.vlsi.util.VmodlEnhancer] com.vmware.vim.vmomi.core.exception.CertificateValidationException: SSL handshake from 0.0.0.0/0.0.0.0:39030 to vCenterfqdn/127.0.0.1:443 failed in 467 ms
com.vmware.vim.sso.admin.exception.CertificateValidationException: com.vmware.vim.vmomi.core.exception.CertificateValidationException: SSL handshake from 0.0.0.0/0.0.0.0:39030 to vCenterfqdn/127.0.0.1:443 failed in 467 msCaused by: java.security.cert.CertificateException: malformed PEM data encountered
at org.bouncycastle.jcajce.provider.CertificateFactory.readCertificate(Unknown Source) ~[bc-fips-1.0.2.1.jar:1.0.2.1]
Caused by: java.io.IOException: malformed PEM data encountered
at org.bouncycastle.jcajce.provider.PEMUtil.readPEMObject(Unknown Source) ~[bc-fips-1.0.2.1.jar:1.0.2.1]Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.