Extracting Host Profile fails with error "1-01-01T00:00:00Z"
Article ID: 320696
Updated On: 02-04-2025
Products
Issue/Introduction
• Extracting a Host Profile in the vSphere UI, fails with an error window containing:
Operation failed!
A general system error occurred:
Error: 1-01-01T00:00:00Z
• vpxa.log on the host shows the following error
YYYY-MM-DDTHH:MM:SS.845Z info vpxa[183597] [Originator@6876 sub=Default opID=lgbahbp8-25132-auto-je5-h5:70005383-58-dc] [VpxLRO] -- ERROR lro-53805 -- HostdHostProfileManager -- vim.profile.host.profileEngine.HostProfileManager.retrieveProfile: vmodl.fault.SystemError:
--> Result:
--> (vmodl.fault.SystemError) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = (vmodl.LocalizableMessage) [
--> (vmodl.LocalizableMessage) {
--> key = "com.vmware.vim.profile.engine.UnexpectedError",
--> arg = (vmodl.KeyAnyValue) [
--> (vmodl.KeyAnyValue) {
--> key = "error",
--> value = "1-01-01T00:00:00Z"
--> },
--> (vmodl.KeyAnyValue) {
--> key = "context",
--> value = "Exception while retrieving profile"
--> }
--> ],
--> message = "Error: 1-01-01T00:00:00Z."
--> }
Environment
Cause
This is caused by a vCenter extension registration containing an invalid heartbeat date/time. The error is most commonly due to 3rd party extensions and has been observed with, but not limited to:
- VxRail HTML5 Client Plugin
- Pure Storage vSphere Plugin for Managing FlashArray
Resolution
Engage the vendor of the specific extension that is causing the issue for assistance which may include unregistering and re-registering the extension after the date and time are corrected on their appliance.
To find the offending extension review the vpxa.log on the host the Host Profile fails to extract from.
• SSH to the host
• Run the following command to find the label of the extension with the invalid date
grep -E "label|lastHeartbeat" /var/log/vpxa.log |grep -B5 01-01T
Example output
[root@localhost:/var/log] grep -E "label|lastHeartbeat" /var/log/vpxa.log |grep -B5 01-01T --> label = "VMware vSphere Update Manager Extension", --> label = "VMware vSphere Update Manager", --> label = "VMware vSphere Update Manager", --> label = "VMware vSphere Update Manager", --> label = "VMware vSphere Update Manager Client 7.0", --> lastHeartbeatTime = "0001-01-01T00:00:00Z",
• From the output we can see the label of the extension causing the issue. In the above example it's the 'VMware vSphere Update Manager Extension'.
Workaround:
To workaround the issue, the heartbeat value for the offending extension must manually be modified in the vCenter DB.
***Create a backup or take all necessary snapshots of the VCSA before proceeding further***
1. Log into the VCSA via SSH
2. Stop the vpxd service
service-control --stop vpxd
3. Run the following to get a list of the extensions and their heartbeat values.
/opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres -c "select ext_id,heartbeat,desc_label from vpx_ext;"
Example output
ext_id | heartbeat | desc_label ----------------------------------+-------------------------+----------------------------------------- com.vmware.imagebuilder | 2023-04-10 20:31:13.926 | ImageBuilder Service com.vmware.ovf | 2022-07-07 23:51:51.747 | Open Virtualization Format Service com.vmware.vim.eam | 2022-09-28 22:26:15.671 | vSphere ESX Agent Manager com.vmware.vsphere.client.h5vsan | 2022-07-07 23:49:00.052 | VMware vSAN UI extension com.vmware.vmcam | 2021-10-28 22:50:18.163 | description com.vmware.vrops.install | 2022-07-07 23:49:00.055 | VMware vRops extension com.vmware.vcenter.wcp | 2021-10-28 22:52:03.378 | WCP Service com.vmware.vcIntegrity | 0001-01-01 00:00:00 | VMware vSphere Update Manager Extension com.vmware.registry | 2023-04-10 20:35:39.766 | Container Image Registry com.vmware.cl | 2023-04-10 20:35:40.057 | vSphere Content Library Service com.vmware.rbd | 2023-04-23 23:01:15.212 | RuleEngine com.vmware.vsphere.client | 2022-07-07 23:51:51.747 | com.vmware.vsphere.client.label com.vmware.vsan.health | 2023-04-10 20:32:00.806 | VMware vSAN management extension com.vmware.vcenter.iso | 2023-04-10 20:35:41.252 | vSphere ISO Service com.vmware.vcenter.vmtx | 2023-04-10 20:35:41.722 | vSphere Content Library com.vmware.lcm.client | 2023-04-10 20:31:09.255 | VMware vCenter Server Lifecycle Manager (16 rows)
4. Confirm the heartbeat value next to the extension found from the vpxa.log is invalid.
5. Run the following to replace the invalid value with the current date/time. Replacing the ext_id with the one found in the above output.
/opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres -c "update vpx_ext set heartbeat = now() where ext_id = 'ext_id';"
Using the above example it would look like
Example:
/opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres -c "update vpx_ext set heartbeat = now() where ext_id = 'com.vmware.vcIntegrity';"
*NOTE: If more than one extension has an invalid date, run the above for each*
6. Re-run the command from Step 3 to confirm the heartbeat value has been updated
ext_id | heartbeat | desc_label ----------------------------------+----------------------------+----------------------------------------- com.vmware.imagebuilder | 2023-04-10 20:31:13.926 | ImageBuilder Service com.vmware.ovf | 2022-07-07 23:51:51.747 | Open Virtualization Format Service com.vmware.vim.eam | 2022-09-28 22:26:15.671 | vSphere ESX Agent Manager com.vmware.vsphere.client.h5vsan | 2022-07-07 23:49:00.052 | VMware vSAN UI extension com.vmware.vmcam | 2021-10-28 22:50:18.163 | description com.vmware.vrops.install | 2022-07-07 23:49:00.055 | VMware vRops extension com.vmware.vcenter.wcp | 2021-10-28 22:52:03.378 | WCP Service com.vmware.registry | 2023-04-10 20:35:39.766 | Container Image Registry com.vmware.cl | 2023-04-10 20:35:40.057 | vSphere Content Library Service com.vmware.vcIntegrity | 2023-04-24 15:26:43.073774 | VMware vSphere Update Manager Extension com.vmware.vsphere.client | 2022-07-07 23:51:51.747 | com.vmware.vsphere.client.label com.vmware.rbd | 2023-04-24 15:30:56.026 | RuleEngine com.vmware.vcenter.iso | 2023-04-10 20:35:41.252 | vSphere ISO Service com.vmware.vsan.health | 2023-04-10 20:32:00.806 | VMware vSAN management extension com.vmware.vcenter.vmtx | 2023-04-10 20:35:41.722 | vSphere Content Library com.vmware.lcm.client | 2023-04-10 20:31:09.255 | VMware vCenter Server Lifecycle Manager (16 rows)
7. Start vpxd service
service-control --start vpxd
8. Log back into vCenter and attempt to extract a Host Profile.
Feedback
