Issue/Introduction:
A SWG VA or Mach5 VA which is restarting unexpectedly.
There are "stolen time" messages in Event Log such as:
2014-06-02 04:50:00+01:00BST "VM: instantaneous stolen time of 60092 ms exceeds threshold 15000 ms" EABC 8D0002:1 main.cpp:251
Restart signatures such as the following are seen in Sysinfo:
Hardware exception code: 0x0 or 0x2
Software exception code: 0x8D0002
Page fault linear address: 0x0
Process group: "PG_CAG"
Process: "CAG_Maintenance" in "" at .text+0x0
Cause:
This restart is expected behavior.
The issue is that the VM had either been suspended then unsuspended, or a VM snapshot restored.
The internal VM time counter detected that there was a discrepancy between the HOST time and the VM time.
This is known as "stolen time".
If this is detected, the SG will reboot in order to get the time back in sync between SG VM and the VMware HOST.
Resolution:
The following actions are recommended to prevent this issue from occurring:
- Do not enable troubleshooting snapshots on the ProxySG VM.
- Do not suspend the VM.
- Configure the VM instance with a single, full CPU processor.
- Do not run Snapshot Consolidation which is known to hang the VM for long periods.
- Run each virtual storage disk on a separate physical spindle. This applies to the large virtual disks, not the small disk.
- Use thick provisioning for disks.