"/storage/core" gets filled with core.pschealthd.xxx files after upgrading to vCenter Server Appliance 6.5 Update 2

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• /storage/core gets filled with core.pschealthd.xxx files

• /storage/core gets filled by 100% in 24 hours.

-rw-rw-r-- 1 support support 122M Jul 2 15:06 core.pschealthd.9414
-rw-rw-r-- 1 support support 122M Jul 2 15:06 core.pschealthd.9541

In the file /var/log/vmware/vmon/vmon-syslog.log, you see entry similar to:

18-06-28T20:29:39.146454+00:00 notice vmon  Constructed command: /usr/lib/vmware-pschealth/sbin/pschealthd -s
18-06-28T20:29:39.146664+00:00 notice vmon  Constructed command: /usr/bin/python /usr/lib/vmware-vmon/vmonEventPublisher.py --eventdata pschealth,UNHEALTHY,HEALTHY,1
18-06-28T20:29:39.751701+00:00 notice vmon  Successfully published health status change.
18-06-28T20:29:40.007428+00:00 notice vmon  Constructed command: /usr/bin/python /usr/lib/vmware-pschealth/vmon/pschealth-vmon-apihealth.py
18-06-28T20:29:40.206376+00:00 notice vmon  Re-check service pschealth health since it is still initializing.
18-06-28T20:29:41.208829+00:00 notice vmon  Constructed command: /usr/bin/python /usr/lib/vmware-pschealth/vmon/pschealth-vmon-apihealth.py
18-06-28T20:29:41.294371+00:00 notice vmon  Service pschealth STARTED successfully.
18-06-28T20:29:41.294742+00:00 notice vmon  Constructed command: /usr/bin/python /usr/lib/vmware-vmon/vmonEventPublisher.py --eventdata pschealth,HEALTHY,UNHEALTHY,1
18-06-28T20:29:42.145314+00:00 notice vmon  Successfully published health status change.
18-06-28T20:30:39.676767+00:00 warning vmon  Service pschealth exited. Exit code 1
18-06-28T20:30:39.677028+00:00 warning vmon  Service pschealth exited unexpectedly. Crash count 6736. Taking configured recovery action.

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vCenter Server Appliance 6.5.x

Cause

This happens because pschealth deamon is crashing and restarting.

Resolution

This is a known issue under investigation.



Workaround:

• In /storage/log/vmware/messages, if you see entries similar to:

2018-06-28T20:19:33.184964+00:00 vcenter-vxlan systemd[1]: Time has been changed
2018-06-28T20:18:40.563847+00:00 vcenter-vxlan systemd[1]: Time has been changed

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

• Fix the time sync issues
• Clear old core files from /storage/core
• Observe the environment for 24 Hours and report the status