Using RealVNC Viewer to run a VM on multiple 4K displays, process fails with out of memory error

Products

• VMware vSphere ESXi

Issue/Introduction

To resolve the out of memory errors.

Symptoms:

• When trying to use RealVNC to run a VM on three 4K displays, process fails. This can happen when using 3840x2160 resolution
• In vmware.log you see entries similar to below 

         2018-09-12T17:34:27.202Z| mks| I125: SOCKET 25 (178) Creating VNC remote connection.
    2018-09-12T17:34:27.222Z| mks| I125: SOCKET 24 (177) recv detected client closed connection
    2018-09-12T17:34:27.222Z| mks| I125: SOCKET 24 (177) VNC Remote Disconnect: socket closed.
    2018-09-12T17:34:27.397Z| mks| W115: VNCENCODE 25 pump failed, error 195: Out of memory
    2018-09-12T17:34:27.397Z| mks| W115: VNC Error (195): Out of memory
    2018-09-12T17:34:27.397Z| mks| I125: SOCKET 25 (178) Deferring VNC error.
    2018-09-12T17:34:29.315Z| mks| I125: SOCKET 26 (174) Creating VNC remote connection.
    2018-09-12T17:34:29.572Z| mks| W115: VNCENCODE 26 pump failed, error 195: Out of memory
    2018-09-12T17:34:29.572Z| mks| W115: VNC Error (195): Out of memory
    2018-09-12T17:34:29.572Z| mks| I125: SOCKET 26 (174) Deferring VNC error.

• This does not occur when using 1080p displays.

Note:The preceding log excerpts are only examples.Date,time and environmental variables may vary depending on your environment.

Environment

VMware vSphere ESXi 6.0
VMware vSphere ESXi 6.5
VMware vSphere ESXi 6.7

Cause

VNC server runs out of memory.

Resolution

This issue is resolved in VMware ESXi 6.7 U2.

Workaround:
Use any of the below workarounds
1. Connect to the VM having 3x UHD displays but a single color desktop.
2. Force a VNC session to use 256 colors instead of Full color mode.
3. Implemement VMware Horizon View to support upto four 4K dsiplays on HWv13 - Refer to VMware Horizon Guide - Supported Multiple Monitor Configurations

Additional Information

For more information refer KB : Adding Video Resolution Modes to Windows Guest OS

Impact/Risks:
None