Windows Guest virtual machines may encounter BSOD in NSX environment when optional NSX File Introspection component is installed

Products

• VMware NSX

Issue/Introduction

Symptoms:
Windows Guest Operating System in an NSX environment may encounter BSOD especially when the system is engaged in active file IO over SMB

1. During removal of file introspection component as part of uninstallation of VMware Tools 12.0.5.
2. During upgrade from VMware Tools 12.0.5 complete installation on the system to any newer version of tools including express or hot patches on top of VMware Tools 12.0.5.
3. In very few tested and observed cases, during complete installation of VMware Tools 12.0.5, after vsepflt driver connects to NSX host component for the first time
4. During vMotion of a guest VM typically with large memory resources.
5. In all above scenarios, Guest VM may encounter BSOD with faulting driver as vsepflt and bug error code PAGE_FAULT_IN_NONPAGED_AREA (50)

Environment

VMware Tools 12.0

Cause

VMware Tools NSX File Introspection driver connects to NSX host component when the host is prepared for NSX use. Barring unrecoverable error conditions, Guest driver connection with NSX Host component is expected to be long lived and is expected to break only if:

1. Guest operating system restarts.
2. Guest VM is moved out of the host.
3. VMware Tools 12.0.5 uninstall or upgrade

If such disconnect with NSX host component happens while the guest is engaged in active file IO, especially over SMB, then there is a likely multi-application file sharing scenario where an application happens to grant an opportunistic lock on the shared file using a handle that it may have obtained from the file system during the NSX host disconnect window to other requesting application. Such a handle when used to grant an opportunistic lock won't yield a backing context in the driver leading to driver accidentally freeing a NULL pointer causing system crash.

Resolution

This issue is resolved in VMware Tools 12.0.6

Workaround:

1. If you don't need to use any NSX Guest Introspection powered features (NSX Identity Firewall, NSX Intelligence, NSX Distributed Malware Prevention or NSX Endpoint Protection powered by third party agentless AV solutions) and need to upgrade to VMware Tools 12.0.5, you  are advised to USE DEFAULT or CUSTOM installation with NSX Guest Introspection drivers (NSX File Introspection and NSX Network Introspection) not selected during installation.
2. If you use any of the above NSX Guest Introspection powered features and are looking to upgrade VMware Tools then you are advised to upgrade to VMware Tools 12.0.6 or newer version.
3. In case you are using NSX File Introspection driver in an NSX environment and has already installed VMware Tools 12.0.5, customer is recommended to use steps mentioned in recovery section below. These steps can be used for removal of VMware Tools 12.0.5 or during upgrade to any newer version of VMware Tools.

Recovery:

In order to cleanly uninstall VMware Tools 12.0.5 or upgrade to a newer version of VMware Tools following things need to be done.

• Stop NSX File Introspection driver so that it doesn't lead to a BSOD during uninstall/upgrade process.
• Uninstall or upgrade VMware Tools as needed

Prevent the vsepflt driver from loading on subsequent reboots:

Note: Before making any registry modifications, ensure that you have a current and valid backup of the registry and the virtual machine. For more information on backing up and restoring the registry, see the Microsoft article 

Disclaimer: VMware is not responsible for the reliability of any data, opinions, advice, or statements made on third-party websites. Inclusion of such links does not imply that VMware endorses, recommends, or accepts any responsibility for the content of such sites.

1. Run the below command from Administrator command prompt:

" sc config vsepflt start=DISABLED"

This should result in command returning < [SC] ChangeServiceConfig SUCCESS

2. Reboot the guest virtual machine.

Uninstall or Upgrade VMware Tools
Run VMware Tools installer to upgrade to newer version of the VMware Tools.