Running antivirus on Pivotal Cloud Foundry Windows Diego Cell

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

Symptoms:

Purpose

The On-Access scanning feature of antivirus programs is incompatible with the BOSH Agent. If you must run an AV program, we strongly recommend disabling On-Access scanning.

When provisioning a VM, the BOSH agent will download various “tgz” packages. The BOSH agent extracts each package into a temporary directory and then renames this directory as the proper location for the package.

The agent does this so that any errors encountered during the download or archive expansion will not lead to the package directory being polluted with partial or invalid packages.

“On-Access scanning” (for example, by McAfee antivirus) interferes with this behavior by preventing the Agent from renaming the package until the entire directory has been scanned by the antivirus (AV) program. This leads to intermittent and hard-to-debug “Access is denied” errors.

We do not believe changing the Agent’s behavior, for example retrying the rename or extracting the package into its final destination, would resolve these issues with “On-Access” scanning. Below are summaries of the two workarounds considered:

Retry strategy

The time required for an AV program to scan a directory is non-deterministic and cannot be queried. Additionally, the error returned is too general (“Access is denied”) to be a reliable indicator that an AV program will be interfering with the rename.

Eliminate rename

Often, the next step after extracting a package is to compile it. Removing the rename step from the Agent would only move the error from the Agent itself to the compiler (golang, in the case of the most CF components). Additionally, packages that require compiling are often those with the most files and thus most likely to require long scans by the installed AV program.

Environment

Resolution

Procedure

While AV software is not supported out of the box with Cloud Foundry Windows cells, it is possible to install and configure AV software if it’s required by your corporate security standards.

It is important when creating a new BOSH add-on to install your AV software that the installation can be performed silently without user's intervention. It is also important to configure the AV agent so that any On-Access scanners exclude some of the underlying Cloud Foundry runtime directories, otherwise intermittent deployment failures may occur. The following directories should be excluded from on-demand scanning:

    c:\bosh
    c:\var\vcap
    c:\containerizer
    c:\provision
    c:\proc
    e:\groot
    e:\jobs
    e:\rep
    e:\packages

Please note that, the e:\ is created by bosh for windows on the each windows_diego_cell.
Additionally, mark the below files at low risk in the antivirus configurations:
    groot.exe
    winc-network.exe
    gdn.exe
    rep.exe
    winc.exe
    winit.exe
 

Failure to exclude these directories may lead to ephemeral permission issues with files in these folders and cause cell and app deployments to fail.