Package 'healthcheck_windows' fails during Windows TAS upgrade

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

Symptoms:
During any Windows TAS upgrade with only one cell, Apply Changes fails due to the below error message:

Error: Action Failed get_task: Task 12f5968a-2068-4e3b-4739-d8eda9c6f325 result: Applying: Keeping only needed packages: Uninstalling package bundle: remove /var/vcap/data/packages/healthcheck_windows/d350b9d92ba5b3c2c9075d0b78fd5d695904ac92\internal\healthcheck.exe: Access is denied.

Environment

Cause

The failing task is trying to remove the health check, but the health check process is still running. In Windows you cannot delete a file that is currently open. Therefore, Windows is unable to delete the open file. 

• When you only have one cell, any apps running are not evacuated to another.
• As a result, the rep hits the evacuation timeout and gives up.
• Evacuation cleanup is supposed to signal to the container processes to stop the container, but on Windows (in the garden implementation) this is a "noop".
• It is likely the evacuation LRP processor does not have time to delete the container before the rep exits, so the container is left stranded with handles still open to processes.

Resolution

To workaround this issue, you need to scale up to multiple cells so evacuation is able to move the running app and delete the containers on the cell that is shutting down.