Installing Antivirus-mirror fails with "Database update failed"

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

When Installing the Anti-Virus Tile and "Mirror for Automatic Database Updates" is set to "Deployed mirror from the Anti-Virus for VMware Tanzu Mirror Tile" in Anti-Virus Configuration, the Anti-Virus Mirror Tile needs to be installed first.

If you run Apply Changes on the Anti-Virus Tile without the Anti-Virus Mirror Tile, you'll get the following error. 
 

Task 197 | 17:03:14 | L executing pre-start: nats/b07fe901-4e39-4c2c-9f1f-b9bb0d780863 (0) (canary) (00:01:00)
                   L Error: Action Failed get_task: Task 260ac77c-41ff-4854-70a2-521d12f8f243 result: 1 of 5 pre-start scripts failed. Failed Jobs: antivirus. Successful Jobs: loggregator_agent, syslog_forwarder, bpm, bosh-dns.
Task 197 | 17:03:30 | Error: Action Failed get_task: Task 260ac77c-41ff-4854-70a2-521d12f8f243 result: 1 of 5 pre-start scripts failed. Failed Jobs: antivirus. Successful Jobs: loggregator_agent, syslog_forwarder, bpm, bosh-dns.

Then if you got into the failing VM and check /var/vcap/sys/log/antivirus/pre-start.stderr.log or pre-start.stdout.log

both will show

Database update failed; freshclam returned exit status 1

This is happening because it's trying to connect to Anti-Virus Mirror database, which is not there (yet).

Environment

Product Version: 2.13

Resolution

Installing Anti-Virus Mirror Tile without removing Anti-Virus Tile will not help since Anti-Virus Mirror also runs the antivirus job, that will try to start before the antivirus-mirror job starts the db, and you will see same error in Anti-Virus Mirror Tile during installation.

 In order to fix this situation you need to

1. Run bosh start on the VM that failed while Applying Changes. (In our example is nats) E.g 

bosh -d cf-2cdc505875732fa3ace0 start nats/b07fe901-4e39-4c2c-9f1f-b9bb0d780863

It's important that you don't run the "--no-converge" flag, since we want the VM to go back to it's previous state, prior to anti-virus job running.

Note: If you are not in a rush to recover your system, you can skip this step since it will also be recovered in next step, when deleting the Anti-Virus Tile
     
        2. Delete the Anti-Virus Tile and run Apply Changes selecting the Tile that failed when installing Anti-Virus Tile.This would usually be TAS, but could be also a different one, depending on your set-up. This will bring the failing deployment to a running state if you skipped first previous step.
        3. Install the Anti-Virus Mirror Tile alone, without the Anti-Virus one.
        4. Install the Anti-Virus Tile running  Apply Changes in all Tiles.