After VCF restart, the EAM reports agencies created by NSX for Malware Prevention Service in Warning status and powering on workload on these clusters fails with InsufficientAgentVmsDeployed

Products

• VMware vDefend Firewall with Advanced Threat Prevention

Issue/Introduction

In an environment with vDefend/NSX Malware Prevention Service deployed, SSP and VCF deployments have been shutdown.

During the bring-up, the ESXi(s) are restarted, but on the clusters where the MPS service has been deployed, workload VMs other than the Malware Prevention Service VM (MPS SVM) cannot be powered-on. 

vSphere Client return the following error: "All required agent virtual machines are not currently deployed on host"  while trying to power-on the VMs. Manually powering-on the SVMs doesn't work. 

The EAM reports the Agencies created for the Malware Prevention Service in Warning state.

Environment

NSX, SSP

Cause

The Malware Prevention Service VM is an NSX driven EAM agent deployment. As the SVMs are EAM managed resources, manually powering-on the SVMs does not work. 

The EAM automatically powers-on the VMs, but waits for certain things to be processed by the NSX manager before marking the agents green.

Hence, the NSX needs to be UP and running before the agency can be marked as green.

Resolution

After a VCF restart, during the bringup, we need to ensure that the NSX is also powered-on and running. Then the MPS SVMs will automatically be marked as available and the workload VMs should power-on without issues. 

NSX node(s) and MPS SVMs Deployed on the same cluster : In scenarios where the NSX is also present on the same cluster where the MPS SVM has been deployed, the NSX VMs will also face the "All required agent virtual machines are not currently deployed on host" error. In such cases, the admin needs to identify the ESX host(s) where the NSX Manager node(s) is/are located and power them on using the ESXi Host Client (UI), PowerCLI or vimsvc on the ESX Shell.

After NSX is up and running, if we still face any issues, we can check the deployments in the NSX Manager (Security-> IDS/IPS & Malware Prevention->Settings) and if the deployments are still shown red, or the SVMs are not powered on automatically, we can click "Resolve" for the deployments in question.