Service Insertion Deployment Cleanup Fails with “An Existing Transaction is Still in Progress” Error in NSX

Products

• VMware vDefend Firewall

Issue/Introduction

In some environments, a SI deployment may become stuck in a failed state. The associated Service VMs (SVMs/Agents) remain in a powered-off state across ESXi hosts, and vSphere ESX Agent Manager (EAM) marks them as “has issues.”
When attempting to delete the affected deployment from NSX Manager, the operation fails with an error indicating:

Error connecting EAM on compute manager <ID>.  
Error An existing transaction is still in progress. TX[####]

From nsxapi logs on the manager

2025-08-11T20:01:07.609Z  INFO http-nio-127.0.0.1-7440-exec-90 NsxBaseRestController 74295 SYSTEM [nsx@6876 audit="true" comp="nsx-manager" level="INFO" subcomp="manager"] UserName:'***' ModuleName:'cm-inventory' Operation:'DELETE@/api/v1/serviceinsertion/services/********-****-****-****-************/service-deployments/********-****-****-****-************' Operation status: 'failure' Error: Error connecting EAM on compute manager ********-****-****-****-************. Error An existing transaction is still in progress. TX[****]

This prevents administrators from cleaning up the failed Service Insertion deployment and leaves stale agents on the cluster.

Environment

• VMware NSX-T 3.x / 4.x

• vDefend Firewall

Cause

The issue occurs when EAM still maintains references to the Service Insertion agency and agent VMs after they are powered off, leaving the agency in a yellow state. Because EAM is in the middle of an unresolved transaction, NSX Manager cannot complete the cleanup of the deployment, resulting in the transaction error.

Resolution

To remediate, the failed Service Insertion agency and deployment must be cleaned up in the correct order:

1. From vSphere Client (EAM):

   • Navigate to:
     Administration > Solutions > vCenter Server Extensions > vSphere ESX Agent Manager > Configure

   • Locate the agency

   • Attempt “Resolve All Issues.”

   • If unsuccessful, choose “Delete Agency” (this action removes all associated Agent VMs).

2. From NSX Manager:

   • Once the agency is deleted from vSphere, delete the corresponding service deployment from:
     System > Service Deployments

3. If cleanup still fails:

   • Verify Compute Manager connectivity in NSX Manager (System > Fabric > Compute Managers).

   • Restart the EAM service on vCenter
     
     service-control --stop vmware-eam
service-control --start vmware-eam

   • Use NSX API to list and delete instance runtimes:

     GET  /api/v1/serviceinsertion/services
GET  /api/v1/serviceinsertion/services/<service-UUID>
GET  /api/v1/serviceinsertion/services/<service-UUID>/service-instances
GET  /api/v1/serviceinsertion/services/<service-UUID>/service-instances/<instance-UUID>/instance-runtimes
POST /api/v1/serviceinsertion/services/<service-UUID>/service-instances/<instance-UUID>/instance-runtimes?action=delete