Reoccurring log event "Error in handlePolicyChange to process aggregated security toggle for /infra/settings/firewall/idfw/cluster/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:domain-c1234"

Products

VMware vDefend Firewall VMware vDefend Firewall with Advanced Threat Prevention

Issue/Introduction

The following error occurs every 5 minutes in NSX Manager the log file /var/log/syslog

2025-01-01T15:00:00.075Z  WARN providerTaskExecutor-1-66 IdfwIdsToggleProviderNsxT 86292 POLICY [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manage
r"] Error in handlePolicyChange to process aggregated security toggle for /infra/settings/firewall/idfw/cluster/########-####-####-####-############:domain-c1234
com.vmware.nsx.management.common.exceptions.ObjectNotFoundException: null
        at com.vmware.nsx.management.policy.providers.security.provider.IdfwIdsToggleProviderNsxT.validateClusterExtId(IdfwIdsToggleProviderNsxT.java:478) ~
[?:?]
        at com.vmware.nsx.management.policy.providers.security.provider.IdfwIdsToggleProviderNsxT.processConfig(IdfwIdsToggleProviderNsxT.java:362) ~[?:?]
        at com.vmware.nsx.management.policy.providers.security.provider.IdfwIdsToggleProviderNsxT.handleIdfwConfig(IdfwIdsToggleProviderNsxT.java:223) ~[?:?
]
        at com.vmware.nsx.management.policy.providers.security.provider.IdfwIdsToggleProviderNsxT.handlePolicyChange(IdfwIdsToggleProviderNsxT.java:137) ~[?
:?]
        at com.vmware.nsx.management.policy.providers.security.provider.IdfwIdsToggleProviderNsxT.pushPolicyIntent(IdfwIdsToggleProviderNsxT.java:557) ~[?:?
]
        at com.vmware.nsx.management.policy.policyframework.engine.ProviderInvocationTask.innerRun(ProviderInvocationTask.java:322) ~[?:?]
        at com.vmware.nsx.management.policy.policyframework.engine.ProviderInvocationTask.run(ProviderInvocationTask.java:156) ~[?:?]
        at com.vmware.nsx.management.common.executor.TaskExecutorImpl$TaskWrapper$1.run(TaskExecutorImpl.java:240) ~[?:?]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_382]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_382]
        at com.vmware.nsx.management.common.executor.TaskExecutorImpl$TaskWrapper.run(TaskExecutorImpl.java:273) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_382]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_382]
        at com.vmware.nsx.util.concurrent.Executors$MeteredRunnable.run(Executors.java:353) ~[nsx-util.jar:?]
        at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_382]

Environment

VMware NSX - All Versions

Cause

Identity Based Firewall (IDFW) was enabled for a vSphere Cluster and the cluster was deleted before disabling the IDFW for that cluster.  This left a stale entry in the database.

Resolution

Run the following API command to find the clusters configured for IDFW and find the UUID:Cluster ID specified in the error.
```
GET /policy/api/v1/infra/settings/firewall/idfw/cluster 
```
Find the UUID:Cluster ID string specified in the error. e.g. ########-####-####-####-############:domain-c1234
Find the Cluster Name is specified in the "display_name" e.g. Node Cluster 12-80
Check vCenter to see if this cluster exists
If the the Cluster exists then
- Go to NSX Manager UI Security > Distributed Firewall > Settings > Identity Firewall Settings
- Toggle the switch on the cluster to enable and disable the cluster again.
- Confirm message is gone.
- If the message re-occurs, then try the DELETE API command below

If the Cluster no longer exists then use the DELETE API command to remove the entry.

DELETE https://localhost/policy/api/v1/infra/settings/firewall/idfw/cluster/########-####-####-####-############:domain-c1234

Example of running the API from the NSX Manager command prompt (engineering mode)

# Run the GET API to find the UUID:Cluster ID in the message

root@nsx-421-01:~#   curl -k -u admin -H "Content-Type:application/json" -X GET https://localhost/policy/api/v1/infra/settings/firewall/idfw/cluster
Enter host password for user 'admin':
{
  "results" : [ {
       "cluster_idfw_enabled" : true,
    "member" : {
      "target_id" : "########-####-####-####-############:domain-c1234",<--------------- UUID:Cluster ID from the error message.
      "target_type" : "Node Cluster 12-80"
    },
    "resource_type" : "ComputeClusterIdfwConfiguration",
    "id" : "0########-####-####-####-############:domain-c1234",
    "display_name" : "Node Cluster 12-80",   <--------------- cluster name in vCenter
    "path" : "/infra/settings/firewall/idfw/cluster/########-####-####-####-############:domain-c1234",
    "relative_path" : "########-####-####-####-############:domain-c1234",
    "parent_path" : "/infra",
    "remote_path" : "",
    "unique_id" : "########-####-####-####-###########",
    "realization_id" : "6########-####-####-####-###########",
    "owner_id" : "########-####-####-####-###########",
    "marked_for_delete" : false,
    "overridden" : false,
    "_system_owned" : false,
    "_protection" : "NOT_PROTECTED",
    "_create_time" : 1759357627566,
    "_create_user" : "admin",
    "_last_modified_time" : 1759357627566,
    "_last_modified_user" : "admin",
    "_revision" : 0
  }, {
  .
  .
  .
  } ],
  "result_count" : 3,
  "sort_by" : "display_name",
  "sort_ascending" : true



# Run the DELETE API to remove the record.   If the cluster exists in the UI, you'll just see the Toggle move to the Off position.

curl -k -u admin -H "Content-Type:application/json" -X DELETE https://localhost/policy/api/v1/infra/settings/firewall/idfw/cluster/########-####-####-####-############:domain-c1234
Enter host password for user 'admin':

NOTE: There is no output for the DELETE if it suceeds

See the API Guide for more information on the API call. https://developer.broadcom.com/xapis/nsx-t-data-center-rest-api/latest/method_ListComputeClusterIdfwConfiguration.html