Proton restarts or NSX Exclusion list modification fails due to invalid group with Ipset/MACAddress in FW Exclusion List (Error code:514051)"
search cancel

Proton restarts or NSX Exclusion list modification fails due to invalid group with Ipset/MACAddress in FW Exclusion List (Error code:514051)"

book

Article ID: 378422

calendar_today

Updated On:

Products

VMware vDefend Firewall VMware NSX

Issue/Introduction

You are using VMware NSX with the distributed firewall (DFW).

When a user or a service attempts to modify the DFW the follow error is reported:

 Error occurred: Invalid group with IPSET/MACAddress in Exclusionlist path=[/infra/domains/default/groups/XXXXXXX]

See below example where this error can impact the operation of NSX:

  1. Making configuration changes to the Exclusion List manually via the UI fails due to the unsupported group.
  2. Deploying / redeploying Service Insertion fails as the NSX manager is unable to add the SVM to the exclusion list.
  3. The vCenter vpxd.log reports "InsufficientAgentVmsDeployed"  for the ESXi where the deployment is failing.
  4. The Proton service on the NSX manager repeatedly restarted resulting the the NSX cluster being in a DEGRADED state.

Logs findings:

  • The nsxapi.log shows NSX management plane starting in every 5- 6 minutes.
2024-10-07T18:39:50.257Z  INFO WrapperStartStopAppMain ApplicationContextManager 227639 - [nsx@6876 comp="nsx-manager" level="INFO" starting NSX management plane application context.
2024-10-07T18:45:43.257Z  INFO WrapperStartStopAppMain ApplicationContextManager 233174 - [nsx@6876 comp="nsx-manager" level="INFO" starting NSX management plane application context.
2024-10-07T18:51:44.257Z  INFO WrapperStartStopAppMain ApplicationContextManager 240292 - [nsx@6876 comp="nsx-manager" level="INFO" starting NSX management plane application context.
2024-10-07T18:58:20.257Z  INFO WrapperStartStopAppMain ApplicationContextManager 251264 - [nsx@6876 comp="nsx-manager" level="INFO" starting NSX management plane application context.
  • The proton-tomcat-wrapper.log shows the JVM received a kill signal and restarting. 
STATUS | wrapper  | 2024/10/07 18:39:51| JVM received a signal SIGKILL (9).
STATUS | wrapper  | 2024/10/07 18:45:44| JVM received a signal SIGKILL (9).
STATUS | wrapper  | 2024/10/07 18:52:45| JVM received a signal SIGKILL (9).
STATUS | wrapper  | 2024/10/07 18:59:21| JVM received a signal SIGKILL (9).
  • The Proton service repeatedly restarting in /var/log/proton/proton-tomcat-wrapper.log:
STATUS | wrapper  | 2024/10/07 18:28:31 | Launching a JVM...
STATUS | wrapper  | 2024/10/07 18:34:16 | Launching a JVM...
STATUS | wrapper  | 2024/10/07 18:40:39 | Launching a JVM...
STATUS | wrapper  | 2024/10/07 18:46:31 | Launching a JVM...
STATUS | wrapper  | 2024/10/07 18:53:32 | Launching a JVM...
  • The following exception is seen repeatedly on the NSX manager's in /var/log/proton/proton-tomcat-wrapper.log
INFO   | jvm 784  | 2024/10/07 18:33:30 | com.vmware.nsx.management.common.exceptions.InvalidArgumentException: Invalid group with IPSet/MACAddress in ExclusionList path=[/infra/domains/default/groups/<GroupID>]
INFO   | jvm 785  | 2024/10/07 18:39:51 | com.vmware.nsx.management.common.exceptions.InvalidArgumentException: Invalid group with IPSet/MACAddress in ExclusionList path=[/infra/domains/default/groups/<GroupID>]
INFO   | jvm 786  | 2024/10/07 18:45:44 | com.vmware.nsx.management.common.exceptions.InvalidArgumentException: Invalid group with IPSet/MACAddress in ExclusionList path=[/infra/domains/default/groups/<GroupID>]
INFO   | jvm 787  | 2024/10/07 18:52:45 | com.vmware.nsx.management.common.exceptions.InvalidArgumentException: Invalid group with IPSet/MACAddress in ExclusionList path=[/infra/domains/default/groups/<GroupID>]
INFO   | jvm 788  | 2024/10/07 18:59:21 | com.vmware.nsx.management.common.exceptions.InvalidArgumentException: Invalid group with IPSet/MACAddress in ExclusionList path=[/infra/domains/default/groups/<GroupID>]

Checking the DFW exclusion list in the UI, the unsupported group illustrated in the error is not visible in the exclusion list.

Using API call lists the unsupported group(s) is in the exclusion list: GET /policy/api/v1/infra/settings/firewall/security/exclude-list

NOTE: The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on your environment.

Environment

VMware NSX-T Data Center 3.0.2 or later

VMware NSX 4.x

VMware vDefend Firewall

Cause

The NSX manager is unable to edit the exclusion list as the exclusion list currently includes unsupported groups.

Groups with IP Sets, IP Addresses, MAC addresses as members are not supported in exclusion list as per this page of the administration guide: https://techdocs.broadcom.com/us/en/vmware-cis/nsx/vmware-nsx/4-0/administration-guide/security/distributed-firewall/manage-a-firewall-exclusion-list.html

Before NSX 4.X groups with unsupported objects (i.e. IP Sets, IP addresses, MAC addresses, etc) are allowed to be added to the exclusion list. In NSX 4.2, a change was made to force validation of exclusion list members. The validation errors in the exclusion list must be rectified before changes can be made.

Resolution

Compare the members of the Exclusion List in UI and  using the GET API /policy/api/v1/infra/settings/firewall/security/exclude-list. The additional members listed in the API call are groups with unsupported objects.

There are 2 resolutions to the above issue:

  1. Remove the unsupported group from the exclusions list.
  2. Remove the unsupported configuration from the group used in the exclusion list.

See below for steps to accomplish the above.

Option 1: Remove the groups in #1 from the exclusion list through rest API PATCH /policy/api/v1/infra/settings/firewall/security/exclude-list. Below is a sample of the body:

{
    "members": [
     //Comma-separated list of valid group paths
    ]
}

See below example: API GET /policy/api/v1/infra/settings/firewall/security/exclude-list reports member NSX_Group_A , NSX_Group_B, NSX_Group_C and NSX_Group_D.

{
    "members": [
        "/infra/domains/default/groups/NSX_Group_A",
        "/infra/domains/default/groups/NSX_Group_B",
        "/infra/domains/default/groups/NSX_Group_C",
        "/infra/domains/default/groups/NSX_Group_D"
    ]

}

To remove a group, remove it from the API PATCH /policy/api/v1/infra/settings/firewall/security/exclude-list body:

{
    "members": [
        "/infra/domains/default/groups/NSX_Group_A",
        "/infra/domains/default/groups/NSX_Group_B",
        "/infra/domains/default/groups/NSX_Group_D"
    ]

}

Option 2:

  • API method:

Use the DELETE API /policy/api/v1/infra/domains/{domain-id}/groups/{group-id}/ip-address-expressions/{expression-id} to delete the IPAddressExpression objects from the group:

Obtain the ip-address-expressions using the GET API call: 

/policy/api/v1/infra/domains/{domain-id}/groups/{group-id}/ 

Example: GET https://<nsx_manager_IP/FQDN>/policy/api/v1/infra/domains/default/groups/NSX_Group_E

{
    "expression": [
        {
            "ip_addresses": [
                "192.168.10.10"
            ],
            "resource_type": "IPAddressExpression",
            "id": "3897bb6f-9f59-4042-93a2-9ef48206ed86",
            "path": "/infra/domains/default/groups/NSX_Group_E/ip-address-expressions/3897bb6f-9f59-4042-93a2-9ef48206ed86", <<<<<<<<< This is a list of the expressions to delete
            "relative_path": "3897bb6f-9f59-4042-93a2-9ef48206ed86",
            "parent_path": "/infra/domains/default/groups/NSX_Group_E",
            "remote_path": "",
            "marked_for_delete": false,
            "overridden": false,
            "_protection": "NOT_PROTECTED"
        }

Use the DELETE API to remove to expression(s):

Example: DELETE https://<nsx_manager_IP/FQDN>/policy/api/v1/infra/domains/default/groups/NSX_Group_E/ip-address-expressions/3897bb6f-9f59-4042-93a2-9ef48206ed86   <<<<<<<< The above expression

  • UI Method:
  1. Navigate to Inventory > Group > Edit > Members.
  2. Remove the unsupported configuration like IP addresses, MAC Address etc.
  3. Save the configuration.

Note: You can also check "show Deleted Entities" and hit remove all.

Powered by Wolken Software