Firewall rules no longer applied in NSX-T after exceeding 128 space limit for ASG's in Tanzu NCP

Products

• VMware Tanzu Platform
• VMware NSX-T Advanced for VMware Tanzu Application Service Term License (50 pack Application Instance)

Issue/Introduction

NSX Container Plugin (NCP) for Tanzu Platform has a limit on number of spaces that can be bound to an application security group (ASG). 

This is mentioned in release notes, see:

Issue 2416376: NCP fails to process a TAS ASG (App Security Group) that binds to more than 128 Spaces

Because of a limit in NSX-T distributed firewall, NCP cannot process a TAS ASG that binds to more than 128 Spaces.

This issue can potentially cause ASG's to not passed to firewall rules in NSX manager. If you depend on ASG rules for traffic to be accepted from container network then this can potentially impact application container connectivity.

Reference KB: Application Security Groups (ASG) and Distributed Firewall Rules (DFW) Integration

Exceeding this limit is one reason ASG rules in CF may not get set in NSX ResourceType FirewallSection and binded assigned_to spaces.

Diego Database (BBS) NCP logs will show errror:

ncp.stdout.log:2025-03-29T02:00:33.507Z ######-####-####-####-######### NSX ##### - [nsx@#### comp="nsx-container-ncp" subcomp="ncp" level="WARNING"] vmware_nsxlib.v3.client The HTTP request returned error code 400, whereas 201/200 response codes were expected. Response body {'details': 'Field level validation errors: {rules[0].applied_tos has exceeded maximum size 128}, {rules[1].applied_tos has exceeded maximum size 128}', 'httpStatus': 'BAD_REQUEST', 'error_code': 255, 'module_name': 'common-services', 'error_message': 'Field level validation errors: {rules[0].applied_tos has exceeded maximum size 128}, {rules[1].applied_tos has exceeded maximum size 128}'}

Environment

This issue specifically applies to Tanzu Cloud Foundry foundations using the NSX Container Plugin.

Resolution

If you are over 128 limit of bound spaces then it will be necessary to unbind some spaces to get under the limit. As mentioned in the release notes:

Workaround: Create multiple ASGs and bind each of them to no more than 128 Spaces

Perform the steps:

1. List out your security groups:

   $ cf security-groups

2. View each security group and verify number of spaces applied to:

   $ cf security-group SECURITY_GROUP

3. Unbind spaces in excess of 128 if an ASG is over the limit:

   $ cf unbind-security-group SECURITY_GROUP ORG SPACE

4. Create a new duplicate ASG rule and bind excess spaces that were previously unbound:

   $ cf create-security-group SECURITY_GROUP PATH_TO_JSON_RULES_FILE
   
   $ cf bind-security-group SECURITY_GROUP ORG --space SPACE

Engineering is considering a feature to impose this as a hard limit on Cloud Controller API (CF CLI).