Displaying Group members is slow in NSX-T UI
search cancel

Displaying Group members is slow in NSX-T UI

book

Article ID: 387140

calendar_today

Updated On:

Products

VMware vDefend Firewall

Issue/Introduction

  • You observe that it takes longer than expected to view the members of a group in the inventory section of the NSX-T UI.
  • You observe in the log file /var/log/proton/localhost_access.log on the NSX-T Manager that similar APIs take some time to process.
    • Note in the highlighted examples below the APIs take 60 seconds to process.

2024-12-31T11:13:08.423Z - "GET /nsxapi/api/v1/infra/domains/default/groups/Test_Group/members/virtual-machines?page_size=50&sort_ascending=true&sort_by=display_name HTTP/1.1" 200 2378 60157 60157 +
2024-12-31T11:13:08.424Z - "GET /nsxapi/api/v1/infra/domains/default/groups/TestGroup/members-count?page_size=50&member_types=VirtualMachine,IPAddress,Segment,SegmentPort,DVPG,DVPort,VirtualNetworkInterface,PhysicalServer,TransportNode&fetch_consolidated_ips=false HTTP/1.1" 200 556 60161 60161 +
2024-12-31T11:17:31.351Z - "GET /nsxapi/api/v1/infra/domains/default/groups/TestGroup/members/virtual-machines?page_size=50&sort_ascending=true&sort_by=display_name HTTP/1.1" 200 2378 60142 60142 +2024-12-31T11:17:31.354Z - "GET /nsxapi/api/v1/infra/domains/default/groups/TestGroup/members-count?page_size=50&member_types=VirtualMachine,IPAddress,Segment,SegmentPort,DVPG,DVPort,VirtualNetworkInterface,PhysicalServer,TransportNode&fetch_consolidated_ips=false HTTP/1.1" 200 556 60156 60154 

  • If you run similar APIs against the NSX Manager you observe the group members are returned without delay.

GET /api/v1/ns-groups/<groupID>/effective-logical-port-members

  • You had AVI load balancer from your environment but has been removed.
    • If you check the below APIs there is no AVI configuration left.

GET /policy/api/v1/alb/controller-nodes/deployments

GET /policy/api/v1/alb/controller-nodes/cluster

GET /policy/api/v1/alb/controller-nodes/clusterconfig

  • You will observe similar logging as seen below in var/log/syslog.log on the NSX-T Manager.


2024-12-31T10:16:31.220Z INFO http-nio-127.0.0.1-7440-exec-6656 AlbControllerRestUtils 5375 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="125408ba-d3c3-4d29-9d6b-efabd2c03f15" subcomp="manager" username="Admin"] [ALB Controller] Inside authentication session for nsxt-alb

2024-12-31T10:16:31.220Z INFO http-nio-127.0.0.1-7440-exec-6656 AlbControllerRestUtils 5375 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="125408ba-d3c3-4d29-9d6b-efabd2c03f15" subcomp="manager" username="Admin"] [ALB Controller] Building http client for rest handler

2024-12-31T10:16:31.220Z INFO http-nio-127.0.0.1-7440-exec-6656 AlbControllerRestUtils 5375 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="125408ba-d3c3-4d29-9d6b-efabd2c03f15" subcomp="manager" username="Admin"] [ALB Controller] BuildHttpClient completed

2024-12-31T10:17:31.342Z WARN http-nio-127.0.0.1-7440-exec-6656 AviRestClient 5375 POLICY [nsx@6876 comp="nsx-manager" level="WARNING" reqId="125408ba-d3c3-4d29-9d6b-efabd2c03f15" subcomp="manager" username="Admin"] REST API failed: /api/systemconfiguration/?include_name I/O error on GET request for "https://<AVI-IP>/api/systemconfiguration/": Connect to <AVI-IP>:443 [/<AVI-IP>] failed: connect timed out; nested exception is org.apache.http.conn.ConnectTimeoutException: Connect to <AVI-IP>:443 [/<AVI-IP>] failed: connect timed out


2024-12-31T10:17:31.342Z INFO http-nio-127.0.0.1-7440-exec-6656 AlbControllerRestUtils 5375 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="125408ba-d3c3-4d29-9d6b-efabd2c03f15" subcomp="manager" username="Admin"] [ALB Controller] Inside retry handler

2024-12-31T10:17:31.342Z INFO http-nio-127.0.0.1-7440-exec-6656 EnforcementPointUtils 5375 POLICY [nsx@6876 comp="nsx-manager" level="INFO" reqId="125408ba-d3c3-4d29-9d6b-efabd2c03f15" subcomp="manager" username="Admin"] No enforcement point path provided in the URL. Using default enforcement point: /infra/sites/default/enforcement-points/default

NOTE: The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on your environment

Environment

VMware NSX-T Data Center 3.X

VMware NSX-T Data Center 4.X

Cause

  • The delay in viewing group memberships is a result of stale entries of ALB enforcement points existing in the default domain endpoint.
    • This will happen if the AVI load balancer was removed and not cleaned up correctly in NSX-T.

Resolution

  • The stale enforcement points need to be idenfied using the below API:

GET /policy/api/v1/infra/sites/default/enforcement-points/alb-endpoint

  • Once the stale enforcement points have been identified they can be removed using the below API:

DELETE policy/api/v1/infra/sites/default/enforcement-points/alb-endpoint

Additional Information

 

Powered by Wolken Software